crawler
noun /ˈkrɔːlə(r)/
/ˈkrɔːlər/
(informal)- (British English, disapproving) a person who tries to get somebody’s favour by praising them, doing what will please them, etc.
- a thing or person that crawls, such as a vehicle, an insect or a baby
- (computing) a program that searches the World Wide Web in order to record and list data
- The more content you've got, the more easily search engine crawlers can find you.
- Search engines like Google depend on web crawlers to index the internet.
see also kerb-crawler