Whenever you think of a GoogleBot, your mind suggests it move into all the corners of Google World and gather the knowledge, where some are yet to be revealed. For People, it is hard to believe that GoogleBot is still a computer program that crawls around all the webs and adds pages to index.  

 google bot

Bot? Crawler? Spider? What are these?  

End of the day, all the terms are meant to be same – It is a bot that crawls the complete web. GoogleBot will crawl the web through hyperlinks. It will analyze the new, and updated content and suggests what to be added to the index and what not to be added. The Index is a Google Brain, to make it more simple, it is like an index of the book, where index is used to find the complete knowledge on the particular topic. The Big G uses tons of computers to forward their crawlers to every part and pieces of the web to find pages in the web and what it actually contains. Like Google has GoogleBot as their crawler or robot, other search engines have their own bot. 


How does it actually works? 

GoogleBot will have a certain set of procedures to be followed one by one. It uses sitemaps and databases of links indexed during last crawls to find where the bot should move next.  

if GoogleBot goes through any new changes in the links or broken links it will be updated in the index. Your site has to be in crawlability phase, so the arrival of bots can happen often.  

 google bot articles

Types of Robots: 

There are different types of robots. For instance, the AdsBot check ad quality and AdSense, while Mobile Apps Android checks Android apps. We consider following are most important ones to be known: 

Name                               User-agent 

Googlebot (desktop)    Mozilla/5.0 (compatible; Googlebot/2.1;  


Googlebot (mobile)    Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X  

Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko)  Chrome/41.0.2272.96 Googlebot/2.1;  


Googlebot Video       Googlebot-Video/1.0 

Googlebot ImagesGooglebot-Image/1.0 

Googlebot News          Googlebot-News 

How GoogleBot Crawls your website: 

Google does not share arrangements of IP address to that the different Googlebots use, since these addresses changes regularly. To see whether a genuine Googlebot visits your site, you can complete a turn around IP query. Spammer or fakers can without much of a stretch satire a client specialist name, however not an IP address.  

 You can use robots.txt to find how GoogleBot visits all parts of your website. Even there you can suggest, GoogleBot not to index that coming all together. This will make the mentioned links out of index.