How Google Index Your Website
How Google Index Your Website
First we are discussing about a “bot” is a piece of software from a search engine that is built to go through every page of your site, included at particular category, and stores into a database.
Google has three well known bots: “The Ad-sense Bot”, “the Fresh Bot” and “the Deep Crawl “.
The Ad-sense Bot is used for publishers who have Ad sense on their sites. As soon as a new page is created, the JavaScript within the Ad-sense code sends a message to the
Ad-sense bot, and it will come within 15 minutes to index the page so that it can serve up the most relevant ads.
But, we are only mainly concerned about the Deep Crawl and the Fresh bot.
The Fresh bot crawls the most popular pages on your website. It does not a matter if that is one page or hundreds or thousands. Sites like Amazon.com , BBC.com and CNN.com have pages that are crawled every ten minutes, since Google has learned that those pages have that amount of frequent changes. A typical site should expect to have a fresh bot visit every weekend days, depending on how popularize those pages are.
What happens to your site on a Fresh bot visit is that it finds all of the deeper links in your site. It places those links into a database so that when the Deep Crawl occurs, it has a reference. Once a month, the Deep Crawl bot visits your site and goes over all the links found by the Fresh bot. This is the reason why it can take up to a month for your entire site to be indexed in Google – even with the addition of a Google Sitemap.Once you are google sitemap created successfully for your website. Google sitemap also useful to get indexed all pages of Your website from google search engine.
So, be patient and keep on adding content to your site, and work on getting valuable in-bound links to your site – Google will reward you for it.
Last year Google set up the sitemaps program to help webmasters make it easier for their site’s pages to be found by the Google spider. To participate, you have to construct an XML file with information about each of your site’s pages and upload this to the main directory of your website. XML files are the same kinds of files used in RSS syndication, but in this case the XML file is meant only to provide information for the Google robot.
If you don’t know how to make an XML file or you have a very large website with hundreds of pages it is best to use an XML sitemap generator. A free, online Google Sitemaps generator can be found at http://www.auditmypc.com/free-sitemap-generator.html
Once you have made your XML sitemap and have uploaded it to your server, go to www.google.com/webmasters/sitemaps and set up a sitemaps account with Google. When you log into your account you will see a tab at the top of the page that says Add. Click it and then check the box, add “General Web Sitemap”. You will be asked to put the URL of your sitemap. Normally the URL of your sitemap will be http://www.yoursite.com/sitemap.xml
The Google sitemaps program is particularly important if you have a large, data base driven site or eCommerce website with many pages that you want to be available in Google searches.
If you want to increase traffic to your site, check and see how many of your pages have actually been included in the various search engines, and then take steps to make sure that all of them are included in future search engine updates. Your efforts to improve the search engine friendliness of your site will be well rewarded.
