Press "Enter" to skip to content

Read Your Search Engine Robot Report

The Search Engine Robot Report is extremely valuable yet seems to be the least queried Web Analytics report (available in many but not all applications).
The Search Engine Robot Report tells you how often the Search Engines query your web pages, and which of your web pages it finds.
The example above illustrates the frequency of visits to unique pages (rows) from the (arguably) top four search engines; Google, Yahoo, MSN and
This report is important for sales managers, marketers and resources responsible for ensuring best practices in your company’s SEO(Search Engine Optimization)strategy.
The first step in your SEO strategy is to assure the search engines ‘crawl’ through all of the pages you want them to see. (To eliminate the search engines from reading private pages, make sure to have an updated robots.txt file for each of your sites. See “”: for more on the topic.

As your pages are modified to attracted the Search Engine s more effectively, you can measure your optimization via these reports.
Additional views can include, days since the robots last visit, number of visits, and the actual date of each robots last visit.
Along with the standard set of best practices, updating your content regularly is a great way to increase the return visits from the Search Engine Robots and therefore increase your natural rankings.

One Comment

  1. Frank Michlick
    Frank Michlick September 7, 2006

    The most valuable statistics in this context are actually often in the referrer stats: Which search engines bring me traffic, which query terms did the user use, and where on my site did the user end up.

Comments are closed.