User Agent String
- Tool from ASAP Consulting s.r.o. for detailed user agent string analysis using an online form. Includes databases of browsers and robots.
- http://user-agent-string.info/

User-Agents.org
- Large list of search engine spiders, similar web robots, and Web browsers: their web-log identification and links to their originators.
- http://www.user-agents.org/

Robotstxt.org
- Information on the robots.txt Robots Exclusion Standard and other articles about writing well-behaved Web robots.
- http://www.robotstxt.org/

Bots vs Browsers
- This large database lists user agents in categories and distinguishes between robots and browsers.
- http://www.botsvsbrowsers.com

Robot IP Address
- Brian Dunnintg provides a list of all the major search engine robot IP addresses, by full class C only.
- http://www.briandunning.com/seo/

Spider Track
- Tracks and displays real time hits and IP addresses of MSNbot, Yahoo's Slurp, and Googlebot. Provides aggregate hits of each spider over the last 30 days.
- http://www.spidertrack.org

HTTP User Agent Index
- An alphabetical list of user agents and the deployer behind them, compiled by Christoph Rüegg.
- http://www.siteware.ch/webresources/useragents/db.html

Search Engine IP Addresses
- Lists IP addresses of search engine spiders. Can be searched by IP address. Also links to resources on spiders.
- http://www.iplists.com/

All About Search Indexing Robots and Spiders
- Search Tools Consulting explains how the search engine programs called "robots" or "spiders" work, and reviews related sites.
- http://www.searchtools.com/robots/

List of Robot Agent Strings
- A list from PGTS of Web robots with the identifying data they leave in Web site logs.
- http://www.pgts.com.au/pgtsj/pgtsj0208d.html

Search Engine Robots and Other User Agents
- John A. Fotheringham presents data in tabular form on the robots sent by search engines and other sites to read and index Web pages: their origins, names and IP addresses.
- http://www.jafsoft.com/searchengines/webbots.html