making your site web bot-friendly

[Randy5]

Any tips for making your website friendly to Internet bots? Is this different from SEO?

[suwunk]

Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled. Make sure it's current for your site so that you don't accidentally block the Googlebot crawler. Visit Frequently asked questions - Controlling Crawling and Indexing - Google Code (http://code.google.com/web/controlcrawlindex/docs/faq.html) to learn how to instruct robots when they visit your site. You can test your robots.txt file to make sure you're using it correctly with the robots.txt analysis tool available in Google Webmaster Tools.
Webmaster Guidelines - Webmaster Tools Help (http://support.google.com/webmasters/bin/answer.py?hl=en&answer=35769)