How do I BLOCK search engines?

Well, there are several options if you're in the boat that you want to block Search Engines from crawling your forums all together.

A good "first step" is to disable SE Friendly URLs, myself I don't like doing this, it's much easier on the eyes and the urls look ugly.

Please note however that this will NOT block search engines, it'll simply make it more difficult for them to crawl your content; so it's not an end-all be-all suggestion.

If the Search Engine respects robots.txt standards, it's even easier (most non-abusive search engines support this), just edit (or add) your robots.txt file (in your web root) and add the following:
User-agent: *
Disallow: /forum

Be sure however to update /forum with whatever your forum's location is (from your domain's web root); so if your forum is in you'd enter "/forums" (no trailing slash).
Posted on June 22nd, 2007


( Posted)
For the best viewing experience please update your browser to Chrome, Firefox, or Opera