How do I BLOCK search engines?

Well, there are several options if you're in the boat that you want to block Search Engines from crawling your forums all together.

A good "first step" is to disable SE Friendly URLs, myself I don't like doing this, it's much easier on the eyes and the urls look ugly.

Please note however that this will NOT block search engines, it'll simply make it more difficult for them to crawl your content; so it's not an end-all be-all suggestion.

If the Search Engine respects robots.txt standards, it's even easier (most non-abusive search engines support this), just edit (or add) your robots.txt file (in your web root) and add the following:
Code
User-agent: *
Disallow: /forum


Be sure however to update /forum with whatever your forum's location is (from your domain's web root); so if your forum is in http://www.ubbcentral.com/forums/ you'd enter "/forums" (no trailing slash).
Posted on June 22nd, 2007

Comments

( Posted)
For the best viewing experience please update your browser to Chrome, Firefox, or Opera