How do I BLOCK search engines? · Blog

Well, there are several options if you're in the boat that you want to block Search Engines from crawling your forums all together.

A good "first step" is to disable SE Friendly URLs, myself I don't like doing this, it's much easier on the eyes and the urls look ugly.

Please note however that this will NOT block search engines, it'll simply make it more difficult for them to crawl your content; so it's not an end-all be-all suggestion.

If the Search Engine respects robots.txt standards, it's even easier (most non-abusive search engines support this), just edit (or add) your robots.txt file (in your web root) and add the following:
User-agent: *
Disallow: /forum

Be sure however to update /forum with whatever your forum's location is (from your domain's web root); so if your forum is in you'd enter "/forums" (no trailing slash).
Posted By Gremelin Posted on June 22nd, 2007


( Posted)
Request a Quote
Please feel free to Request a Quote at any time.

By requesting a quote you allow us to adjust pricing for your product on a per-project basis. Note that most of the prices on our pricing page are just our "common" pricing and can vary on a per project basis. It will also allow us to add any per project discounts where we see fit (non-profit organizations, affiliations, etc).

If your "type" of project isn't listed in our menu's to the left, please Request a Quote; we can do much more than what is listed however building pages for it all would take a great while.
Financing Available
We utilize StableHost for the hosting of our all of our VPS'. They're also #1 host that we recommend to every one of our clients, friends, and family. For more Web Hosting recommendations, please see our Recommended Hosts page!

StableHost Web Hosting

For EMail hosting we recommend G Suite (Formerly Google Apps and Google for Business).
Buy Us a Coffee