Tag Archives: spiders

Keeping out the spiders

I spend most of my time telling people how to best allow spiders into a website, however it can sometimes be just as important knowing how to keep them out. Why would you want to want your content kept out of the search results? There could be a number of reasons: Unwanted duplication – if [...]