Tag Archives: robots.txt

Pressing the wrong international buttons

A Tale of Edinburgh University Press and search engines This weekend I was doing some research for additional content for my Scottish Books site and had occasion to do a Google search for Edinburgh University Press. To my surprise their site didn’t appear in the first page of results, or the second, or the third. [...]

Keeping out the spiders

I spend most of my time telling people how to best allow spiders into a website, however it can sometimes be just as important knowing how to keep them out. Why would you want to want your content kept out of the search results? There could be a number of reasons: Unwanted duplication – if [...]