More Google site command oddities – and Bing too

A few weeks ago I mentioned I was seeing more and more erratic behaviour in the once-reliable Google site command. Today I noticed more strangeness on this very site. In March I had 163 pages indexed, at the beginning of April I had dropped to 138. Now I’ve dropped again to 113. But the really weird thing is that it’s not any of the blog pages that have vanished, which might be at least understandable; it’s some of the main site static pages which are missing.

Checking the internal links in Webmaster Tools shows they’re still recorded – although as has been the case for some time the number of internal links being shown is wildly wrong.

I’d be inclined normally to discount this as unimportant but I’m also seeing odd yoyoing effects in my rankings which are attributable to them occasionally “losing” the natural page for a particular query and using another page which doesn’t deserve to rank as much. Previously I’d have put this down to a page dropping into the supplemental index temporarily – now I’m wondering if they’re losing it altogether.

And Bing Too

Bing’s site command has been very strange almost since they took over from MSN. All of a sudden on this site they’re showing 102 indexed pages in the World results and when you switch to UK-only results they only show 36 !!

And people wonder why SEO’s are paranoid ;-)

Personalised and real-time search – let me turn them off!

Call me an old Luddite if you will, but to me – as a user rather than just as an SEO – personalised search is a faulty concept. The whole point of “search” is that I’m looking for something I usually don’t know much about – so why would I want sites that I already know about appearing in search results? If they had what I wanted I wouldn’t need to do a search! Personalisation only really makes sense if you use search engines merely as a navigation tool rather than a search tool. Ok we know some people do that but are we really restricting¬† the capability and functionality of something as important as search on the habits of people who can’t be bothered to learn to use bookmarks?

As regards real-time search I only see the point in it for news-based queries – otherwise being new doesn’t make something any more relevant or carry any quality signal, so why focus on it? And it seems I’m not the only one to think so according to a new study summarised here.

The appearance of Twitter or Facebook entries in search results is completely irrelevant for the sort of searches I make, and most of the time so is video. I’d like to be able to turn them off and only include them on the rare occasions when I think they are relevant. I already use a Firefox add-on that kills Adwords for my personal searches since I *never* use PPC ads and they’ve become progressively more annoying. Sadly Google seem determined to show me what they think I should see and not what I want to see, and they resist all suggestions to allow us flexibility. They insist on devoting all their visible efforts to these increasingly complex layouts and features that push real organic results further down the page, yet the quality of the organic results is probably worse now than it was 5 years ago and they seem to be unable to filter out obvious spam of a type that they claim to hate and we ethical SEO’s have been railing against for years.

The end result is a low quality experience for many users. I would say to them to concentrate on quality and simplicity – which was what made them popular in the first place – and give us a choice of what we want to see in the results rather than foist unwanted “features” on us. Because if they don’t then they really are risking losing users to someone else that will give us what we want. The online world is littered with once-popular sites and companies that have died the death, and no-one, not even Google, is immune from changing public preferences.

Ethical internet marketing in an unethical world

I’ve never been a great fan of television but this last week or so I’ve been down with flu and often unable to sleep for coughing, but when I wasn’t in bed I had very little concentration to do any work that I felt I could rely on, so had to resort to the dreaded tube to while away the hours while my body fights off the virus. So I’ve been seeing rather more adverts than I’d like.

Now being an SEO means I’m involved in marketing, and that’s something that didn’t come easily to me. In fact I once, many years ago, defined marketing as persuading people to buy things they don’t need by lying to them. That was back in my book trade days with James Thin when one of my colleagues was doing a marketing course. One of the reasons I enjoyed working in the book trade was that we basically took an ethical approach to selling books. We kept as wide a range of quality books as we were able to sell profitably, and we mainly sold to people who loved books and wanted the knowledge and literature they contained. (Maybe that’s why the old book trade has largely been wiped out by more business-savy competitors who were prepared to play dirty!)

Gradually I wised up and came to realise that marketing is at least a necessary evil if you’re going to stay in business, but in my subsequent move into website design, search engine optimisation and ecommerce I’ve always tried to stick to honest businesses and quality approaches to advertising, in the same way as I try to write quality content rather than regurgitated rubbish. But the problem is the world doesn’t always work the same way.

Those TV adverts I mentioned I’ve been seeing; I’m fine with the ones trying to get you to come to their store by highlighting cheap prices, even if they aren’t always typical. I’m ok with the ones selling dreams – “your life will be far better if you only had a …..”, even if they don’t always seem very realistic, caveat emptor and all that. I start to have problems with banks who advertise great interest rates which turn out to be fleeting or misleading, especially when we just bailed them out with money the government will claw back from us. But the one’s I really dislike are at another level entirely. What are those? Well for example there’s all those companies trying to part people from their gold jewellery at knock-down prices, who thankfully are reportedly being investigated (though it doesn’t seem to be discouraging the ads). Now there seems to be a spate of “need money quickly” ads with staggeringly high interest rates – 2356% APR on one I saw. (Isn’t that kinda like the rates that illegal moneylenders charge?)

TV ads like those make me ashamed to be human let alone be in a marketing-related business. The people who fall for them may be guillible but they might also be desperate, and I see no justification in taking advantage of them. We’re supposed to be a bit more evolved than that.

I sometimes wonder if the internet is evolving in the wrong direction. It started very ethical with lots of optimistic, almost utopian visions, but it’s become progressively more commercialised and with that has come the seedier side of human nature. I seem to remember a promising internet company that had a motto. What was it again? Oh yeah – “Don’t be Evil”. For some time everyone wondered how they would make money – all they seemed to want to do was provide good quality search results. Then they invented Adwords and money started pouring in. Unfortunately they also went and invented Adsense, and that generated loads of spammy low-value sites built purely for rankings by Black-hat SEOs to generate money-making clicks. And to combat those leech-like sites more people started more trickery and more low-value, often autogenerated or scraped content sites.

I could build loads of sites like that and make money, but that’s not my way. I want the internet to be better than that and I care more about being able to look at myself in the mirror than being rich.

Flu-fuelled thoughts aren’t always the clearest and I when I came back to it I realised that this post was meandering a bit. While I was working out how to finish it off I saw a new post from Aaron Wall – Slow & Steady vs Hype, Crash & Burn – which deals with the sort of get-rich-quick scammy marketing that I dislike so much. This is the sort of thing that gets internet marketing a bad name. Yet remarkably often the perpetrators have developed sufficient reputation to get away with it for a long time before the smelly stuff hits the air distribution system. I’m reminded
of the heavyweight financial gurus and dodgy accountants that turn out to have been swindling their customers for ages (and then often get let out of jail on grounds of age and ill-health). And I’m horrified that my daily SEO group updates from LinkedIn seem to be increasingly populated by Multi-Level-Marketing related posts. The last thing the SEO industry needs is to be associated with such blatant pyramid schemes or the stuff that Aaron’s talking about.

I still haven’t thought of a snazzy hook but maybe that’s the point of this post – there aren’t any shortcuts for good businesses. Good long-term marketing has to be built on solid ethical approaches.

So if you’re looking for someone to get you search rankings for some get rich quick scheme by dodgy black hat tricks or worthless comment spam then move on, you’re in the wrong place – don’t call me. I’ve spent a lifetime developing principles and I don’t plan to break them now. On the other hand if you have an honest business and you want advice on how to maximise it through ethical internet marketing and SEO then that’s the way I like to operate (when I don’t have flu).

PageRank Distribution and Navigation Systems

Ok, this post is aimed at SEOs rather than a general audience, and is something I’ve been mulling over for quite some time without getting round to posting about it. I got the nudge to finally do so from a post on Search Engine Land – PageRank Sculpting Leaves NoFollowed Tags Behind .

In that article the author essentially discusses PageRank Sculpting, and I should immediately say that in an article-length piece as opposed to something more thesis-length there is a limit to how much can be discussed, so we both have to keep things simpler than we might prefer. We also have to disregard for the purposes of discussion that since you can’t measure real PageRank (as opposed to the hopelessly inaccurate Toolbar PageRank) it’s extremely difficult to know whether your efforts at sculpting have worked.

That said, as I mentioned in my comments to that article there are some fundamental assumptions and questions inherent in trying to decide on a real-world strategy based on this type of sculpting by navigation.

All links are equal but some are more equal than others

The first one is that all links are created equal, but are they? There has been much written about the idea that editorial links, that is links surrounded by body text, are more important than navigational links in the menu system. We don’t know how this works of course – it could for instance be part of the relevancy aspect of the algoritmh that boosts the body links or it could be part of a PageRank algorithm that reduces the value of menu links.

Shooting yourself in the foot(er)

Similarly with footers and other items that can be thought of as boilerplate content. There has been some suggestion that such boilerplate content gets “ignored” for content purposes. Does that mean the links also get ignored or downgraded? And again is that to do with context or PageRank value?

Navigating  Рby the seat of your pants?

Then there is the thorny question of sitemaps. Not in this case the Google/XML kind (they are thorny as well but for entirely different reasons which I’ll leave for another post) but the old-fashioned HTML sitemap. Many SEOs (and many amateurs) trot out the mantra all too easily of “you must have a sitemap”, “it helps the spiders find your pages”. But what are the implications of a sitemap for how the PageRank flows in your site? This has always seemed to me to be a crucial consideration which has usually been ignored.

I well recall spending ages playing about with and discussing the results of the tool at
http://www.webworkshop.net/pagerank_calculator.php?pgs=
when it first appeared back around 2002. Again it’s a very theoretical tool and we don’t know how much the PageRank implementation has changed since then so I’m wary of reading too much into the results these days, but a bit of experimentation with it suggests some pretty major changes in PageRank distribution by using different methods of navigation.

Some people may suggest that a smaller site should use a “get anywhere from anywhere” type of navigation and there is often much benefit to the users in that approach. Some may suggest a more siloed method where different aspect of the site are largely kept separate from each other. Testing the effects of these different scenarios using this tool suggest very different PageRank distribution – sometimes concentrating most of it on the home page and sometimes distributing it almost evenly throughout the site. I’ll leave you to play around with it and see these effects for yourselves, but it may explain some of the odd effects with certain sites where the home page seems to be the one that ranks for everything and the deeper pages don’t seem to get the rankings their content deserves.

My point here is that if you simulate a site and observe the PageRank distribution, and then add a simulated sitemap, even if it’s only linked to from the homepage, then the distribution changes a great deal. Now whether this is matched in a real world situation is another matter, but it bears some thought before blindly following the advice to add a sitemap.

And of course if there is a negative bias against footer links and your sitemap is linked from there… Complicated isn’t it?

Targeting effectively

And it doesn’t stop there. Generally it’s the higher level pages that target the more generic terms and thus need more PageRank to succeed in a competitive market. But generic rankings often don’t convert well, so does that suggest that we should be distributing PageRank more evenly in order to get more of it to the lower level pages where the better-converting specific terms are being targeted by product pages that will actually produce profits?

Good job we don’t have to include clients with their own agendas into this mix or it would get really complicated… Oh! We do.

Now I don’t have the answers to these questions, and maybe no-one outside Google does, but I think they are important questions to consider and discuss – which is the point of this post. Have you had experience of trying to sculpt PageRank and had unusual ranking results. Have you any pages that have loads of good content but seem dead in ranking and traffic terms despite seemingly normal navigation to them? Have you any ideas about the questions I’ve posed here. Maybe we can all gain some insights into a very complex area, but at least I hope I’ve encouraged you to think about it.

Google site command oddities

For some time I’ve been noticing irregularities in the results of the information that Google provides through the various commands and through the Webmaster Tools facility. Of course we all know the link command has been hopelessly broken for many years and frankly I don’t know why it’s still there, but the site command was always fairly reliable until a few months ago. The other fact about it was that it didn’t matter which version of Google you used, which seemed to confirm what we were told about the same information being stored internationally and simply filtered at run-time.

Like many other people I often use the shortcuts available in excellent tools such as the SearchStatus extension for Firefox in order to get indexing and link data from the “big 3″ search engines but do spot checks manually to make sure there are no anomalies. Today, when checking results for a client who sells children’s bedroom furniture and accessories I noticed a large difference in the results of a site command in Google.com (which is what SearchStatus points to) and the same query run manually in Google.co.uk

I wonder what that tells us about how Google stores information and returns it for different queries? Various possibilities come to mind – one that would be nice is if the .com results are the pages in the main index and the others are in the supplementary index, but this clearly needs investigation and thought.

Whatever the reasons behind this discrepancy I’ll certainly be keeping a close eye on it with all my sites.

Who’s feeding the spammers with information?

Spam. We all get it, but for a long time I’ve been nursing a wee suspicion that it’s a bit too well targeted. Over the years I’ve seen a number of odd “coincidences” – spam from people with names very similar to friends of mine, subject lines extremely similar to various subjects I have interests in that are very unlikely to have been selected at random.

But in the last few days I’ve noticed something else and recalled a couple of other earlier similar spams. A couple of days ago I sent a package via an international carrier and within a day I received a message purporting to be from that company trying to tell me that delivery had failed but as usual containing a disguised file that will undoubtedly have been a virus of some sort. I remembered that once before I had sent a another package via another carrier and had later received messages disguised as being from them. Yesterday I paid my phone and broadband bill and today I received a spoof message apparently from the company with a link to a fake login to fix the “failed payment”. Fortunately I’m both experienced and wary, but it would have been so very simple for someone to have clicked on something very similar to a message they were expecting.

Now I don’t believe much in coincidence, and this is all getting to be far too much not to have some sort of causative basis behind it. I increasingly suspect that somehow the spammers are being fed with or are intercepting, more detailed information on all our communications than should be possible. And that, if true is a very worrying development.I may have to dig out my old copy of PGP, but sadly not that many people use it.

Anyone else seeing this sort of pattern in their spam?  Come and join me on the grassy knoll, conspiracy theories rule ;-)

Be careful out there, and don’t forget your tinfoil hats.