Search Engine Musings

So the much speculated bid has finally happened. Will they say yes? If they do what will the search industry look like in a years time.

Recent figures on search market share in the USA suggest that Yahoo may be losing ground and MSN/Live gaining, though there seems little suggestion of it in the UK. Indeed there is a suggestion in some figures I’ve seen that Ask is finally beginning to make an impact and is close to MSN/Live, though both are at a very low level.

Looking at the respective assets of Microsoft and Yahoo there is a great deal of overlap that would seem to suggest that any amalgamation is going to be painful for both staff and users. Both companies have mail systems, both have instant messenger systems, and of course both have search systems. The chances of duplicate systems surviving the accountants’ eagle eyes seem to be low, yet a lot of brand loyalty could easily be killed off if it’s handled badly. There some areas that will be attractive to Microsoft - the Flickr photo archive for instance - but the conventional wisdom amongst most commentators (Aaron Wall being a notable exception) seems to be that they want Yahoo for search.

Now Yahoo search is a strange beast these days. Aaron calls it stale and there is something in that description. Their speed of response seems slow - at one time they were faster than Google but these days it seems to take weeks before they respond to site changes - and their results are often very strange. The odd thing is that they themselves took over some interesting search engines but don’t seem to have done much with them - Alta Vista may have been an odd takeover choice but AllTheWeb was a very promising engine that seems to have been sidelined rather than incorporated. Maybe that’s one of the things that Microsoft want. Certainly they seem to be getting nowhere with their own search system; it was promising for a while about 18 months ago but once Live Search arrived it seems to have become erratic and on some searches produces some of the worst results I’ve seen. Another possibility is that they want the Overture PPC part of the business to shore up their falling ad revenues

Quite honestly I can’t see Google losing any sleep over the search aspects of the combination - they are so far ahead in that area that it seems impossible for anyone to challenge them unless they themselves make some bad blunders. Not impossible, given the negative press they’ve been getting over privacy for instance, but fairly unlikely. But they certainly seem to have been stung into action given their hard-hitting comments about “illegal practices” being carried into the internet arena. Maybe that’s just a knee-jerk reaction or maybe they think Microsoft are up to something and are desperate to nip it in the bud. Apparently they were concerned enough to offer Yahoo a deal to stave off the takeover.

All-in-all my suspicion is that the war between these two giants is going to be more far-reaching than just search. Such is Google’s lead that Microsoft can’t win on just search and they are a company that doesn’t like to come second. They must also be worried that Google’s attempt to switch users to online-based data storage and office apps is threatening Microsoft’s traditional monopoly in the PC office suite and operating system area. If they were to lose that then they’d be in trouble. So the search market may not change all that much this year but a lot of other things might.

I noticed a curious phenomena in my Google rankings this week. While my general rankings based on the main site are doing well and mostly improving, all the terms that were ranking via this blog have dropped out of sight. Now I haven’t been blogging as much recently due to other committments so now I’m wondering if the drop is simply a Google glitch that will recover as they generally do, or some more fundamental change in the algo that has hit my blog results, or if it’s simply because my content isn’t being updated as quickly as before.

If it’s the latter then I wonder if we’ve all made a rod for our own backs in that we need to keep on writing new articles more and more or face the consequences. As I blogged about in my Coals to Newcastle post I’m reluctant to churn out the same stuff that everyone else is doing because that feels wrong and pointless. On the other hand I’m a bit miffed to find that the top rankings I had for  terms like “Scottish SEO consultant” and which I blogged about in Despairing of Google, have collapsed down to about 280th place.

Anyone else seeing a blog dropping out of the rankings?

Google announce the end of the supplemental index

In a curiously timed announcement, just when many people are turning to thoughts of Xmas rather than SEO, Google have said that from now on all search queries will access the full index for relevant results, effectively ending the problem of good quality pages being ignored because they’ve been relegated to the supplemental index.

Supplementals have been a serious problem for the last few years for many sites which have either a deep navigation structure or insufficient inbound links - both situations that result in pages with a low PageRank; which is known to have been a trigger for dropping out of the main index and into the supplemental one. Because Google was only employing the supplemental index when there were insufficient relevant pages in the main index for a query, any pages with rich content but poor PageRank were effectively being excluded from achieving good search results.

Some commentators have suggested that the supplemental index goes hand in hand with PageRank to enable a fast enough response time to queries - Dan Theis has a blog posting on the subject which makes interesting reading and it’s intruiging to wonder what Google have been doing under the hood if his theory was right and how they’ve overcome the performance problems it suggests.

So far I haven’t seen any evidence of change in search results, but that may be because I’ve been used to designing with a view to avoiding the problem. However I’ll be revisiting a number of sites which I know have suffered in the past to see if matters improve for them.

It has to be said that a number of SEO commentators are sceptical about whether there will be any real benefit or if it’s just window-dressing. Andy Beard is one of those who also isn’t seeing any change in the SERPS. However to be fair there may be a delay in getting any changes rolled out through the vast number of data centres that Google uses. One thing I’ll be watching closely is the listing of internal links in Webmaster Tools, because I’ve long suspected that this only lists pages that come from the main index. If that is true then we should see all the pages in a site appearing in that listing. I’ll keep watching this and return to it in a few weeks time.

Sometimes I really do want to beat my head off the keyboard! Like many others who may be reading this I epouse ethical search engine optimisation techniques. I advise clients not to use any of the spammer’s tricks and to produce quality content. I tell them not to use hidden text or dodgy redirects or spammy meta-tags, that Google and the other engines will pick that stuff up and kick them out of the indexes.

And then what do Google do? I’ll demonstrate.

For quite some time I’ve ranked number one in Google for “Scottish SEO consultant”. I’m not precious about it, there are some excellent SEO consultants in Scotland and I’m happy to be in amongst them - the phrase happens to be in my blog title so that gives me an advantage on that one out of scores of other similar phrases. Tonight I checked some of my rankings as Google has been erratic again recently (coincidence with the big PageRank debacle?) and saw that I was down to number six for that phrase. No problem, noticed that Shaun Anderson’s Hobo site was above me, that’s ok it’s a damn good site. A bit peaved to notice that above him was a company in Derbyshire (with a greyed out PR bar!) whose only connection to Scotland seems to be that they did a site for a Scottish property company.

But what really got my back up was the site at number one -
Work at Home - company scotland seo directory
www.easywebcreator.com/workathome0/company%20scotland%20seo

This is basically a directory search result on a site that advertises franchise businesses and is crammed full of Adsense ads and padded out with Google News items. But that’s not what really bugged me. If you visit that page go and view the source code and look at the meta-keywords tag. For those of a non-technical disposition or who would rather not visit it I’ll describe it. I copied it into a Word document to check the number of characters. It filled 9 pages! It has 5622 words, and 41,653 characters!! These include such relevant terms as “abraham lincoln”, “angelina jolie”, “Black Sabbath”, and yes, you guessed it “Britney Spears”.

This is what the world’s most popular search engine thinks is the most relevant site for “Scottish SEO consultant”. A company full of the finest brains in the world who spend enormous amounts of time devising methods of catching spam, and they can’t find a page which has a keywords tag full of total unmitigated spam.

Matt Cutts, are you listening?!!! Because as long as this sort of stuff is not only allowed through the indexes but actually gets to the top of search results your appeals for webmasters to do stuff like add “nofollow” to links are totally laughable. It undermines any chance of ethical SEOs nudging our clients in the right direction towards quality sites.

I’m off to find another keyboard, this one’s got a dent in it.

Either someone at Google has a sense of humour or they really are out to make robots of us all. Let’s hope the former is the reason for calling their latest venture Android. This is the apparent reality behind all the long running rumours of a Google Phone. In fact it appears to be an attempt to bring an open standard operating system to enable extensible gadgets and capabilities for internet-capable phones.

Such a system would have great attractions for a company that wants to put itself into every aspect of communications and potential advertising media, but the move must have a lot of phone manufacturers wondering what their long term chances of survival are. Can any of them afford to go it alone or is it a case or collaborate or die. Or will Google just take over anyway? It’s also another battleground for Microsoft who have been trying to push the use of their own mobile operating system in a market that currently has quite a few of them.

The search world has been in meltdown for the last few days because of a little green bar.

Toolbar PageRank (tbpr) is the source of more wasted blog time and forum comments than any other topic. Despite repeated statements from the more sensible end of the SEO market that toolbar pagerank is neither a true reflection of the real unseen PageRank, nor in any way worth bothering about, the mass of webmasters seem to obsess about it.

Last week a lot of people reported falls in their tbpr and the speculation machine surged into action.

• ‘They’re penalising paid links’ - hmm, so why are sites being affected that don’t sell links?
• ‘They’re penalising blog networking’ - hmm, so why aren’t some very obvious link schemes being affected?
• ‘We’re all doomed’ - hmm, your tbpr seems to have gone back up again.

None of these experienced commentators seem to have followed the usual golden rule in these situations - sit on your hands, do nothing, and wait to see what happens after a couple of weeks. After every piece of twiddling or lever pulling, whether minor or major, there is always a period of instability. Sit it out and then review the situation. Wildly changing some aspect of your site only adds another variable into a situation that already has too many of them.

Meanwhile Google themselves were remarkably quiet. Official mouthpiece Matt Cutts has been making no search related blog posts for a few weeks, and no other comment has been forthcoming. We seem to be back to the bad old days of complete non-communication. This really isn’t good enough for a company that expects us to trust it with all of our data. It has to realise that if it leaves a vacuum then speculation will inevitably rush in; most of it ill-informed and potentially harmful to both webmasters and to Google itself.

For instance the latest turmoil could easily be interpreted by the cynical as being the third year in a row that the natural SERPS have been thrown into randomness so that Xmas-dependent businesses would have to spend money on Adwords. Is that true? I don’t know, but I do know that a lot of people suspect it to be, and in the absence of any explanation from Google that is the view that will spread. Is that what they really want?

The fact is we don’t know if this is a major update, or a change in PR algorithm, or a manual blitz, or a warning shot across the bows of  link sellers, or an experiment (failed or successful). No-one can respond sensibly to such a situation, so if it is a warning shot as some have speculated, then it’s not very well aimed. And as has been mentioned by many, it’s one that won’t even be understood by most web businesses because outside of the search industry most people haven’t a clue what a nofollow tag is or why it might be important. They also have no idea about selling PageRank - they are just trying to make some money from advertising on their site. That’s the reality - most site owners don’t know much about web design and know nothing about SEO and it’s related subjects. They just see a medium for communication - and that includes advertising the same as any other medium does. They see Google selling adverts with links on them - must be ok then.

Come on Google - get rid of the PR and make with the Public Relations.