Bing and Twitter could shift the search landscape

As Danny Sullivan has just announced Microsoft’s Bing is launching has launched a Twitter Search system www.bing.com/twitter which includes a front page featuring a hot-topic cloud and list of top tweets.

This gives Microsoft a major boost in the increasingly important area of real-time search, an area in which Google is perceived by some to be at its weakest. With Bing soon to be providing search results to Yahoo this marks the beginning of a new phase in the battle for search market share.

In recent years Google has been supreme – in the US they are approaching 70% of conventional searches and in the UK it’s around 80% – but now it seems that the landscape could be moving under their feet. Real time search is becoming the “in” thing and here it’s Twitter that has the most up to date data streams, while in the area of people search and social connection Facebook contains a massive amount of data that is mostly hidden from search engines.

It’s not yet clear whether Twitter’s data will also be available to other companies, including Google, but certainly Bing has stolen a march here and if it can become seen as the standard place to undertake searches of this type then it could swing the balance of power towards Microsoft. The next few months could be very interesting indeed.

Webmaster Tools drops the pixie dust

In the last few days there have been some additions and one removal from Webmaster Tools. The removal is in some ways the more interesting for what it might signal for the future – Pagerank is no longer being mentioned.

Now it was never much of a feature – they told you if you had a lot, a medium amount, or not very much PageRank, and they told you which of your pages had the most. Not exactly rocket science that last one, given that it’s very unusual for the home page not to be the one with most PR. The assumption was that the bars indicating the amount of PageRank were the same as the PR on the Google Toolbar, thought I’m not certain if that was ever explicitly stated and there was never any use of the numbers that appear on the Toolbar.

Barry Schwarz on Search Engine Land believes that they should drop Toolbar PageRank as well to prevent webmasters obsessing about it. In many ways I agree with him although in my experience it’s clients that are the ones that obsess about it – them and the crazy link builders who send you emails requesting link swaps only from high PageRank pages. However in some ways we’re all correct to obsess about PageRank, but the real stuff rather than the poor out-of-date approximation that the toolbar shows. Because it’s still the lack of PageRank that prevents perfectly good content from ranking by dropping the page it’s on to the supplemental index rather than the main one. The fact is we need a genuine indication of PageRank so we can get an idea of how it’s being distributed across our sites.

I’ve recently taken to showing the SEOMoz mozRank in my Firefox status bar (using the SearchStatus extension) instead of the Alexa rank which I find less useful. It’s interesting to compare it to the Google version because sometimes the Google figure shows zero on pages that you’d expect to have a figure of 2 or 3, and you’re never quite sure if there’s a problem with the internal linking structure or if it’s just Google being obtuse. So far the mozRank figures are suggesting there’s nothing wrong and the link strength is being distributed as expected, but when you have a page that isn’t the one that’s ranking in Google for it’s main target keywords then you have to wonder if the toolbar is telling the truth and they’ve dropped it to supplemental status. What you don’t know of course is why. The thing is, if the internal linking isn’t giving the page enough strength for some reason known only to Google then the only recourse is to try and get someone else to link to it for you, and that is one good reason why some people have an obsession with PageRank.

To be honest, Webmaster Tools aren’t as useful as they used to be. Since the changes they made a few months ago the external link results have been both much lower than previously and very inconsistent, while the internal links have been showing some very odd figures – often the home page is being shown as having far fewer links to it than some other pages despite it being actually linked to from all the pages of a site. Such results make it hard to have any confidence in what we’re being told and harder for genuine webmasters who want to build good sites to do so. There’s no point in telling us, as they perpetually do – that we should just build good content when very often that good content ranks far behind some spammy pile of scraper droppings simply because of a lack of PageRank and we have no tools to measure what’s going on.

And then they wonder why some people are also obsessing about link building!

Who’s side(wiki) are they on?

Google have now released something they’ve been beta-testing for a while – SideWiki is a commenting system built into the latest incarnation of the Google Toolbar. Basically what is does is shows a sidebar in your browser where it lists any comments that anyone makes about your site. Systems designed to do this sort of thing have been tried before but never by anyone with the marketing power of Google before.

It’s not entirely clear why they think this is a good idea – in the face of initial criticism the attempted defence by Matt Cutts that you can use it to warn people off a spam or scraper site is perhaps the weakest pronouncement he’s ever come up with – but if it does take off then an awful lot of webmasters are going to be very unhappy about it.

Basically it presents content that you, as a webmaster, are not in control of to users of your site, and there has to be something fundamentally wrong about that. If someone wants to criticise my site on their site then that’s their right (as long as they don’t cross the line into libellous areas). If they want to criticise using my own commenting system then I have some control over that and can reply to their comments. But to present content as part of the experience of visiting my site, without my permission and with no means of me responding or complaining is simply hijacking. There could be links to competitors sites, there could be spam, there could be adverts, there could be objectionable material such as racist comments. The average user simply won’t understand that this material is not mine and as always mud will stick.

Sorry Google, you have got this horribly horribly wrong. I hope it falls flat on its face the way the earlier systems did, or that you reconsider and withdraw it.

For the sake of the web in general – which you yourselves described as a cesspit – it’s an open door to the worst kind of spam and dirty practice.

For the sake of all the genuinely ethical small web businesses who already have far more to do than they can handle to keep themselves going in the face of spammers and scrapers.

And actually for the reputation of Google itself, because this will be a massive PR blunder in which you are seen to be setting yourself up as the supreme ruler of the internet and doing something which is directly against the interests of the people who build and maintain websites.

Greatest wastes of time in SEO #2 – superfluous meta-tags

To contnue the theme of the first “wastes of time” post, here’s some more activities that still get thousands of pointless column-inches in SEO forums and people still spend countless hours on. Meta-tags are a prime source of mis-information simply because you can pretty much create anything you like and claim it’s useful. Some of the worst examples are the following:

The Revisit tag

<meta name="revisit-after" content="15 days">

Supposed to be an “instruction” to the spiders about how often to return. Totally useless. Was never implemented by the major engines and is never likely to be.

The Meta-Title tag

<meta name="title" content="">

Not to be confused with the Title tag. Not supported by any of the major search engines. Complete waste of SEO time.

The Index Follow tag

<meta content=”index, follow”>

The waste of time here is the time it takes to get it right. All pages will be treated as to be indexed and followed by default (assuming that your robots.txt file hasn’t already blocked the page) so what’s the point of including an instruction to do what the spider is going to do anyway? But I’ve seen so many mis-formed versions of this tag , some so bad that they could potentially disrupt the spidering action, that if you’re going to do it you need to spend time getting it right. Don’t bother – just take it out and get a smaller page as well.

The Dublin Core tags

These are a set of tags created originally for library classification (the name refers to Dublin Ohio, not Dublin Ireland). They are an extensive system of categorisation that is of considerable value in their correct context. You may decide that you want to have them if your site has the sort of context that merits their use, but be aware that they will take a lot of time to set up correctly. I’m definitely not saying they are useless, but as far as SEO is concerned the time taken to add them is wasted time because again none of the major search engines pay any attention to them. It could be argued that they should, but that’s another matter.

The Meta-Keywords tag

And now we move to that most talked about tag in amateur circles, the one that people spend hours debating and constructing. Just last week Matt Cutts had to reiterate what any remotely professional SEO knew years ago – that Google don’t use it for ranking calculations and haven’t for many years. And yet blogs and forums all over the world have reacted as if it was major news! Come on guys, where have you been for the last five years?

Once upon a time in a far distant galaxy this was a useful tag – not vital, but certainly useful. Last century it could get you a ranking almost on its own. I can remember even in 2002, when I was working for Bigmouthmedia, carefully arranging keywords in different orders so that they automatically created phrases within the keywords tag. By the following year it was already becoming a very secondary activity and it soon became pretty pointless. Worse, many sites would put ALL their keywords into every page instead of targeting them at the pages that were relevant to them. All that does it get you flagged as a potential spammer. In recent years only Yahoo has used the keywords tag in any way at all for ranking. I’ll sometimes stick a few main terms into the tag for a page, more for the sake of completeness or because a client expects it, but often I’ll leave it out altogether – there are far more important things to spend time on.

How mature is Bing and is it worth optimising for it?

While in the US they seem to be raving about the rise in search market share that Bing has achieved, even before the extent of the Yahoo results takeover becomes clear, here in Europe its making little impact on Google’s dominance. In fact the figures I see in the major market share analyst sites doesn’t seem to bear out the hype headlines anyway, which seem to be focusing more on % growth rates rates than actual share, but that’s beside the point I want to make right now.

I monitor a number of sites for search results and the impression I’m getting from these results is that Bing is far from stable. I’m seeing results yo-yoing wildly and the page being ranked changing frequently.

Yesterday I saw a strange result concerning two pages dealing with particular product names. The result for product  A dropped from near the top ranking to the sixth page, because page A was no longer ranking, instead page B was the one being ranked. Page B has no mention whatever of product A apart from the main menu entry. In another instance the ranking from product C moved from Page C to the home page, which again has no mention of product C.

Does Bing have a supplemental index?

So is this because the Bing boffins are still tweaking their algorithm, or is it maybe because they still haven’t finished a full round of indexing. Or could it perhaps be another reason?

This is the sort of behaviour we might expect from Google’s supplemental index when a page that has borderline PageRank drops from the main index to the supplemental index – though with Google you would expect the secondary ranking page to have at least some mention of the product being searched for!

Could it be that Microsoft have ambitions to index the same sort of volumes of data that Google aim at and are running into the same problems of having to exclude some pages from their main index in order to speed up search times? Now this is pure speculation on my part but if it’s true then what signals are they using to decide which pages are main index material and which are relegated to whatever their equivalent of the supplemental index is (assuming they don’t just tip them out of consideration altogether)? Does Bing have something like PageRank or is there some other indictaor that they use?

The reason this might be important is that I’ve seen a number of articles (admittedly American) suggesting that we should all start optimising our sites for Bing. Now aside from the fact that we should optimise primarily for users, not search engines, how would we go about this? If they do have a means of separating out the pages they take notice of we’d need to have a good idea of what it was before we could be sure that the sort of scenario I’ve mentioned above can be avoided. We know very little about the Bing algorithm and we don’t know yet how Yahoo will use their results when they start displaying them. In the UK it probably isn’t that important right now but if something were to cause a swing away from Google then it could get to be. And in the US it could affect 20% of the market.

Webmaster Tools – new feature lets you exclude parameters

Last night while going over some clients’ accounts in Google Analytics and Webmaster Tools I noticed a new feature which hadn’t been there a few days ago. Under Site configuration / Settings there is now a section called Parameter Handling where you can set whether or not Google should ignore any of the parameters that are contained in your URL strings.

Please be very careful with these unless you’re absolutely certain you know what you’re doing. While the tool may well be useful to avoid pages appearing under multiple URLs, a mistake here could easily drop pages from their index that you want in there. There’s a description of the new tool from ex-Google employee Vanessa Fox published on the Search Engine Land site yesterday. Recommended reading. I’ll add any other discussions I see on it that look useful.