Bing Brings its A Game

by

[Update: Bing won the race to index this blog post. A search 30 minutes after this post went live for <bing brings a game> showed up on Bing but not Google (Yahoo! wants to spell-check “bing” to “being”).]

Wow. I just did fifteen general non-structured searches to compare Microsoft’s new search engine, Bing to Google and Yahoo!.

I must be like Woz, because after a full bout (fifteen rounds), while there was not a clear knockout, I have to say I’m amazed at how well Bing works. Noticeably better than Google for many queries, in fact.

I liked Bing’s snippets much much more than Google’s, but they should adopt Google’s indentation policy for multiple results from the same site (that does cause re-ordering, but it’s useful for me as a consumer). The Bing snippets would be even better if they highlighted the search terms!

In most cases, Bing had a noticeably better diversity of results. For those two reasons alone, I’m going to switch to Bing for the near term and see how I like it.

I’m not happy with either Bing’s or Google’s attempt to “facet” results (by clustering or what?). I’m very happy with how up-to-date both engines are (e.g. <French Open>) and how they present a few news headlines.

Yahoo! has the best followup and related query suggestions as you type, but the ones they present with results aren’t as good as the live ones. Bing and Google take different approaches (related versus refinement), and both are OK, but not particularly insightful.

Google’s spell checker seems awfully aggressive. And I’m feeling that they’re being a little too agressive stemming (I need some good examples of this — none came to mind right away; but for example, “infinity” is a synonym for LaTeX’s “infty” command, at least if you can go by highlighting!).

Google and Bing are both surprisingly up to date. I’d hate to have to solve the spidering and updating problem they face.

I like Bing’s clean layout — Google’s results feel cramped by comparison. Now that I look at it, Google’s one-column layout feels very late 1990s, early 2000s. Bing’s narrow left-hand column with types of news and related searches (for query refinement) seems cleaner than Google’s approach, where you have to scroll down to related searches.

I really appreciate the big targets for the paging at the bottom of Bing (ditto Yahoo!); clearly Google’s designers have never heard of Fitt’s Law.

I’m really impressed with how fast Bing is. It’s at least as responsive as Google if not more so. Let’s see how that goes if it gets more popular.

Query Comments
woz bing tie; both present news and put original source first
tortoise I was looking for the band, who I just saw on Sunday; no tour schedule or MP3s, but Bing crushed the contenders in terms of diversity and also sort order. Many more ads on Yahoo!, none on Bing. Yahoo! missed TortoiseSVN, and Google’s still touting TortoiseCVS. The Yahoo! related concepts are nice (e.g. band members and types of the reptile)
tong zhang Everyone gets his current Rutgers home page and then publications. Yahoo! has lots of irrelevant results and Google lots of stale results (like his old IBM page).
little professor baseball This is a baseball simulation I wrote, and all three sites get its home page and the math/stats page right. Bing had more phrasally relevant results than Google later. Not enough diversity on Yahoo!’s results.
andrew mccallum’s publications for some reason, Google’s “spell check” wants to get rid of the possessive, which is silly — this is what you get for relying on query histories! Both Bing and Yahoo! seem more on target than Google.
eos 5d review These results are much better than the last time I checked. Different orderings, but all the sites seem to be taking care of the spam pretty well. Google had the only resonable query refinement suggestion, <eos 5d mark ii review> Bing suggested <Canon eos 5d>, which doesn’t exactly help matters; we need followup terms that will discriminate, not ones that are related. Oddly, Yahoo!’s live suggestions are quite reasonable as are the related searches.
carpenter chu-carroll call routing I needed this search for the previous link! Why isn’t the ACL archive better indexed? Bing at least gets both the journal paper and conference paper on the top page. Yahoo!’s clearly defeated here, for some reason going with the Singapore mirror of citeseer.
pittsburgh hamburger Bing’s pushing Wendy’s! The right answer is Tessaro’s, the best burger I’ve ever had. Google lists it in the second hit. Yahoo! loves Burger King and McDonalds. WTF? But Yahoo! comes through with the Yelp! page for Tessaro’s. Yahoo!’s suggestions were also good, citing city down to neighborhood.
arithmetic coding I like Bing the best, because my tutorial is ranked 5 and the home page for my implementation is in the top 10; it’s 9 on Google and not on the first page from Yahoo! Everyone pushes Wikipedia to the top and Mark Nelson’s great survey second. Good job.
definition of L0, L1, L2 L infinity norm The answers are pretty good for this tough query. For some reason, the last time I searched for this topic I couldn’t find it.
spell chcker Bing and Yahoo!’s interfaces clearly let you know they’re searching for <spell check> or <spell checker> as well, and let you easily restrict to the actual query; Google implicitly searches for <spell checker> but I don’t see how to limit results to just what I typed. Bad Google.
gary king text company Let’s say I forgot the company’s funny name (Crimson Hexagon). Not much help from any of the search engine’s here. At least that means there’s still work to do.
david smith umass OK, so who’s still using Yahoo!? Both Bing and Google get the two with UMass affiliations on their home page or LinkedIn, listing first the right David Smith’s home page (nice photo, David).
bed and breakfast san francisco Amazingly, Bing’s doing as well at geolocation and displaying hits on a map as Google (word on the street is that Metacarta does this best); the old LiveSearch used to be crazily agressive at trying to sell you “local” ads. Yahoo! had very nice faceting of the results (labeled under “Yahoo! Shortcut”), but I didn’t check their validity. Bing supplied other things I might be interested in (e.g. <san francisco restaurants>, <san francsisco tourist attractions>) as related searches, whereas Google hewed to refinements (e.g. <victorian bed and breakfast san francisco>)

The champ’s a bit woozy, and there’s still one round left:

Query Comments
boxing scoring decision The snippet of Bing’s first hit explains the whole concept, which is way better than the other engines.

It’s too close for me to call with just these 15 queries. But clearly Bing’s a contender for general web search.

Should Google be worried? Is the ad business zero sum? (Those questions weren’t rhetorical — I know nothing about the ad business other than that it’s been profitable for Google and annoying for me as a user.)

6 Responses to “Bing Brings its A Game”

  1. Brendan O'Connor Says:

    Very interesting. I like that your blog post is probably the only one on the internet that reviews Bing deeper than a blogger’s one or two vanity queries :)

    I’m also pleasantly surprised by its excellent responsiveness. Once upon a time, after seeing some positive relevance test results, I tried to use Yahoo Search as my default search engine. But the biggest annoying thing was the higher latency over Google. There’s just such a strong feeling of 400ms vs 100ms latency.

    As far as layout goes, Bing does kind of cheat in that its right column is free of ads, so there’s room for those lovely longer-summaries-on-hover. Assuming they want to cram that space full of ads, like G and Y do, something’s gonna give.

  2. Brendan O'Connor Says:

    and also .. gary king has a text analytics company?? he did coauthor that neat paper on political event extraction, and i thought he and/or other harvard social scientists were doing some big blog crawling/mining work, but huh.

  3. Marco Says:

    The big question now is whether Microsoft will take a big chunk out of Yahoo’s paid search revenue, as mentioned here: Does Microsoft Big Change Paid Search.

  4. lingpipe Says:

    Check out Dolores Labs’ Evaluating Bing [vs. Google] with Mechanical Turk, which gave the edge to Google (though see my comments which question the accuracy of the study).

    Here’s one query Google won hands down: [Peter Jackson West Thompson]. I only went back to Google because the Bing results were so bad for this query.

    There’s all kinds of interference from the film director, footballer, and AP newswriter on Bing’s results, which is what I’d expect. Google’s results after the first two were all over the place (West Thompson lake at rank 3?).

  5. Vaibhav Says:

    Hi, your experience with Bing is quite different from mine (as you noted in comments on my blog – I replied to it, so you may want to come back and weigh in) (http://blog.gadodia.net/bing-vs-google-no-fancy-analytics-pure-personal-experience)

    Having said this, your results are what most people are reporting – Bing is better at certain things than Google and vice versa. I hope someone can start categorizing this better, because then we can use both engines simultaneously for whatever they are better at :)

  6. David clark Says:

    I would like spell check on my bing bar.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s