“Facts are stubborn, but statistics are more pliable.”

At last night’s AiMA event on search engine strategies, the speakers referenced a study where users showed no significant preference to Bing or Google. After a short web search (via Google), I found the research paper by the Catalyst Group (see below). In the study, users reported that they wouldn’t switch from their current search engines even though Bing possessed some favorable improvements to Google.

While the findings attempted to explain how Bing will never catch up to Google, I was surprised that the speakers chose to quote the study as fact. There were two glaring issues that I immediately noticed once I located the study:

• The participants used Google as their main search engine.
• The study involved only 12 participants.

While some may argue that the opinion of the participants was tainted (since none used Microsoft Search as their primary search engine), I was even more surprised that no one considered the small participants pool. While I subscribe to the notion that we’re creatures of habit and users won’t change when they can’t find a significant value in the new “shinny object,” the bottom line is that the study’s findings are statistically invalid. While I’m fairly certain that the speakers were not aware of flaw in the study, it demonstrates how easy it is to rely on and propagate bad statistics and how careful marketers need to be when they quote a study.

NOTE: I recently wrote a blog post about Twitter data and Rapleaf based on an NPR radio bit. Rapleaf reached out and explained the issue with the misinformation that I had referenced. It goes to show that we’re all human; we make mistakes.

Catalyst Group Bing V. Google Usability Study

The dashboard is a welcomed service for two reasons:

It is an initial privacy control mechanism. While Google remains heavily dependent on search, Google continues to expand their services offering. In reviewing these services, it has become clear that Google controls every medium of communication with the exception of fax and snail mail:

NOTE: Google has Gmail (Email), Google Chrome (Web Browser), Google Wave and Google Talk (Chat/IM/Video), and Android and Google Voice (Mobile, Phone).

If Google decides to continue their acquisition streak, Google may find themselves in the eyesight of privacy advocates that are worried about Google owning too much information about you. If Google sits on the sidelines, Google may be forced eventually to disclose personal information to their end users or to limit the information that they can collect. The Google Dashboard gives Google leverage over privacy advocates in that they give you control over your information.

It prepares users for Google Chrome OS. A few months ago, Google announced the development of an OS for netbooks (source). Additionally, Google is known for their exceptional user experience. It seems that the Google Dashboard makes an initial attempt at consolidating account preferences into a single view, a feature that one would expect when Google releases their OS.

While the Google Dashboard is by no means perfect, it is a fine start.