Published November 2003

When polling the marketplace, understand that pitfalls do exist

The recall election in California resulted in Arnold Schwarzenegger’s replacing Gray Davis as governor. The political implications will undoubtedly take a while to shake out, but while we wait for that, there are a few management lessons for us if we look closely at what happened.

From a management perspective, the first thing that jumps out at us about the events leading up to the recall movement, the campaigns and then the recall election itself was that somewhere between, most and almost all of the opinion polls were wrong.

They were so consistently and sufficiently wrong, in fact, that they cast doubt on the few surveys that got it right. And these opinion polls were important to the candidates’ strategies, just as market research information is to business strategies.

It is difficult to say, even in retrospect, what went wrong with the opinion polls, but there are at least two likely causes, and they are both relevant to the business world.

The first is that polling firms are no longer simply polling firms. Many have become politicized, just like everything else in America, so that we now have Republican pollsters and Democratic pollsters.

Extensive contact with just one side, or one client, exerts a subtle influence on pollster attitudes. And while everything is done professionally, using “state of the art” sampling and statistical analysis, they cannot eliminate entirely the kind of bias that plagues all research: the tendency to find what you set out to look for. As a form of research, opinion polls are notoriously sensitive to subtleties in words, phrasing, voices and scripts — when telephone surveys are conducted — and appearance, dress, manners and location of pollsters if a face-to-face survey is used.

The second is that Americans have become far more determined to avoid contact with pollsters. The days of being flattered by the idea that someone cared what you thought are long gone.

Because of cost considerations, most opinion polls are conducted by telephone, and the percentage of hang-ups and “no response” has been growing steadily over the past half-century and especially so in the past decade.

People are a lot busier today — and a lot less polite — than in times past. They resent intrusions into their routine and are not shy about letting callers know their feelings. It isn’t an accident, for example, that people have signed up to place over 50 million of their telephone numbers on the national “do not call” list. (Ironically, the list may not screen out political polls, but pollsters dialing up the numbers on that list should brace themselves for some negative attitude at the other end of the line.)

Perhaps related to Americans’ resentment of poll-takers is their newly discovered willingness to deceive them. During a recent election, one set of survey-takers, conducting exit polls of voters — and predicting the wrong outcome — was shocked to conclude that “… they lied to us.”

These same factors, bias and difficulty in extracting accurate information, play a significant role in business. They are most visible when a firm is considering the introduction of a new product or service, but they also show up when management is trying to improve, or reposition, an existing product.

A big problem with using market research on a new product or service is identifying any biases that might have altered the picture painted by the reports. Professional firms can help you with this, but even they, very naturally, often unconsciously develop a “pro-product” outlook — because they’re on your side and they want it to succeed.

Similar problems can come up even if your company lacks the budget to use professional marketing research. One technique that small firms use, for example, is to “ask your best customer” about a new product or a product change.

Unfortunately, this often produces positive feedback for a product that goes nowhere when it is introduced to a larger market — because firm and customer share an interest in the product’s success.

We should keep in mind that these problems are not related to the quality of the product itself, only the customers’ perception of it. That’s why, in some cases, your “best customer” is not your best judge of, say, an improved product, because they are not seeing it standing alone, as the public see is, but as a better version of something they already know and like.

So what’s a manager to do? First, simply be aware that the information you are getting may be too good to be true. Check it out a different way: If you’re using your “best customer,” for example, see what a customer picked at random says, too. Second, and most important, don’t forget to trust your instinct. In a world of uncertainty, it’s worth a lot.

James McCusker, a Bothell economist, educator and small-business consultant, writes “Your Business” in The Herald each Sunday. He can be reached by sending e-mail to otisrep@aol.com.

Back to the top/November 2003 Main Menu