How do we effectively use the opinions of others to make decisions?

To the annoyance of my family I like a nice cup of coffee. And so when we are out and about I like to pick a coffee shop that is going to meet that expectation. 

Like many of you I rely on the reviews left by others on different websites and select an establishment to try. Sometimes they are good, sometimes……well, not what I expected from the review.

Because here is the thing: reviews are subjective

One person’s 3 stars can be another person’s 5.

Sure, everyone has a different opinion, that’s life. But assuming we can accurately and objectively measure that opinion is harder than it first seems. 

If I was to ask you to rate how today has gone so far on a scale of 1 to 5 what would you give it?

Is that any different to yesterday? If so, why? What has changed?

And what could make the score different tomorrow?

Now think about that score. Why did you give it say  3 and not a 4? Why is a 3 better than a 2?

This isn’t an arbitrary or obscure issue to consider. There are many times where you not only reply on such evidence to make judgments but probably ask them too.

Questions, like the one above, which ask for a rating on a scale of 1 to 5, the number of stars or Very good to very bad, will be familiar to many of you.

Many of you will incorporate them into surveys you (routinely) use in your work. And like the coffee shop you will probably make decisions based on the general concensus.

But ask yourself, what is the difference between each point on those scale? Exactly what defines and differentiates a “very good” from merely a “good”?

When you ask a customer to rate your service, what are you asking them to measure? More to the point, what are they comparing your service to if at all? 

When my wife and I were selecting hotels for our honeymoon we soon realised that the scores given for hotels were influenced largely by the country the reviewer was from and the expectations they had of, sadly, the bathroom and toilet size in a hotel.

Qualities matter when giving opinions. Scores and scales alone don’t provide enough information to understand them.

And so it is with coffee. What do I mean by “nice”? I can’t tell you but I’ll know when I’ve found it.* 

Sometimes it is the quality of the coffee gets top billing. More often than not it is the deliciousness of their pastries, the ambience of the shop, the attitude of the staff or even just the company the reviewer was in at that moment. 

These qualities explain more about the review. Sometimes they are useful proxies for the quality of the coffee but we can’t take that for granted either.

But what they do give us is a more rounded picture of those opinions and the factors which shaped them. It pays to combine the way we gather those opinions so you have better information on which to make your decisions.

So next time you ask for opinions in a survey think how you can overcome those pitfalls of scale by asking for a few small extra details too.