"Stay classy" ... Will Ferrell in movie Anchorman.
There’s a great line in the Will Ferrell movie Anchorman about market research. Asked about the power of his cologne in attracting women, a TV reporter says: “They’ve done studies, you know. They say 60 per cent of the time, it works every time.” Judging by recent advertising, more Australian companies are trying to persuade consumers using market research that makes little sense.
Take pizza as an example. One I bought on the weekend had a “five star” rating, according to a consumer products survey. This, of course, was more stars than its nearest competitors. Pity the pizza tasted like two stars. I’m sure the pizza box could achieve one star for taste if the right questions were asked.
I’m no statistician, but I always check the survey sample, how the survey was conducted, its independence, and the motives of those marketing the survey – at least for big purchases. Granted, when I’m hungry, little thought is given to food ratings.
You might think I’m nit-picking, but who holds flimsy consumer products surveys to account, or the advertising material they fuel? How many people buy things because of survey ratings?
What’s your view?
- Do you believe all these customer “ratings” on various products?
- Are you seeing more companies using star ratings to promote their products?
- Do you question consumer surveys or just take their results at face value?
- What are the best examples of dodgy market research to sell products?
- Or companies twisting market research to suit their purposes?
I shouldn’t just beat up on consumer products surveys. The next day I saw a university advertising high ratings for student satisfaction. Pity so many Australian universities have much lower ratings in research. Some would be lucky to get one or two stars for research by global standards.
I wondered: do smaller universities that attract lower-quality students (compared to top unis) in turn get higher satisfaction ratings because their students expect less? What if a university gets a low rating because it pushes students to excel and fails underperformers (something increasingly rare in many business schools)? Does a teacher who marks hard and is tough in class get a lower student satisfaction rating than another teacher with poor standards – and suffer as a result?
And how did these universities rate in other survey aspects? You have to look for this information, and realise that most universities look good on at least a few survey aspects, such are the number of surveys each year and large range of data points within them.
Don’t even get me started on user-generated ratings on music and book portals. You know the ones: a user rates an album five stars and comments: “This is the best album of the year. Every song is great. You won’t be disappointed buying this album or the earlier ones.” Yeah, right. I can just picture the record company’s marketing team writing that rubbish or paying someone to.
And what about all the investment products and services that win awards for this or that, yet have lost investors so much money over the past few years?
Also annoying is star ratings that bunch together. Why do movie reviewers criticise a movie and still give it a three-star rating, or gush about a movie and provide a four-star rating? Surely these reviewers should make full use of the star system, from one to five. Yet most ratings seem to cluster in a meaningless blur of mediocrity.
Don’t get me wrong: some rating systems provide great value. Car safety ratings, and some of the work Choice does, to name a few. But good rating systems are being swamped by other consumer products surveys that lack independence and provide scant information on survey methodology. In some ways, the internet has lowered survey quality by making it much easier to collect results.
Other surveys are only about marketing, with reliable information a distant second. Business is especially guilty here. You know the drill: survey a group of customers, to emphasise a particular problem that only your firm can solve. Focus on the most extreme survey results and encourage customers to take urgent action – again using your firm. And flog the survey to the media.
I’ve lost count of the number of small business surveys – all from companies that provide products to small and medium-size enterprises – predicting the SME world is in dire trouble.
Maybe someone should start an independent star rating system that rates the ratings providers. Or a survey that shows how effective and robust all these other surveys are. Maybe the Australian Competition and Consumer Commission should force companies to exercise more care in their reporting of survey results in marketing material, and provide more information on the survey itself.
Consumer and business surveys can add great insight and value. That’s why they should be protected from the growing number of flimsy surveys that are disguised marketing tools.
13 comments so far
More comments
New user? Sign up
Make a comment
You are logged in as [Logout]
All information entered below may be published.
Thank you
Your comment has been submitted for approval.
Comments are moderated and are generally published if they are on-topic and not abusive.