You've successfully subscribed to Thematic
Great! Next, complete checkout for full access to Thematic
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

Why you need to mine positive scores for negative themes

Your survey results look awesome. The scores are high and customers seem generally happy. Most survey responses have top-box ratings, which means the customer selected one of the two highest rating options. Nothing to worry about, right? Not so fast.

There might be a hidden danger lurking in those positive surveys. One client I worked with had as many of 5% of their top box survey scores contain negative comments. In other words, the score was good, but customers still had a few complaints.

3 ways it can happen and what you should do about it:

1. Protecting an employee

The customer was upset. Not at the person who served her. She felt the employee was friendly and knowledgeable. The customer didn’t like the policy the customer service representative had to enforce.

The customer wanted to share her feedback on the company’s customer service survey, but she didn’t want to get the employee in trouble. So she gave a good rating and then shared her complaint in the comment section.

This happens often, especially if employees are incentivized in some way for good survey scores. These employees frequently engage in what’s called survey begging, a practice where employees plead with customers to give them a good survey score and may even offer a small favor in return such as a discount or a free item. The employee may still look good, but the problem remains.

2. User Error

The customer had nothing good to say about the experience. He wrote comments complaining about poor policies and unhelpful employees. According to this customer, service at the location he recently visited was on the decline.

So why did he give a top score on the survey?

Customers are often in a hurry when completing surveys, so they accidentally mark the wrong score. Others are confused by survey questions written in a way that tricks customers into giving a higher score than they intended.

Either way, the result is a score that doesn’t match the customer’s true feelings unless you read their comments.

3. Inconsistent Perceptions

A colleague of mine, Ken Phillips, does an amazing exercise during conference presentations. He asks participants to assign a percentage to various words like these:

  • Always
  • Never
  • Sometimes

Always should be 100% but people tend to apply their own filter. The average response was about 80%.

The same goes for Never. The dictionary says that’s 0%, but people would inflate that up to as high as 30%. The percentages for Sometimes are all over the place, ranging from 30 to 80%.

The point is that customers interpret a survey scale differently. To some customers, a perfect score doesn’t equate to perfect service. It simply means good.

So you really need to dig into the comments to see how customers feel. A survey with a 10 rating might indicate a happy customer, but the comments could also indicate the customer feels there is room for improvement.

Conclusion

There’s always something to learn from analyzing survey comments.

A couple of years ago, I analyzed the Yelp reviews for one of the top-rated restaurants in my hometown of San Diego. It had hundreds of reviews and a 4.5 star rating. Yet there were still opportunities to improve.

Ratings tended to go down when the restaurant was busy. Patrons perceived a decline in service quality and weren’t happy about wait times. Advance reservations were hard to come by unless you booked a week or more ahead of time.

Fast forward a couple of years and the restaurant is still good. It still has a 4.5 rating on Yelp. But it’s not as busy as it once was.

The analysis proved that even the best businesses can’t afford to get complacent with high ratings. Smart customer service leaders look to the comments for opportunities to continuously improve.

About the author:

Jeff Toister is a consultant, trainer, and author of The Service Culture Handbook: A Step-by-Step Guide to Getting Your Employees Obsessed with Customer Service.

Start your free trial

Thematic is the easiest way to discover the best insights in feedback. Act on what matters to your customers and make an impact.

Recent posts

How to theme qualitative data using thematic analysis software
How to theme qualitative data using thematic analysis software

If you ever had to analyze customer feedback, you will know that the most difficult part is to create a perfect code frame. You need to understand the dataset, the stakeholders involved and the ideal outcomes of the analysis. You will have to iterate before settling on a solution, which

AI & NLP
How to measure customer satisfaction: the complete guide
How to measure customer satisfaction: the complete guide

Everyone says they want customers to be satisfied, but what are you actually doing to make customers happy? How do you know if you’re on the right track? How do you know if your customer satisfaction efforts make a difference? Why even aim for customer satisfaction at all? We

Churn & Loyalty
How to super-charge your Qualtrics setup with Thematic & Power BI
How to super-charge your Qualtrics setup with Thematic & Power BI

Qualtrics is one of the most well-known and powerful Customer Feedback Management platforms. But even so, it has limitations. We recently hosted a live panel where data analysts from two well-known brands shared their experiences with Qualtrics, and how they extended this platform’s capabilities. Below, we’ll share the

Customer Experience