You've successfully subscribed to Thematic
Great! Next, complete checkout for full access to Thematic
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

Why you need to mine positive scores for negative themes

Your survey results look awesome. The scores are high and customers seem generally happy. Most survey responses have top-box ratings, which means the customer selected one of the two highest rating options. Nothing to worry about, right? Not so fast.

There might be a hidden danger lurking in those positive surveys. One client I worked with had as many of 5% of their top box survey scores contain negative comments. In other words, the score was good, but customers still had a few complaints.

3 ways it can happen and what you should do about it:

1. Protecting an employee

The customer was upset. Not at the person who served her. She felt the employee was friendly and knowledgeable. The customer didn’t like the policy the customer service representative had to enforce.

The customer wanted to share her feedback on the company’s customer service survey, but she didn’t want to get the employee in trouble. So she gave a good rating and then shared her complaint in the comment section.

This happens often, especially if employees are incentivized in some way for good survey scores. These employees frequently engage in what’s called survey begging, a practice where employees plead with customers to give them a good survey score and may even offer a small favor in return such as a discount or a free item. The employee may still look good, but the problem remains.

2. User Error

The customer had nothing good to say about the experience. He wrote comments complaining about poor policies and unhelpful employees. According to this customer, service at the location he recently visited was on the decline.

So why did he give a top score on the survey?

Customers are often in a hurry when completing surveys, so they accidentally mark the wrong score. Others are confused by survey questions written in a way that tricks customers into giving a higher score than they intended.

Either way, the result is a score that doesn’t match the customer’s true feelings unless you read their comments.

3. Inconsistent Perceptions

A colleague of mine, Ken Phillips, does an amazing exercise during conference presentations. He asks participants to assign a percentage to various words like these:

  • Always
  • Never
  • Sometimes

Always should be 100% but people tend to apply their own filter. The average response was about 80%.

The same goes for Never. The dictionary says that’s 0%, but people would inflate that up to as high as 30%. The percentages for Sometimes are all over the place, ranging from 30 to 80%.

The point is that customers interpret a survey scale differently. To some customers, a perfect score doesn’t equate to perfect service. It simply means good.

So you really need to dig into the comments to see how customers feel. A survey with a 10 rating might indicate a happy customer, but the comments could also indicate the customer feels there is room for improvement.


There’s always something to learn from analyzing survey comments.

A couple of years ago, I analyzed the Yelp reviews for one of the top-rated restaurants in my hometown of San Diego. It had hundreds of reviews and a 4.5 star rating. Yet there were still opportunities to improve.

Ratings tended to go down when the restaurant was busy. Patrons perceived a decline in service quality and weren’t happy about wait times. Advance reservations were hard to come by unless you booked a week or more ahead of time.

Fast forward a couple of years and the restaurant is still good. It still has a 4.5 rating on Yelp. But it’s not as busy as it once was.

The analysis proved that even the best businesses can’t afford to get complacent with high ratings. Smart customer service leaders look to the comments for opportunities to continuously improve.

About the author:

Jeff Toister is a consultant, trainer, and author of The Service Culture Handbook: A Step-by-Step Guide to Getting Your Employees Obsessed with Customer Service.

Ready to scale customer insights from feedback?

Our experts will show you how Thematic works, how to discover pain points and track the ROI of decisions. To access your free trial, book a personal demo today.

Recent posts

What is Thematic Analysis? And How To Do It (Manual vs. AI)
What is Thematic Analysis? And How To Do It (Manual vs. AI)
Members Public

Discover the power of thematic analysis to unlock insights from qualitative data. Learn about manual vs. AI-powered approaches, best practices, and how Thematic software can revolutionize your analysis workflow.

Customer Experience
How Watercare drives customer excellence with VoC and Thematic
How Watercare drives customer excellence with VoC and Thematic
Members Public

When two major storms wreaked havoc on Auckland and Watercare’s infrastructurem the utility went through a CX crisis. With a massive influx of calls to their support center, Thematic helped them get inisghts from this data to forge a new approach to restore services and satisfaction levels.

Customer Journeys
How to theme qualitative data using thematic analysis software
How to theme qualitative data using thematic analysis software
Members Public

Become a qualitative theming pro! Creating a perfect code frame is hard, but thematic analysis software makes the process much easier.