Why you need to mine positive scores for negative themes

Your survey results look awesome. The scores are high and customers seem generally happy. Most survey responses have top-box ratings, which means the customer selected one of the two highest rating options. Nothing to worry about, right? Not so fast.

There might be a hidden danger lurking in those positive surveys. One client I worked with had as many of 5% of their top box survey scores contain negative comments. In other words, the score was good, but customers still had a few complaints.

3 ways it can happen and what you should do about it:

1. Protecting an employee

The customer was upset. Not at the person who served her. She felt the employee was friendly and knowledgeable. The customer didn’t like the policy the customer service representative had to enforce.

The customer wanted to share her feedback on the company’s customer service survey, but she didn’t want to get the employee in trouble. So she gave a good rating and then shared her complaint in the comment section.

This happens often, especially if employees are incentivized in some way for good survey scores. These employees frequently engage in what’s called survey begging, a practice where employees plead with customers to give them a good survey score and may even offer a small favor in return such as a discount or a free item. The employee may still look good, but the problem remains.

2. User Error

The customer had nothing good to say about the experience. He wrote comments complaining about poor policies and unhelpful employees. According to this customer, service at the location he recently visited was on the decline.

So why did he give a top score on the survey?

Customers are often in a hurry when completing surveys, so they accidentally mark the wrong score. Others are confused by survey questions written in a way that tricks customers into giving a higher score than they intended.

Either way, the result is a score that doesn’t match the customer’s true feelings unless you read their comments.

3. Inconsistent Perceptions

A colleague of mine, Ken Phillips, does an amazing exercise during conference presentations. He asks participants to assign a percentage to various words like these:

• Always
• Never
• Sometimes

Always should be 100% but people tend to apply their own filter. The average response was about 80%.

The same goes for Never. The dictionary says that’s 0%, but people would inflate that up to as high as 30%. The percentages for Sometimes are all over the place, ranging from 30 to 80%.

The point is that customers interpret a survey scale differently. To some customers, a perfect score doesn’t equate to perfect service. It simply means good.

So you really need to dig into the comments to see how customers feel. A survey with a 10 rating might indicate a happy customer, but the comments could also indicate the customer feels there is room for improvement.

Conclusion

There’s always something to learn from analyzing survey comments.

A couple of years ago, I analyzed the Yelp reviews for one of the top-rated restaurants in my hometown of San Diego. It had hundreds of reviews and a 4.5 star rating. Yet there were still opportunities to improve.

Ratings tended to go down when the restaurant was busy. Patrons perceived a decline in service quality and weren’t happy about wait times. Advance reservations were hard to come by unless you booked a week or more ahead of time.

Fast forward a couple of years and the restaurant is still good. It still has a 4.5 rating on Yelp. But it’s not as busy as it once was.

The analysis proved that even the best businesses can’t afford to get complacent with high ratings. Smart customer service leaders look to the comments for opportunities to continuously improve.

About the author:

Jeff Toister is a consultant, trainer, and author of The Service Culture Handbook: A Step-by-Step Guide to Getting Your Employees Obsessed with Customer Service.