Why most customer feedback analysis tools suck and how to fix this

They collect scores into pretty dashboards, but don’t actually tell what the feedback is or how to achieve customer loyalty. Customer feedback analysis tools are all the rage, but most of them suck.

If you ever left a review yourself, you will know that your score is not nearly as valuable as the review itself. Think about it: as a business, how useful is the average of 100 scores compared to 10 customer comments on why they gave you that score?

The Value of NLP

The sad truth is that most businesses collect customer comments but do not look at them. It’s just too hard.

Excel, Tableu and most other tools cannot turn text into a useful chart, and reading comments by hand is tedious and prone to bias. Clued-up customer experience professionals turn to NLP, or Natural Language Processing. It’s an area of artificial intelligence (AI) that deals with text and can transform text into usable data.

But what should you realistically expect from NLP when it comes to making sense of customer feedback?

Customer Feedback is Gold for Businesses

The most common customer success metric is NPS, a single number produced from answers to a question, “On a scale from 0 to 10, how likely are you to recommend us to your friend?”

More specific customer feedback like answers to “How would you rate our service?” or “How would you rate our quality?” still result in scores, not insights. If you want customer feedback that’s truly valuable, ask questions like “Why did you give us this score?”, “What impressed you?” and “What can we do better?”

Answers to open-ended questions contain the true nuggets companies should be mining and this is why:

1. Customer comments can explain drops and rises in the score
2. If the score stays the same, comments will add depth
3. Comments from unhappy customers contain clues as to who should be diffusing the situation and how

Understanding customer comments is not just about looking at nuances, it’s about understanding the big picture of customer sentiment and thus should not be an afterthought.

But for companies who receive thousands of comments per month reading them isn’t an option.

NLP Can Turn Chatter into Actionable Insight

NLP tools can summarize customer feedback automatically. There are open-source libraries, generic APIs, semi-automated coding interfaces and survey specific tools.

The key problem they solve is that people may describe the same thing in many different ways due to synonyms, paraphrases, misspellings, grammatical variations and even autocomplete errors.

So one person could be complaining about the fact that his “newspaper got wet on delivery”, another person saying “my paper got soaked by the rain again, impossible to read”. Capturing each issue accurately is essential because otherwise the importance of issues gets misrepresented.

NLP algorithms can figure out what is and isn’t the same issue using a variety of techniques like stemming, dictionaries, supervised categorization and language modeling. You don’t need to know how each of these work, but make sure your NLP solution these three features:

1. Link to originals for verification and action

Context is king, as they say, and without context it is hard to interpret and understand the results. You also need to be able to easily verify that the results are correct and to trace themes back to original customers for action. Imagine being able to find those implying that they may be canceling your subscription.

2. Be transparent and allow you to edit results

Transparency in how the algorithm works is important because only then one can make adjustments. Internal company knowledge is vital to decide what to track and what to ignore. Sometimes there is a wrong and right answer. “Soccer world cup” is almost always the same as “football world cup”. Other times it depends on your priorities. You may want to track “rugby world cup” separately from “soccer/football world cup” or as the same thing.

3. Provide actionable insights

Actionable data allows your business to make changes in response to customer feedback. It could be something as simple as making sure newspapers are delivered in plastic bags to keep then dry. Insightful data is about finding out that this is a problem you never knew about before.

Perhaps this is news and putting everyone’s papers in plastic bags will be the point of difference that keeps your business alive. Ideally, categories or themes returned by an NLP solution need to be specific. A customer experience manager should be able to find easy to fix issues, rather than things that are already on their radar.

Remember, an effective customer feedback analysis tool should not be about collection scores or pretty dashboards. Surfacing the reasons behind the scores is the only way you can improve the scores and turn your detractors into promoters.