You've successfully subscribed to Thematic
Great! Next, complete checkout for full access to Thematic
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

12 big mistakes when collecting and analyzing customer feedback

Are you getting the most out of your customer feedback? How can you ensure your feedback will transfer to solid actionable insights that make a difference to your business?

Here, we share some common mistakes we’ve seen companies do when collating and analyzing feedback – make sure you’re not committing these faux pas. Once you’ve got the deep insights that are also accurate, they can transform your business strategy.

The biggest mistakes you don’t want to be doing:

1. Don’t ask the wrong questions

If you run an NPS survey, don’t just ask the “Why did you give us this score?” question. If you have a high number of Promoters, you’ll end up with comments such as “great product”, “great company”, which aren’t very actionable. Instead, always ask all of your customers: “What can we improve?”. This will give you valuable suggestions on how to keep your promoters for the longer term to achieve higher customer satisfaction.

2. Don’t ask for ratings of specific attributes

Don’t ask people to rate/comment on various attributes of your business. Often we see companies ask questions like “Please rate X”. For example, for a supermarket it could be “Please rate our meat department”. While this may collect some comments that can be useful to the meat department, it won’t tell you whether the quality of the products and services at the meat department actually matters to customers of the supermarket overall.

3. Don’t be too vague

The “Any comments” question is another common culprit. Customers aren’t sure what to write there. Instead, always be specific. If you’d like to ask for improvement suggestions, ask “What can we improve?” or “What should we do differently”?


Example of how Glassdoor asks for feedback.

4. Don’t be impersonal

Not being personal enough is a common mistake, especially for B2B surveys. Always write a personal note why they should do it. Make the email sound as personal as possible by using their first name and mentioning their last interaction with your company.


5. Don’t send surveys that are too long or complicated

Keep it short! Your survey should take no more than 2-3 minutes of their time, however if a customer would like to talk in depth, offer a way of contacting you.

6. Don’t forget to mention how much you value their feedback

Not letting them know how much you value the feedback and what you’ve done after collecting the feedback last time, will make respondents feel like their response is not important or won’t make a difference. A good place to share this is in the invitation email, or in the thank you note.

7. Don’t just focus on detractors

If you’re analyzing an NPS survey and only looking at your detractors, you’re doing it wrong. Something will have gone completely awry by the time someone gives you a 0 or 2. Don’t forget that you can get valuable feedback from passives or promoters as well, which will appear in their free-text responses. Instead of constantly putting out fires, get a strategic view of what’s working and what’s not, for all of your customers. Learn who your key promoters are and why. Learn what the passives aren’t seeing. In order to do this, all of your responses need to be analyzed consistently over time. Not just the ‘bad’ ones!

8. Don’t manually code just a subset of the data

If you manually code say 20% of your data this can lead to a situation where you can’t segment the data properly to find reliable, statistically significant insights, especially in smaller datasets. For example, when creating pivot tables which can show how popular a particular theme is across demographics or marketing personas. And without this, you may struggle to get to the core of the problem.

Example of a Pivot table showing an insight: “14% of the Midwest customers love the brand, whereas less than 4% of the South customers do”.

9. Don’t keyword spot

Dabbling in keyword spotting is a very common mistake we often see which means your data simply won’t be analyzed completely. No matter how smart you may be, it’s incredibly difficult for anyone to guess the ways in which others might refer to one and the same thing. Even linguists struggle to come up with all synonyms of the theme “too expensive”. We’ve counted hundreds of ways people might talk about it. From single keywords such as “overpriced”, to negated synonyms like “not affordable” and colloquialisms like “rip off” and so on. In one dataset, the keyword spotting approach missed the second most common theme and placed it as the 8th most common theme, due to the variety of ways people mentioned it.

10. Don’t just focus on the known issues

At one point in time, Facebook had a 2.5 star average review rating on the App store. The product team thought of the possible things that could go wrong and categorized the feedback into “Speed”, “Ease of use” and “Missing features”. What they failed to do, was find the crucial number one theme in the feedback: “Candy Crush saga”. When used in conjunction with Facebook, the Candy Crush Saga app caused crashes and freezes for Facebook users. The moral of the story: Use an inductive approach to find common themes (see our post on this topic).

Even the most popular apps struggle with poor reviews.


11. Don’t focus on just volume

Make sure to look at impact, too. Often we see that companies look at their most common themes, but ignore the scores associated with these themes. For example, if a lot of passives talk about how easy it is to work with you, but your promoters don’t even mention it, it means that the ease of working with you isn’t actually a driver of NPS. Here at Thematic, this is how we calculate impact from Net Promoter Score data.

12. Don’t forget to ask yourself the strategic questions

When analyzing, don’t forget to ask:

  • How can we turn passives into promoters?
  • Are they aware of the key elements that drive promotion? Sometimes customers haven’t discovered all the features of a product, or they weren’t properly onboarded.
  • How can we reduce the number of detractors?
  • Are there specific customer segments that care the most about the specific complaint? Take care of those first.
  • Which agent/representative/store/subsidiary has the best satisfaction score? Which one has the worst? Why?
  • Has action X that we’ve done in previous month helped with Y?

If you’ve collected the right data and analyzed it completely and consistently, without biases, you’ll find the answers to these questions, and thereby ensure that you have actionable insights.

Download Thematic eBook ‘How to leverage AI to improve Customer Experiance’

Keep reading

Text Analytics Software – How to unlock the drivers behind your performance

Text Analytics Software – How to unlock the drivers behind your performance

Many organisations, large or small, gather customer feedback to improve their CX efforts and ultimately their bottom line. But gathering feedback alone can’t make much of a difference. We need to analyze our feedback to discover insights that inspire us to drive action at our organisations. Enter; Text

Melanie Disse
Melanie Disse
Thematic Analysis Software: How It Works & Why You Need It

Thematic Analysis Software: How It Works & Why You Need It

You know you need to collect customer feedback. You send out surveys and read reviews. But once you have customer feedback in-hand, what do you do with it? How do you identify common themes in customer responses — and turn that into actionable business insights?   You could spend long

Alyona Medelyan PhD
Alyona Medelyan PhD
10 insider customer experience tips according to Shep Hyken

10 insider customer experience tips according to Shep Hyken

Shep Hyken knows a thing or two about customer experience. He needs no introduction in industry-wide circles, but in case you’re not familiar with his name: Shep is an award-winning customer service and customer experience speaker, a New York Times and Wall Street Journal best-selling author, A.K.A

Agi Marx
Agi Marx
Churn & Loyalty