12 big mistakes when collecting and analyzing customer feedback

Are you getting the most out of your customer feedback? How can you ensure your feedback will transfer to solid actionable insights that make a difference to your business?

Here, we share some common mistakes we’ve seen companies do when collating and analyzing feedback – make sure you’re not committing these faux pas. Once you’ve got the deep insights that are also accurate, they can transform your business strategy.

The biggest mistakes you don’t want to be doing:

1. Don’t ask the wrong questions

If you run an NPS survey, don’t just ask the “Why did you give us this score?” question. If you have a high number of Promoters, you’ll end up with comments such as “great product”, “great company”, which aren’t very actionable. Instead, always ask all of your customers: “What can we improve?”. This will give you valuable suggestions on how to keep your promoters for the longer term to achieve higher customer satisfaction.

2. Don’t ask for ratings of specific attributes

Don’t ask people to rate/comment on various attributes of your business. Often we see companies ask questions like “Please rate X”. For example, for a supermarket it could be “Please rate our meat department”. While this may collect some comments that can be useful to the meat department, it won’t tell you whether the quality of the products and services at the meat department actually matters to customers of the supermarket overall.

3. Don’t be too vague

The “Any comments” question is another common culprit. Customers aren’t sure what to write there. Instead, always be specific. If you’d like to ask for improvement suggestions, ask “What can we improve?” or “What should we do differently?"

4. Don’t be impersonal

Not being personal enough is a common mistake, especially for B2B surveys. Always write a personal note why they should do it. Make the email sound as personal as possible by using their first name and mentioning their last interaction with your company.

5. Don’t send surveys that are too long or complicated

Keep it short! Your survey should take no more than 2-3 minutes of their time, however if a customer would like to talk in depth, offer a way of contacting you.

6. Don’t forget to mention how much you value their feedback

Not letting them know how much you value the feedback and what you’ve done after collecting the feedback last time, will make respondents feel like their response is not important or won’t make a difference. A good place to share this is in the invitation email, or in the thank you note.

7. Don’t just focus on detractors

If you’re analyzing an NPS survey and only looking at your detractors, you’re doing it wrong. Something will have gone completely awry by the time someone gives you a 0 or 2. Don’t forget that you can get valuable feedback from passives or promoters as well, which will appear in their free-text responses. Instead of constantly putting out fires, get a strategic view of what’s working and what’s not, for all of your customers. Learn who your key promoters are and why. Learn what the passives aren’t seeing. In order to do this, all of your responses need to be analyzed consistently over time. Not just the ‘bad’ ones!

8. Don’t manually code just a subset of the data

If you manually code say 20% of your data this can lead to a situation where you can’t segment the data properly to find reliable, statistically significant insights, especially in smaller datasets. For example, when creating pivot tables which can show how popular a particular theme is across demographics or marketing personas. And without this, you may struggle to get to the core of the problem.

9. Don’t keyword spot

Dabbling in keyword spotting is a very common mistake we often see which means your data simply won’t be analyzed completely. No matter how smart you may be, it’s incredibly difficult for anyone to guess the ways in which others might refer to one and the same thing. Even linguists struggle to come up with all synonyms of the theme “too expensive”. We’ve counted hundreds of ways people might talk about it. From single keywords such as “overpriced”, to negated synonyms like “not affordable” and colloquialisms like “rip off” and so on. In one dataset, the keyword spotting approach missed the second most common theme and placed it as the 8th most common theme, due to the variety of ways people mentioned it.

10. Don’t just focus on the known issues

At one point in time, Facebook had a 2.5 star average review rating on the App store. The product team thought of the possible things that could go wrong and categorized the feedback into “Speed”, “Ease of use” and “Missing features”. What they failed to do, was find the crucial number one theme in the feedback: “Candy Crush saga”. When used in conjunction with Facebook, the Candy Crush Saga app caused crashes and freezes for Facebook users. The moral of the story: Use an inductive approach to find common themes (see our post on this topic).

Even the most popular apps struggle with poor reviews.

11. Don’t focus on just volume

Make sure to look at impact, too. Often we see that companies look at their most common themes, but ignore the scores associated with these themes. For example, if a lot of passives talk about how easy it is to work with you, but your promoters don’t even mention it, it means that the ease of working with you isn’t actually a driver of NPS. Here at Thematic, this is how we calculate impact from Net Promoter Score data.

12. Don’t forget to ask yourself the strategic questions

When analyzing, don’t forget to ask:

• How can we turn passives into promoters?
• Are they aware of the key elements that drive promotion? Sometimes customers haven’t discovered all the features of a product, or they weren’t properly onboarded.
• How can we reduce the number of detractors?
• Are there specific customer segments that care the most about the specific complaint? Take care of those first.
• Which agent/representative/store/subsidiary has the best satisfaction score? Which one has the worst? Why?
• Has action X that we’ve done in previous month helped with Y?

If you’ve collected the right data and analyzed it completely and consistently, without biases, you’ll find the answers to these questions, and thereby ensure that you have actionable insights.