You've successfully subscribed to Thematic
Great! Next, complete checkout for full access to Thematic
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

Product Feedback: Ultimate Guide - How to Gather, Analyze & Share

Alyona Medelyan
Alyona Medelyan

How insights and analytics professionals can turn overwhelming volumes of customer feedback into a continuous product feedback loop.

In this guide
Introduction
Part 1: Different types of product feedback and how you can benefit from them
Part 2: How to improve your product feedback strategy
Part 3: Analyzing product feedback
Part 4: How to share product feedback with others
Conclusion

Introduction

Whether your product is a vacuum cleaner, a mobile app, or a SaaS solution, product feedback should drive decision-making. Sadly, this feedback is often not analyzed and when it is, the insights from it are rarely acted upon.

I spent the last 5 years building Thematic, a customer feedback solution. After speaking to hundreds of customer analytics and insights professionals about their feedback strategy, here’s what I’ve learned:

  1. Companies have unprecedented volumes of feedback sitting on their servers, and they continuously ask for more. It’s easy to ask for feedback!

  2. However, there are no great best practices for what to do with that feedback. So each team invents their own approach, with varying degrees of success.

  3. Insights teams that translate feedback into action do so by turning an overwhelming flood of feedback from disparate channels into a product feedback loop that aligns people, processes and technology in a way that is responsive to what customers are saying. 

product feedback loop


We’ll first discuss the types of feedback you currently have or may want to collect. Next, you’ll learn how to think about your feedback strategy: stakeholders, tools and process. In Part 3, we’ll do a deep dive into feedback analysis. In Part 4, you’ll learn how to build a product feedback loop that enables your stakeholders, managers and executives to make customer-focused decisions. 

Part 1: Different types of product feedback & how you can benefit from them

First, let’s break down the types of feedback there is and the channels used to gather it.

Product feedback can be solicited or unsolicited. Feedback is solicited when you specifically ask others to provide it. For example, you sent a survey or you ask for feedback during an interview. Feedback is unsolicited when others provide it on their own accord. For example, when they leave a review online or message your company’s tag on Twitter. 

To get a full and accurate picture of what others think of your offering, and the features they desire, you need to include both types of feedback from as many channels as possible.

Solicited product feedback

Solicited feedback can be quantitative (ratings, scores) if you are asking rating questions in surveys, or qualitative (text) if you are asking open-ended questions or running customer interviews. 

You could be soliciting feedback from your existing customers or target people via a panel. Panels give you access to people who fit your pre-defined characteristics. Feedback could be on both your own and your competitor’s products. 


Some of the targeting options on SurveyMonkey Audiences 

Customer surveys

Customer surveys are useful to keep a pulse on things, to get continuous feedback as you are improving your product, or to quickly get feedback on a specific aspect of your product. 

  • The most popular customer feedback surveys are Net Promoter Score (NPS) and Customer Satisfaction (CSAT) surveys, which are typically sent daily, monthly, quarterly, once or twice a year, depending on how many customers you have and how quickly you iterate. 
  • A customer survey that’s gaining popularity is a Product Market Fit survey. Its key question is “How would you feel if you could no longer use the product?”, and it’s particularly useful for new disruptive products. 
  • While the first two are continuous (or tracking) surveys, there are also one-off surveys to capture answers to specific questions in an efficient way.

If you are in a highly competitive space, companies like ROIRocket provide double-blind NPS surveys via panels to get deeper insights. 

Advantages: Easy to setup, cheap, scalable with the right survey analysis software. Most aren’t anonymous and customer metadata (spend, behavioural analytics and demographics) can be linked to feedback. 

Disadvantages: Survey fatigue (since everyone is surveying), low response rates, bias towards those who respond to surveys, varying level of insights, depending on how engaged are your customers or the quality of your survey.

How to best benefit from customer surveys: Make sure you have a powerful system for the analysis of survey feedback in place. Once you have it, if you have a specific question, always check this feedback first. It will be the richest feedback due to scores, structured open-ended questions and customer metadata. It can also be a starting point for deeper customer interviews.

Customer interviews, panels & focus groups

In surveys, customers often don’t expand on why they care about something and how specifically you should address their issues. In customer interviews, you can dig as deep into an issue as you need.

Focus groups are similar to customer interviews, except multiple people participate at once. These can be your pre-selected customers or you could use a panel. They can be done virtually, and automated using solutions like Remesh

Advantages: Great for deep insights into a specific issue or when you aren’t sure which questions you should be asking. Particularly useful for discovering unknown unknowns (things you didn’t know you didn’t know) about new products or new product features.

Disadvantages: Not scalable, expensive to run and setup. Difficult to analyze the results. The outcomes can be inconclusive or biased towards either the sample of participants, or the people moderating and analyzing results. 

How to best benefit from customer interviews: Only interview those who truly care about what you are researching. Don’t just email any customers to say “let’s chat about X”, find customers who commented on “X” in other channels of feedback (surveys, support).

Unsolicited product feedback

Unsolicited feedback is typically qualitative, text-based and requires some effort to gather, categorize and analyze. But it’s worth it.

Communities and forums

Communities and forums can be either solicited or unsolicited, depending on how they are setup and run. For example, if a community emerges on a public forum like Reddit or Discourse, the feedback is unsolicited. If the company creates a community using a tool like UserVoice, then the feedback is solicited. In any case, this can be a source of valuable feedback.

Advantages: Often unmoderated, unfiltered and therefore the most truthful. In large communities, members can support each other and reduce support costs. Public content in communities can be indexed by search engines and benefit marketing.

Disadvantages: Mostly unstructured. Hides the identities of users. Difficult to create from scratch, since people choose to engage and participate. Might not work for all types of products. Difficult to find useful insights, because people tend to go off topic. The insights can be biased towards those who are most active in the forum instead of those who use the product the most. 

How to best benefit from this feedback: Start with existing communities. Know where you and your competitors’ users hang out and provide their feedback. 

Contact centers, support & complaints

Today when customers need help, they have many options:

  • Call a help line or a call center,
  • Use a live chat or a bot on your website,
  • Raise a support ticket,
  • Email or tweet a complaint.

These requests are a great source of product feedback. They don’t just show what’s not working, but also which features customers care about.

Example of unsolicited feedback

Example of unsolicited feedback on Twitter

Advantages: You already have this data as part of support & customer service operations. Customer metadata is available for each piece of feedback via ids, unless it’s social media requests.

Disadvantages: Feedback may refer to not just product, but aspects of service, which means a solution for filtering relevant feedback is required. Can be siloed and inaccessible, especially if it’s in the form of emails or audio.

How to best benefit from this feedback: Don’t only respond to support tickets. Use this data strategically. Know how to prioritize fixes, which means that you won’t just improve the product but also reduce the support costs.

Online reviews

Occasionally reviews can be solicited by companies, but in most cases, customers use existing platforms to provide reviews of products they care about. There are dozens and dozens of websites which capture reviews for different types of products, and you should use the ones that are most relevant to you.

Some examples:

  • Google play and iOS AppStore for app reviews 
  • Trustpilot for B2C products 
  • G2Crowd and Capterra for B2B products
  • Amazon and other e-Commerce platforms for review of physical (and digital) products
  • Google or Tripadvisor for reviews of hotels, restaurants, and sights
  • Booking.com for reviews of hotels
  • Facebook and Instagram for reviews of local businesses

Advantages: Some structure (they have scores!), feedback of both your and your competitors’ products. 

Disadvantages: Might be fake, biased towards what has been solicited by others. Often short and not actionable. Lack of knowledge of who left the feedback.

How to best benefit from this feedback: Find a solution that unifies and gathers reviews in one place (they can be scattered across many sites) and provides analysis for easy comparison of you vs. competitors. Ideally with trends over time.

Tip: Don’t just look at your own reviews: See what your competitors’ customers are saying! Incorporating their reviews into your product feedback loop can surface competitive gaps to fix or exploit.

Part 2: How to improve your product feedback strategy

Let’s talk about what to do with all that feedback!

Usually you don’t design a product feedback strategy from scratch. Typically, you already have something in place, which may be either incomplete, outdated or not scalable. Product feedback strategy is like pricing strategy: It needs to evolve with the business.

As you scale, you will need to hire Product Operations specialists who make sure that feedback is collected, analyzed and shared in an efficient manner using the right product feedback platforms. They also ensure there is an effective product feedback loop, and the feedback is considered at all life cycle stages of the product development.

Choose the right product feedback tools

What do you think is the most commonly used product feedback tool?

Correct, it’s a spreadsheet. 

I’ve seen mega Google Sheets used to gather and analyze feedback, edited by hundreds of people at once. Some are even hooked up via Zapier to Google Mail to send emails to customers once their feedback is actioned on. Can you imagine what might happen if such a spreadsheet gets into the wrong hands?

At the very least, a spreadsheet can be a common place that unifies feedback across many different channels. Some use AirTable for this purpose.

The next iteration up is a data warehouse like AWS Redshift which feeds both feedback and other customer data into Tableau to create complex dashboards.

We’ve also seen company wikis, like Confluence and Notion, being used for capturing feedback in a single place. Some companies use Jira boards for submitting customer requests for product features.

Specific feedback SaaS offerings we’ve seen in the market are ProductBoard, Aha! and DoveTail. They have features for tagging feedback by hand with commonly recurrent themes. The disadvantage is that feedback is limited to customer interviews or inbound requests. A lot of unsolicited feedback such as online reviews and support tickets either isn’t captured, or needs to be transferred over manually.

All of these solutions are created to capture, then organize and analyze by hand. But as you can imagine, if you include high volume sources such as live chat or large-scale NPS surveys, it’s going to be hard to tag every single piece of feedback manually, and it can be biased too!

So, there is another set of solutions, like Thematic, who focus on not just gathering but also analyzing the feedback using a combination of Natural Language Processing and human input to ensure that the analysis is accurate and relevant. These types of solutions are often described as ‘Customer Insights’ platforms, or more recently, Unified Data Analytics. Below I’ll share how you can ensure that your feedback is analyzed effectively.

Design an effective product feedback loop

Aggregating feedback in one place isn’t enough. You need a process that turns feedback into actionable insights to drive the actual product development and improvement. And make sure that customers are aware of the changes you are making. Many people overlook this!

Here is an example of an internal product feedback loop: You gather feedback by showing your product to users, or using existing feedback sources (both solicited and unsolicited). You then turn feedback into insights, identifying drivers of satisfaction, loyalty, and differentiation from competitors. You then share these insights with the people responsible for product decisions based on this data, and continue to gather feedback by showing your updated product to users.

User feedback to product decisions

How to Build a Startup by Sam Altman

Customer feedback also helps create more scalable operations. For example, it has insights into why customers might be contacting your costly contact center. By fixing issues, you can continuously reduce the number of inbound calls and support requests.

Do not forget to close the loop with the customers! 

Who wants to shout feedback into the void? 

Tell customers how you’ve addressed their feedback. You will create meaningful relationships and customer goodwill to give you more feedback later as you iterate.

Before improving your company’s product feedback strategy, consider these questions:

  • Who needs to be involved in gathering feedback and how? 
  • What stakeholders are involved in product decisions, what metrics are they responsible for, and what data (such as churn, growth, or operational costs) do they use to make decisions?
  • How will you tie customer feedback to those metrics and data sources?
  • How will you share insights with others? And how often?
  • What’s the best medium and frequency for notifying our customers about changes we’ve made as a result of their feedback? 

Part 3: Analyzing product feedback

Analyzing customer feedback in a consistent and accurate way is difficult, and the reason why I co-founded Thematic. Here are some of the challenges that you might not have considered:

  • It takes a long time to code feedback by categories, tags or themes: between 30 min and 1h for just 100 pieces of feedback, if done manually. You will also need to manage the list of categories, tags or themes because they tend to be either too generic or too difficult to manage, as people create more and more by hand. 
  • Just as product analytics is only one part of the picture, so is customer feedback! You will need to link the two in a meaningful way, and include other relevant metrics and data sources. 
  • Visualizing this data is tricky, since customers are likely to mention between 50 and 500 distinct themes in their feedback, depending on the product, and there isn’t an easy chart for this even in Excel or Tableau. 

So, how can you practically solve these challenges?

Here is our 4-step approach:

Step 1: Turn qualitative feedback into quantitative data

You need to organize feedback based on what customers are actually saying. One might say “your product has very poor search functionality”, another might say “I couldn’t find what I was looking for”. Tag these with “improve search”. If you are tagging feedback manually, keep track of different variations of how a tag might be called, so that your colleague doesn’t create a tag “poor search”. 

There are different approaches to organizing and automating feedback analysis, which we’ve covered in previous posts. Depending on the size of your company, one of these might really resonate:

If you choose to analyze feedback manually, beware of bias! It’s easy to see what you’re looking for, given current priorities, and miss more impactful insights into what actually matters to customers. 

Tip: Don’t create tags in advance of reading feedback. Whether you use a manual or an automated solution, make sure that your themes and tags emerge from the data itself (in Market Research this is referred to inductive vs. deductive coding), otherwise, you’ll miss critical insights.

Step 2: Link product feedback to behavioural and demographic customer data

Not all feedback is equal. So, simply looking at common themes by volume won’t provide the most accurate insights. Make sure that you link each piece of feedback to as much information (such as demographic, behavioural, CRM and operational data) as possible that’s relevant to how your company views their customers:

  • New or old customer, existing or churned?
  • What did they buy from you and who is their account manager?
  • How much do they spend annually with you and how often do they use your product?
  • What are the NPS ratings they’ve given you (or any other ratings/scores for that matter)?
  • Where are they located? What type of location is it?
  • How did you acquire them and how much did it cost you?
  • What’re their demographics (only if it’s appropriate to what you sell)?

Tip:  Consider all aspects of who your customers are. We recently did an exercise that captured data for 1000 companies in our database and learned that many attributes of our customers weren’t differentiative. So we dug deeper and realized that our customers have an additional attribute we hadn’t considered: They might be selling services, software products or hard products (that are hard to change), or a combination of these. Once we cut the data by this attribute a new insightful pattern emerged: Those in the latter category gain the most benefits from Thematic.  

Step 3: Visualize data effectively

There are two most common ways of visualizing product feedback and I’m not a fan of either. The first version recently got featured in a document that landed on my desk: the quote! 

This is an equivalent of a vanity metric in product analytics.

The second one is our not so beloved word cloud. 

Word cloud maker

An example of a word cloud maker in Qualtrics

These two methods of visualizing feedback are neither effective, nor accurate, and are too easy to either dismiss or take out of context.

It doesn’t have to be this way! 

We’ve written an article on alternatives to word clouds and better ways of visualizing feedback

To summarize its key points: 

  1. Use themes instead of keywords
  2. If in doubt, use a bar chart and use relative numbers rather than absolute numbers. 

Tip: The most effective visualizations are those that answer a specific question. We’ll cover the best questions to ask using product feedback in the next section.

Step 4: Be thorough and consistent over time

Oftentimes feedback analysis is done in spurts that are driven by urgent projects and deadlines. When this happens only a portion of customer feedback is analyzed. Only certain themes are coded, and the result is a narrow perspective. Because fire drills like this are so painful, the results of these ad-hoc analyses are often pushed aside once the crisis is over, never to be reviewed again.

The problem with this approach is that it wastes valuable time, creates disparate snapshots from different angles that are rarely complimentary or useful for future projects. Using feedback to validate any decisions made on that data in the future, will require another disjointed effort, for which bandwidth is unlikely.

The payoff for consistency and thorough theme identification is that it enables an entirely different class of insights to emerge, with speed and simplicity. Ad-hoc questions (“Did it work?”, “What was the impact?”) become easy to answer, and insights on how feedback changes over time in response to product or market changes become possible. 

Tip: If you’re stuck in spreadsheets and don’t think you could justify the investment in a solution (like Thematic!) that would automate the analysis process, check out our post on The True Cost of Not Understanding Customer Feedback. 

Part 4: How to share product feedback with others

Who should care about product feedback? We think everyone! 

Regardless of what you do in the company, you need to understand your customer. 

First, let’s look at the goals of why your colleagues might need product feedback. Second, let’s see how you might share feedback to support those goals.

Align insights with stakeholder goals

Product feedback goals

If gathered, analyzed & visualized correctly, feedback should be used in the same manner other data is used throughout the company. It’s goals are to:

  1. Test hypotheses, e.g. we think that users in category A are more interested in B
  2. Verify assumption, e.g. I believe our customers value C above all else
  3. Evaluate experiments, e.g. do customers see our D as positive after we changed it?
  4. Discover unknown unknowns, e.g. an emerging competitor or a sudden bug
  5. Find actionable insights, e.g. when customers struggle with feature E
  6. Align teams, e.g. make sure that everyone knows what drives KPIs such as NPS

You can see that pretty much every team in the company would benefit from having the supporting data for these goals. In theory, this sounds easy. In practice, however, there are a few hoops to jump through. 

In order for insights to be used, they need to be “sold” to internal customers AKA stakeholders. That is, they need to be convinced that what you are sharing with them will not only help the company and the customer, but that the insights will allow them to do their job more successfully. 

Tip: Our webinar “Proving the ROI of CX” has 5 keys for turning insights into action and influencing stakeholders. Check it out for ideas on how to turn feedback into insights that will matter to stakeholders at all levels of the organization. 

Sharing feedback with others

Insights also need to be “delivered” in ways that make them easy to “consume.” In some companies, that means sharing a slide deck, doing a weekly print out and round of desk drops. In others, email updates and dashboards work best. 

Tip: Do an internal customer journey map. List out the different stakeholders involved in turning your insights into action. Then in columns, list their priorities, metrics that matter most, ways and times they consume information. Then create another column to see whether you produce the insights they need in a format and cadence that will be helpful. 

Here are a few examples of ways that feedback can be shared: 

Lifecycle Events

Every organization has key weekly, monthly, quarterly and yearly cycles that center around meetings. From board meetings to sprints, these are your key windows for delivering feedback for maximum impact. Identify the ones where decisions are made in your organization, and ask contributors what would be helpful for you to share at these meetings. 

Ad-hoc queries

Several times a month, the product operations, research or insights team will receive questions that support the goals of various teams. Typically, they want to test a hypothesis or evaluate an experiment they run. Their questions might be:

  • I think I should work on X. Is X a problem? How badly does it affect our key metric, how many customers talk about this, and what’s the overall trend?
  • We have done some work to Y, did customers notice? Has the impact of Y on our key metric increase?

Make sure your visualization toolkit includes charts and graphs to support these questions.

Regular updates to align the team

Once a month or a quarter, most companies make sure to review their key metrics. Whether you are using a dashboard or a slide deck that captures the key points of the update, make sure to answer the following questions:

  • What’s our key metric and what was it last time?
  • If it went up, why? If it dropped, why?
  • What impacts this key metric overall, and for our key customers?
  • If you have departments, teams or location, which one has the best results and why?
Visualizing product feedback
How Thematic visualizes feedback on why a score dropped

Customer in the room

For any product roadmap meetings, make sure customers are in the room. Sure, you won’t be able to have the actual customers be present. But given the multitude of ways the customer has already provided their feedback to you, if it’s all in one gathered place, make sure your team can look up whatever they are addressing right at that moment and find more nuance. Find users who provided feedback on that topic and follow up with them to learn more.

Amazon’s Jeff Bezos has an empty seat at his company meetings that is designed to represent the voice of the customer. A symbolic gesture to assure that the customer is always represented and considered in decision making.

DIY / Self Service

One of the things I love hearing from our customers is that they no longer feel like a bottleneck for questions and answers. When product feedback is analyzed thoroughly and consistently and is made easily accessible to anyone in the organization, it takes on a life of its own. 

Seeing multiple stakeholders across the company using a central source of customer feedback just feels right.

In conclusion

Product feedback has traditionally been seen as anecdotal, unscientific and mushy.

We hope this article has shown that it doesn’t have to be this way. 

With huge volumes of feedback most companies already collect and can gain from online sources, there shouldn’t be an excuse for not knowing who your customers are and what they like and dislike. 

With the latest technology, tools and solutions, analysis and sharing can be streamlined. You can answer questions using data within minutes and not days.

The most amazing thing: If we hear and act on what our customers are telling us, everyone will benefit! Who doesn’t want to see better customer retention, marketing via word of mouth and, ultimately, more profitable business.

Feedback AnalysisProduct Experience

Alyona Medelyan

Alyona is one of the founders of Thematic. Her love of writing comes from spending years of publishing papers during her PhD. She believes customer feedback should be top of mind at every company.