Product validation: Using customer feedback analytics to build a roadmap

Product validation is the process of determining whether a product concept or idea has the potential to meet the needs of customers, consumers, and stakeholders.

It’s discovering whether your new or existing product will be able to generate traction in the real world, and whether your target market will be as excited about it as you are.

This makes the product validation process a key part of product development, whether you are rolling out something new or assessing an existing project. Either way, you’ve got a dream, and you know what you’d like to bring to market.

We’ll go into how to validate your product in more detail later. But first, let’s step back and ask a few basic questions:

• When should you validate your product?
• What are the types of product validation?
• Can I do product validation on a budget?
• And, of course, why is product validation important?

When should you validate your product?

Validate your product early in the planning process. This applies to new products, and to any new features or changes in product direction for existing products.

Validating your product will ideally be the third iteration of validation that you run. Before you even get to the drawing board, you should be validating your market, studying the specific demographic you aim to serve and learning their real and felt needs.

It’s when you are validating your market that you’re likely to hit on a specific problem you’d like to solve. Then you’ll begin validating the problem: checking and double-checking it’s one your target consumer feels is worth solving.

So if you’ve done it right, validating your product shouldn’t come up with any huge surprises. Do it as soon as you’ve got a concrete idea of what you’d like to create, and when you’re still flexible with almost everything about your startup.

What are the types of product validation?

There are two types of product validation that you’ll need to explore before you’re ready to begin full-scale production or software roll-out. You could make that three, as user validation should focus, separately, on two important facets of your hypothetical product: usability and desirability.

• Feasibility testing
• User validation: Usability & desirability

Feasibility testing

Feasibility testing explores the logistics of bringing your product to market. For a physical product, this includes cost analysis and manufacturing limits.

For digital products, you’ll focus on whether your idea is buildable: is the technology currently available to make it a reality? Together with your engineering team, you’ll explore various venues and determine how feasible your plan actually is.

User validation

Usability

Usability and desirability testing involves working with your UX designers to put together a prototype of your product. While the back end of your system may not be there yet, the front end should be basically intact. Start with internal team testing, and when you’re happy, extend this to sample users from your target population.

Have these users explore your UI, complete with a simulated backend, and then get feedback on the user experience. Is it intuitive, manageable, and user friendly? How high is the learning curve, and will your target user be able to get results with a reasonable amount of effort?

Desirability

Once you’ve determined your product is usable you’ll need to focus on the second question - is it desirable? Do prospective users or potential customers want what you’re offering enough that they’ll sign up, pay for it, or jump through whatever hoops are required?

The information you glean here can also be based on feedback and surveys, but if you want accurate data, you’ll need to be careful not to ask leading questions. Essentially, you want to find out if your product addresses and mitigates a pain point - a recurrent, keenly felt frustration or annoyance - or whether you’ve come up with a solution to a problem that never actually existed.

Can I do product validation on a budget?

While you don’t want to skimp on product validation, as doing it right at this early stage can make the difference between eventual failure and success, it is possible to validate on any budget. This is something accessible to anyone, from a lean startup to a multi-million dollar enterprise.

Your major costs will be:

• prototyping,
• collecting user feedback,
• and analysis.

Prototyping costs have gone down significantly in recent years. If you use an AI-based feedback analysis tool like Thematic you will end up saving countless hours compared to having your team run the analysis by hand, using old school methodology.  

If product validation is completely impossible for you at this stage, you can use alternative ways to explore desirability or market appeal.

For instance, consider creating a landing page for email signups or publicizing a waitlist for a private beta. This type of market research doesn’t have to cost a lot, and you’ll still be able to collect important information on customer needs that can help you create a great product.

Check Google Trends to determine market demand, or run surveys among potential users. A crowdfunding campaign is another effective way of determining whether your customer base cares about your product enough to pay for it. You’ll get the added benefit of raising funds at the same time.  

None of these budget validation methods addresses the question of usability, so you’ll still need to do prototype-based testing whenever it becomes feasible for you.

Why is product validation important?

Is it worth spending your meager budget on something so theoretical? The answer is a definite yes: product validation is the single best determiner of your success, and the most effective way to ensure you begin with the odds stacked in your favor.

We’ve touched upon the why briefly in our intro: product validation matters because it gives you a peek into the future. It allows you to catch a glimpse of what results you’re likely to see before you waste capital on product development for something no one wants to buy. There’s a lot more going on behind that, but here are the cliff notes:

• Eliminate false starts - there’s nothing worse than bringing a product to market only to find that you missed the mark. Product validation stops you going down dead ends, so you can use your time and energy on plans that have real potential.
• Minimize costs - the shortest road between two points is a straight line, and when you’re talking about software rollout the same goes. The sooner you learn what your final product should look like, the more efficiently you can get there.
• Provide concrete, fact-driven data for stakeholders and potential investors - There’s nothing potential investors like better than concrete facts and well-run research on product-market fit. This is a big one for getting buy-in from stakeholders and executives on decisions for existing products too.

Basic Product Validation: From Product Idea to Roadmap

Now that you know product validation is important, how should you go about it? No crystal balls needed; all you have to do is follow a simple five-step process:

1. Refine your product concept
2. Build a prototype
3. Test the prototype
4. Collect user feedback
5. Run customer feedback analytics to identify actionable insights

Actionable insights, with a little work from your design team, can be turned into a fully-justified roadmap that gives your entire team everything they need to begin work in earnest.  

1. Refining your product concept

If your product concept is still vague, make it concrete before you send it to your product designers and engineers to prototype. Whether you’re starting from scratch, or making a change to an existing product, distill your idea down to its core elements. Have a good look at your value proposition and identify the net benefits you hope to provide to your customer.  

During this time, look at the feasibility of your plan and ensure the technology is there for you to accomplish what you’ve set out to do.

2. Build a prototype

Think of your prototype as a small scale architect’s model: it’s all there, it’s just not fully powered.

Ideally you’ll want to create a prototype that you can use to obtain user feedback, such as a minimum viable product: a lean, bare-bones version of your offering that actually works. Using a MVP allows you to maximize your validation and market understanding while minimizing your work, and is a great way to achieve a fully optimized product.

Note that ‘minimum viable’ doesn’t mean your team has to actually do all the coding and create a backend that makes your front-page sing. You might create a front page that looks automatic, but everything behind the scenes is done by an ‘invisible hand’ (an emulator or even manual data processing). That’s fine, for testing purposes. There just needs to be enough of a semblance of your final offering that users will be able to give relevant feedback.

If you’re torn between two manifestations of your product, consider creating two prototypes and running A/B testing, a randomized analysis of UX on both platforms. This will help you determine which version of your product is most likely to gain traction.

3. Test your prototype

Run your initial tests internally, but don’t stop there. Your team can give a certain amount of usability feedback, but their background knowledge will make them more adept at navigating the UX than your average user. They’ve also got too much invested to be able to offer unbiased desirability feedback.

Once your prototype has the go-ahead from internal testers, recruit a sample set of users from your target audience. Ask them to spend a certain amount of time playing with your mock-ups.

While close acquaintances are a frequent fallback, a random sample is best. Consider online forums and communities as a place to recruit users. Once you have your market sample, set a context, give them a task, and ask for their feedback.

4. Collect user feedback

User feedback can be collected in the form of reviews, surveys, or user interviews. Stay away from leading questions, and focus on user behavior, not user opinion. Ask users “When would you use X?” rather than “Would you use X?”, and how they would accomplish a given task rather than whether they like a particular part of your solution.  

It’s also important to stay open - think about what you really don’t want to hear, and then ask a question that will enable your user to tell you just that. Remember that product validation is not about feature requests or customer-driven roadmapping. Sure, you’ll be creating an optimized roadmap for a great product based on what you find out through user testing - but you’re not asking them what their ideal app would look like and then trying to create it.

5. Run customer feedback analytics

Once you’ve collected your user feedback, you’ll need to analyze it to identify actionable insights. If you’ve recruited a user pool of ten or fifteen, you can do this by hand, given a little bit of time and an Excel spreadsheet.

But if you’re working on a large scale project, you’re better off using customer feedback analytics software. Software that uses AI and natural language processing (NLP) to elucidate trends, themes and key ideas in any number of user reviews is ideal.

Thematic is one example of feedback analytics software. Using Thematic, you can quickly discover the specifics of any themes in your data, and track sentiment to learn whether your users are positive, negative, or neutral about various issues.  

It’s important to discount negative comments or reviews that are focused on the unfinished prototype or bugs in the mockup. This has nothing to do with your end product validation, and should be considered as simply noise. Software like Thematic can help you filter out this type of feedback and focus on what is relevant for your product development.

Your roadmap, validated

Once your unstructured user feedback has been analyzed, you’ll have insight into which aspects of your product resonate most with users, and where you may need to put in extra work or change direction.

The insights you can take action on are your actionable insights, and these are critical in understanding user experience and guiding your decision making. You’ll know what to focus on as you put together your product roadmap.

A well-validated roadmap will set you well on the way to a high quality end product. But remember, it’s never one and done. If you’ve put together a streamlined process with easy-to-use text analytics software, you’ll be able to make validation a constant reiterative part of your product development and keep your processes optimized at all times.

Next steps: Visualizing insights

Clearly present your product roadmap to potential investors or current stakeholders and back it up with visualizations from your analysis. This will help ensure everyone is on the same page and cement support for your plan.

You can take the time to create charts and graphs from scratch, or use visualization software for reporting, such as Power BI or Tableau, Google Studio or Looker.

If you're using a feedback analytics platform like Thematic, intuitive visualization tools are included. Your key insights are automatically turned into graphs, which can be exported straight into Powerpoint ready to be presented.

Further reading

You can find out more about analyzing and leveraging customer feedback in these guides:

• How to code qualitative data
• How to code and analyze open-ended questions
• Guide to thematic analysis software
• How to analyze customer and product reviews
• How to analyze survey data

We also have some free feedback tools and resources that may help you:

• Feedback collector (browser extension)
• Calculating the ROI of CX: complete guide
• Buyer's guide to feedback analytics