How to choose a survey tool to measure customer experience: the ultimate guide

You have decided to collect feedback from your customers via a survey tool. Or, you might be looking for a new survey tool, because you are unhappy with your current setup.

There's no shortage of survey solutions and customer feedback management platforms. If anything, there are too many options to choose from. This can become overwhelming. How can you make sure you are making the right choice given your needs?

We've put together the ultimate guide to finding the most appropriate survey tool given your requirements and preferences.

In this article we'll cover:

1. Key factors to consider before choosing a survey tool

   • How often will you survey? Three use cases
   • Integrations: Will you integrate contact and other meta-data?
   • How do you want to analyze and report on your survey responses?
   • Survey design: Will you self-serve or hire a professional?
   • Will you choose a survey tool or a customer experience management platform?
   • Check what existing customers are saying
     
     

2. What makes a good or bad survey platform? Our analysis of public reviews

   • How to understand the analysis
   • What do people value most about survey tools?
   • What do people dislike the most about survey tools?
   • Which survey tools are the best and worst at setup & implementation?
     
     

3. The top 10 survey tools: strengths and weaknesses
   
   

4. Conclusion: Which survey tool is for you?

At Thematic, we integrate with many different survey software, which means we understand this space. Because we do not offer surveying capabilities ourselves, we can provide an unbiased view. And to make it even more impartial, we're not only sharing our own expertise... We have also analyzed thousands of customer reviews of different survey solutions using our proprietary AI-powered feedback analytics. So, this guide includes opinions by thousands of people who already use the different survey tools.

Before we go into the specifics, let's take a detour. After all, it would help if you knew what kind of survey tool or survey program you need, right?

Here are some of the key factors you need to consider to help you choose the right solution for your business.

Key factors to consider before choosing a survey tool

How often will you survey? Three use cases for customer surveys

In market research, you can split customer experience surveys into three broad groups. Most of what we'll discuss also applies to employee engagement, by the way.

Surveys can be either ‘ad-hoc’ (also known as one-off or dip), transactional (also called journey-based) or relational.

An ad-hoc survey is one that is conducted sporadically (or only once). These online surveys take a selection of the customer base and survey them at a particular point in time. The researchers then analyze the results and take business action based on the findings. Often the survey questionnaire won’t be used again in its original form. For the next ad-hoc survey, companies will most likely modify some of the questions.

The transactional survey is conducted continuously, usually triggered by a specific interaction with the organization. The purpose of this survey is to understand the perception the respondent has of the brand in that specific moment in the relationship. This can be straight after the interaction or hours, days, weeks later. In customer experience, it is common to send a questionnaire immediately following a sale, a service interaction, or a check in 6 months in, for example.

Finally, the relational survey is the in-between. You would run it at a set frequency and include a broad selection of your customer base. It doesn't matter if they interacted recently or not - if anything, having this information makes for great insight too. The frequency is often set yearly, quarterly or monthly. This allows companies to gauge the overall health of the relationship with the brand (hence the name).

Some companies send out a combination of all three types of surveys. This is typically managed by the Customer Experience team and is known as a Voice of Customer program. Some of these programs include requirements to follow up with each respondent to "close the customer feedback loop". There are many solutions that offer these capabilities as part of a single Customer Experience Management platform.

That being said, a full-blown Voice of Customer program might be too expensive for your needs. For example, if you are only running a relational survey using a metric like Net Promoter Score, and some ad-hoc surveys, a simple survey tool might be all you need. However, the more complex you go (adding skip logic, automation or SMS invites for example), the more it will pay off to have a complete solution. This becomes particularly relevant if you need to share customer contact details with your survey tool to trigger survey invitations and you'd rather do this automatically rather than manually.

Integrations: Will you connect survey responses to your customer data?

How rich do you want your insights to be? This is a key consideration to take into account in your decision making.

A survey can be run stand-alone, that is, only contain the information that the respondents give you. Or you could decide to connect your survey responses to information you already hold about that customer in your CRM. E.g. how much they spent to date or whether they complained in the last month. This will likely require some automation. Connecting data in this way is powerful. It enables you to get a fuller understanding of your customer base. It also means you won’t need to bombard them with questions you should know the answer to.

If you decide you want to connect your CRM data to your respondent data, there are two ways you can go about it:

1. Connecting customer data after surveying

You could manually or automatically connect survey responses to your customer data after running your online survey via a tracked customer id. Some companies store customer responses in a database or a data lake, alongside all other customer data.

This has the advantage that all customer data, qualitative and quantitative, operational and strategic, is in one place. Most survey tools can be used to gather feedback as long as they have export or API functions. When specific analytics or reporting is required, you can pull just the data you need internally, without having to share it with an external vendor.

The disadvantage of this approach is the need for data engineering resources. Multiple teams will be involved in such a project. So, you will need to be meticulous about documenting the processes for gathering, storing and retrieving the survey data.

2. Connecting customer data before surveying

Another approach is to transfer the data you already hold to the survey software provider alongside contact details.

The advantage of this approach is that the data is connected from day one. Some survey tools provide built-in analytics, which means that you can get access to initial insights faster.

But this approach poses its own set of problems. You will need to know in advance which data you will need to transfer to your survey provider. This is often impossible to know in advance, and you wouldn’t want to copy everything. There needs to be a process to ensure that the customer data is current. And there is also the whole aspect of data privacy to consider.

Data transfers outside of the organization means data risk. Your data security team may not be comfortable with this - particularly if you have data from people living in the European Union, think GDPR. Sharing this data externally means you need to have a reliable vendor to support you. Enterprise-grade solutions are built to embed automation in their processes and to stand the scrutiny of detailed data privacy reviews. We're not claiming a simpler solution doesn't need to be secure. Instead, it is all about having an appropriate and measured set of risk controls in place.

While traditionally most large companies opted in for the second approach, we’ve seen two new trends in the industry. As more companies realize the value of data, they employ data engineering teams and invest heavily into internal data warehousing. This means the first approach is becoming more common. We are also seeing CRMs like Microsoft Dynamic and Salesforce offer simple built-in survey tools to ensure that customer feedback is instantly stored alongside other customer records.

Analytics and reporting: Key to acting on your survey responses

You know how you want to collect your data, how often and if you want to connect it to data in your CRM. You still need to understand how you want to analyze the data once you have it.

Data analysis and reporting is the most important part of running a survey program. After all, what’s the point of gathering data if you won’t do anything with it.

What matters here is understanding when, how often and who needs to see the survey results. Let's imagine you run a single one-off survey and are happy to look at the data with CSV exports into Excel or SPSS. If you do it only once a year or once a quarter and everyone needs to see the same report, a more affordable solution might be all you need.

If, on the other hand, you need to update the results in near-real-time or daily for 23 divisions inside the organization, with three different viewpoints in each, your reporting needs are complex. Some survey tools might require a lot of effort and take too much of your team's time. In these cases, you will nee a solution that has data visualization embedded or that connects to another analytics platform. This could be a dashboarding tool like PowerBI or Tableau, or a customer feedback analytics solution like Thematic, or both. This will allow you to report frequently, at scale, and to engage all the relevant stakeholders in your program.

However, not every organization has someone with the right skillset to conduct analytics or set up dashboards. Or that person may not have the bandwidth to support market research projects. Whether you have or lack this resource is going to be a determining factor in your survey tool selection. If the skillset exists, a data collection tool might be enough. If it does not, a solution that does the legwork for you and offers you pre-built statistical testing and modelling can take your survey data to the next level.

You don't need to go for an all-in-one solution though. You could have a survey tool to collect your data and complement it with a third-party solution to analyze it. For example, customer feedback analytics platforms like Thematic can pull both your survey responses and your public reviews. Thematic will provide in-depth analytics of not only your scores and structured data but also open-ended qualitative feedback.

Additionally, if the analytics are key to your decision-making, it may be prudent to outsource analytics to a reputable vendor and their specialist team. This not only means more free time for your team, but it also allows you to use more complex analytics on your data. It also has the advantage of having an unbiased partner highlight insights in the data. Such credibility can help gain buy-in from your stakeholders across the business.

Survey design: Will you use self-serve or professional services?

Last, but definitely not least, what about designing your survey program? There are many solutions that now allow self-serve, i.e. users can create their own survey questionnaires. This is useful for teams who rarely tweak their surveys or need a rapid turnaround.

The trend of anyone being able to survey customers on a whim has sadly resulted in a so-called survey fatigue. The response rate to surveys has been dropping over time. Often this is due to poorly designed surveys - writing a good online survey isn't as simple as most people assume. Because self-serve surveying capabilities are democratized, we have an explosion of badly designed surveys. These surveys not only annoy customers, they also rarely offer anything insightful.

Do you have the right skill set within the organization to build a sensibly designed survey or research program? Competent survey design delivers best practice insights instead of unusable and unreliable data. If no one in the organization specializes in survey design, select a vendor that can support and guide you through the process. We have lost count of brands, large and small, that put out surveys that don't match their reputation for excellence elsewhere. The respondent experience is a key touchpoint for your customer experience and your insight efforts.

If in doubt, do not risk your brand; get help instead.

Survey tools vs. customer experience management platforms

You could group the providers of surveying capabilities into two categories.

1. Survey tool
2. Comprehensive experience and survey management platform

Those in the survey tool category would more often focus on the fundamental aspect of collecting respondent data. They often lack the more advanced features but this is reflected in the price tag. They are great value for money if you don't have complex needs or are just starting out.

Customer experience management platforms can give you the most flexibility. And offer a range of features. Some platforms excel at certain features, where others have invested elsewhere. Here are some of the types of functionality you could expect from different CX management platforms:

• Advanced survey design functionality
• Complex survey logic and a variety of question types for survey creation customization
• Cutting-edge reporting technology
• Impeccable user experience
• Strong focus on data security
• Professional services and support

Some platforms will even partner up with you and take you on a journey towards insight led CX change within the organization. They can help you shortcut your learning curve and reap the benefits of customer insights faster.

Check what existing customers say: The social media verdict

All solutions have their strengths and weaknesses. They could be displayed over a spectrum or a two-axis graph like many analyst firms do, like the Forrester Wave or the Gartner Magic Quadrant.

In the end, it will all come down to the provider's ability to deliver great respondent user experiences (in this case, for your CX team). You want to work with a brand that practices what it preaches with their own target audience. You'd be reluctant to hire an overweight personal trainer, wouldn't you? Likewise, be weary of running your online survey program with someone who has a poor reputation or that is unable to deliver value for money.

As highlighted at the start of this guide, we decided to provide insights into public reviews of survey tools and customer experience platforms. Below, we share our approach to analyzing these reviews, as well as the key pros and cons for major survey tool providers.

What makes a good or bad survey platform? We analyze public reviews

The data included and how to understand the analysis

We analyzed over 3,500 reviews in total. Every survey tool provider or a customer experience management platform was included, as long as we could find at least 100 public reviews.

The data was automatically structured into themes and subthemes using Thematic’s proprietary text analytics solution, designed specifically to code qualitative feedback. Each review could break down into multiple themes, each with its own sentiment (positive or negative).

For example, the below two reviews tell us about:

• organizing survey questions,
• ease of use on mobile devices,
• export to CSV capabilities,
• stability of the survey platform, and
• customization of survey templates.

Below there are several themes, across two reviews with varying positive and negative sentiments. The themes found within the review are below the review, and the state of the emoji indicates the quality of the sentiment in which the comment was expressed.

Across the entire set of surveys, a number of themes emerged as the most fundamental ones for end users of survey platforms, all listed below.

Within each theme, specific sub-themes were discovered. Under features, you could read about numbers of features, tagging, editing, language support, videos, widgets, pop-ups, feature requests, real time and so on.

This should give you a clear idea of how theming works within Thematic analysis. Now let's get deeper into the analysis.

What do people value the most about survey tools?

Those who rave about the survey tool that they use tend to talk of great user experience the most. More specifically, the ease of building surveys and the speed to get things done earned platforms top marks in several reviews.

For example, one reviewer said:

Perfect for collecting user feedback! Good bye Spreadsheet. Simple web UI, sometimes deceptively simple, there's some initial overhead in ensuring that people use it properly and don't create duplicates.

Here’s the breakdown of the top themes, as well as the top subthemes under user experience.

What things do people dislike the most about survey tools?

On the flip side, the one thing that tends to turn people into detractors the most is… user experience! In these scenarios, the pitfalls of poor user experience seem to be lacking the required features and the inability to use the tool as intended.

“It would be nice to have the option to schedule a survey out by specific date and time vs. just day of the week.”

“Setting up surveys [or] questionnaires on the admin side still leaves a bit to be desired. I have worked in research for 10+ years and often find it difficult to get the interface to do exactly what I want.”

Interestingly, setup and implementation (which includes the ease of building surveys) is mentioned next as the top 3 most frequent drawbacks when it comes to survey software.

Which survey tools have the best & worst user experience?

The crown for best user experience goes to Alchemer (formerly SurveyGizmo), who just narrowly took the lead ahead of Delighted (owned by Qualtrics), Survicate and SurveyMonkey.

Talking about Qualtrics, reviewers have given them the unenviable top spot for most difficult to use. Overall, Qualtrics, Apptentive and UserVoice reviews are significantly more likely to mention aspects of poor user experience than other survey tools overall. Here's a list of the best Qualtrics alternatives.

Which survey tools are the best & worst at setup & implementation?

The survey platforms that are most praised for their implementation are: Alchemer, SurveyMonkey, Confirmit, Wootric & GetFeedback.

Those who received most criticism for their setup were:

Interestingly, Alchemer and SurveyMonkey are mentioned in both rankings. This likely tells us that some people love their ease of setup and the amount of customization, but others find it either too restrictive or too overwhelming. It seems you either love it or hate it, a bit like how people feel about Marmite.

However, there is so much more to a survey platform than just implementation and ease of use. Why don’t we deep dive into each solution separately to identify what their customers see as their respective strengths and weaknesses?

The top 10 survey tools: strengths and weaknesses

The charts below show theme volume for two different questions in the review:

• Strengths: What do you like the most about survey tool X?
• Weaknesses: What do you like the least about survey tool X?

Blue color shows the volume in reviews by the survey tool in question.
Orange color shows the volume in reviews of all other survey tools.

Let's dig in!

1. SurveyMonkey

Strengths: SurveyMonkey makes it easy to collecting feedback through its intuitive survey builder. Great user experience.

Weaknesses: Users believe SurveyMonkey's pricing is steep for what's included in the paid plan.

Users also felt like the look and feel (logo and branding) makes the end result look less polished than it could be.

2. Alchemer (formerly SurveyGizmo)

Strengths: Similarly to SurveyMonkey, Alchemer’s strength is in setup and implementation. The customization capabilities are however mentioned more often than with SurveyMonkey (27% versus 17%). Alchemer knocks it out of the park in terms of many other positive attributes too, such as their rich feature set as well as look and feel. In particular, the survey templates and a slick professional design for the respondent were big selling points in favor of Alchemer.

Weaknesses: Similar to SurveyMonkey, price is seen as a drawback of the solution (they do have a free version, it just is hidden away). Customers feel they don't get their value for money because of limited editing capabilities, possibly on the free version.

We also found that Alchemer users are more likely to mention bugs in this survey platform. More than 6% people talk about it, compared to just over 1% in reviews of different platforms.

3. Qualtrics

Strengths: Compared to other platform, Qualtrics users are more likely to use it to design Customer Experience surveys. They also clearly walk the talk, as "great customer service" is mentioned by 2x as many reviewers.

Qualtrics surveys are also twice as likely to appear professional compared to other solutions. Finally, they appear to be on par with others when it comes to implementing the solution and on their reporting & analytics features.

Weaknesses: Despite the great customer service, the most frequent pain point in using Qualtrics is, well, that it is hard to use.

Several people stated that the tool wasn’t intuitive and required to go through training. Only then would you enough know-how to make the most of the platform.

4. Medallia

Strengths: Users of the solution call out their ability to drive genuine improvements in customer service and customer experience within their business. Their solution offers strong closed-loop capabilities, enabling seamless issue handling from customers at scale.

Weaknesses: Negative reviews of Medallia comment and its limitations when it comes to functionality/features, issues with mobile app, as well as reporting capabilities.

Twice as many reviews (proportionally) comment on poor reporting capabilities in Medallia than the reviews of all other surveying solutions.

5. GetFeedback (owned by SurveyMonkey)

Strengths: GetFeedback is comparatively easier to use than the other survey tools in the study, particularly in creating surveys. Customers also say they have great survey templates too.

A huge advantage of GetFeedback is its Salesforce integration, something that is mentioned four times more frequently than for the other solutions. Why is that? Well, they state on their website they are ‘the only CX solution purpose-built for Salesforce’. Goes to show it pays to niche down and specialize!

Weaknesses: Interestingly, GetFeedback seems to be the ‘average survey tool’ when it comes to drawbacks: they have the same challenges as their competitors and their stats are surprisingly similar to the average of the study. Nothing really stands out as a big drawback here, although it is worth calling out that some people say the solution is ‘too simple’ and lacks customization.

6. AskNicely

Strengths: A recurring theme around AskNicely is their great customer service (cue is in the name, I guess?), their integrations (in particular with Slack), and their reporting dashboards. They are fierce promoters of the NPS approach, which earns them far more mention than average on their NPS tool. Finally, the praise they receive for the response rate they manage to obtain is threefold that of the average competitor – not bad!

Weaknesses: No specific weaknesses stand out for AskNicely. While many customers like the integrations and dashboards offered by AskNicely, a small proportion of its users believe these features fall short of delivering to their expectations.

7. Nicereply

Strengths: Their key strengths are delivering great customer satisfaction and enabling organizations to improve their customer service. Their focus is on supporting the front-line staff in performing better and improving customer satisfaction overall.

Weaknesses: Nothing really stands out for Nicereply, but if we were to rank the Cons, this would be the ranking:

8. Wootric

Strengths: A great NPS tool! People comment positively on their integrations, in particular with Intercom. Many reviews comment on great reporting and analytics.

Weaknesses: Interestingly, similar things are mentioned among both the strengths and the weaknesses of Wootric. People are more likely to comment on analytics when it comes to Cons.

And while integration is one of the strong points of Wootric, the API it offers seems to be lacking. While the Intercom integration is praised by many, it would appear it is not as comprehensive or easy to use as some would like. The Salesforce integration also appears to need some attention on their side too.

9. Apptentive

Strengths: A strong focus and expertise with app store reviews and ratings, enabling companies to add custom tags, which is not a given in this space.

Weaknesses: When it comes to weaknesses, Apptentive users would like to see more customization, for example when it comes to reporting & analytics. Dashboards aren’t as powerful and the UX could be better.

10. UserVoice

Strengths: Great product for managing feature requests, logging issues and ideas.

Weaknesses: As with Apptentive, UserVoice’s Cons include lack of reporting and analytics, customization, but also poor ability to export the data.

Conclusion

This concludes our review of survey tools and solutions. We tried our very best to be fair in highlighting each platform’s strengths and weaknesses. We strongly believe that there is no single winner or loser, since every company’s requirements are different. As they say, every lid has a pot!

This summary should serve as a guide based on your needs. If user experience matters to you, make sure you can test that. If API integrations are important, make sure you do a deep dive including your technical team before buying.

If there is anything we have missed in this guide, please reach out! We plan to keep updating it with new trends each year.