Streams of unsolicited customer feedback (support tickets, chats, reviews, social posts) converging into a single customer experience score.

How to Measure Customer Experience Without Relying on Surveys

Survey response rates keep falling. Here is how to measure customer experience from the unsolicited feedback customers already give you, then turn it into scored, trackable metrics.

Insights
>
>
How to Measure Customer Experience Without Relying on Surveys
While you're here

TLDR

You can measure CX without surveys by analyzing unsolicited signals (tickets, calls, chat, reviews, social) instead. Theme them bottom-up, then score them into synthetic NPS, CSAT, and effort metrics you can track over time. Validate against any surveys you still run, and route each metric to the team that owns the driver.

Fewer customers answer surveys every year. Email response rates for NPS and CSAT surveys fell from 20-25% in 2019 to 10-15% by 2025, according to CX analytics firm Clootrack. The Qualtrics XM Institute reports that customer survey response rates dropped 7 to 8 points since 2021. The people who do respond are not a random sample. Response rates fall fastest among younger and more diverse customers, so the score you report drifts further from the customers you are trying to understand.

You can measure customer experience without relying on surveys. Analyze the feedback customers already give you unprompted, then convert it into scored, trackable metrics. Thematic does this in three moves. It unifies unsolicited signals: support tickets, call and chat transcripts, reviews, and social posts. It themes them from the bottom up. Then it generates synthetic outcome metrics such as predicted NPS, CSAT, and customer effort with its Scoring Agent. The result is a CX measure that updates continuously and covers the silent majority a survey never reaches.

Below is the five-step process, a worked example from Atom Bank, and the mistakes that trip teams up when they make the shift.

The five-step process

  1. Inventory the unsolicited and behavioral signals you already collect.
  2. Theme the feedback from the bottom up, without forcing a survey taxonomy.
  3. Convert the themes into trackable metrics you can compare over time.
  4. Tie each metric to the drivers and business outcomes behind it.
  5. Validate the metrics and route them to the teams who act.

Step 1: Inventory the signals you already own

Start by listing every place customers already tell you what they think without being asked. Most enterprises are sitting on far more of this than they realize. The Qualtrics XM Institute estimates that 85% of customer and employee feedback is unsolicited and unstructured. Industry analysts estimate 80 to 90% of all enterprise data is unstructured.

The signals worth inventorying:

  • Support tickets and contact center notes
  • Call and chat transcripts
  • App store reviews and third-party review sites
  • Social media posts and mentions
  • Complaint logs and escalations
  • Product usage and in-product behavior

These sources reach the customers surveys miss. CX Dive reports that for every four customers who complain, 96 stay silent. Those 96 rarely fill out a survey. Many of them leave a review, open a ticket, or churn. Cataloging these signals is the foundation, because you cannot measure what you have not connected.

Step 2: Theme the feedback from the bottom up

Once the signals are connected, you need structure. A survey gives you structure through fixed questions. Unsolicited feedback does not, so the structure has to emerge from the feedback itself.

Bottom-up theme discovery reads the raw text and surfaces the themes customers actually raise, rather than sorting comments into categories someone defined last year. This matters because unsolicited feedback contains issues a survey would never think to ask about. Forcing a support ticket into a preset NPS taxonomy throws away the specificity that made the ticket useful.

Thematic builds themes from the language in the feedback. It keeps every theme traceable back to the raw comments behind it. A skeptical stakeholder can click into any theme and read the verbatims that produced it. That traceability is what lets an unstructured signal stand up to the same scrutiny as a survey score.

Step 3: Convert themes into trackable metrics

A theme tells you what customers are talking about. To measure experience, you need a number you can track over time and compare across channels. This is the step most teams skip, and it is the one that makes survey-free measurement real.

Thematic's Scoring Agent generates a predicted score from unstructured feedback. As Thematic describes it, "this synthetic score is an outcome metric inferred from what customers said," produced even when there is no linked rating attached to the comment. You define in plain language what you want to measure, and the agent produces consistent, metric-like scores across datasets.

The metrics you can build this way include:

  • Synthetic NPS, CSAT, and customer effort scores
  • Propensity to churn and likelihood to expand an account
  • Sentiment-weighted theme volume and its movement over time

As Thematic puts it, "the richest signals live in call center transcripts, support tickets and complaints, but they are hard to standardize into metrics that track improvements over time." Scoring is what turns those signals into a measure a leadership team can read like an NPS.

Step 4: Tie each metric to its drivers

A number on its own invites the question every executive asks: why did it move? Surveys answer this poorly, because a rating scale records the score but not the reason. Measuring from unstructured feedback flips that.

Here the division of labor is clean. Scores quantify the outcome. Themes explain the drivers. Because each score traces back to the themes and verbatims that produced it, you can move from "effort went up three points this month" to the specific themes pushing it up in a single view. Thematic links scores to drivers through its impact and waterfall charts, so the measure and its explanation live together rather than in separate reports.

This is also where measurement stops being a scorecard and starts driving decisions. When the driver behind a moving metric is visible, the team responsible can act on the cause instead of debating the number.

Step 5: Validate the metrics and route them to teams

A synthetic score is only useful if people trust it and act on it. Two moves earn that trust.

First, validate. If you still run any surveys, use them as a benchmark. Check that the synthetic NPS tracks the survey NPS on the segments where both exist, then rely on the synthetic score for the far larger population that never answered. Thematic builds this in through a Define, Validate, Track flow with human-in-the-loop review, so an analyst confirms the score behaves before it goes to a dashboard.

Second, route. A metric that lands in a monthly slide gets ignored. A metric that reaches the team who owns the driver gets acted on. Forrester's research found that 96% of CX programs regularly collect and analyze surveys, yet only 50% are effective at analyzing even their own survey verbatims. The gap is rarely data. It is getting the measure and its cause to the right team fast enough to matter.

A worked example: Atom Bank

Atom Bank, a digital-first UK challenger bank, had customer feedback scattered across app store reviews, Trustpilot, complaint logs, and surveys, all using different rating scales. No single number described the experience. The survey portion covered only the customers who bothered to respond.

Using Thematic, Atom Bank unified those sources into a single Customer Goodwill Score mapped on a 1-to-100 scale. That gave one consistent measure across every touchpoint rather than one score per channel. Most of the inputs behind that score, the reviews, the Trustpilot ratings, and the complaint logs, were unsolicited. The survey was one input among several, not the whole measure.

Acting on what the unified score surfaced, Atom Bank reduced contact center failure demand by 30% and reached a 4.6 out of 5 Trustpilot rating. The measure worked because it drew on the feedback customers were already giving, at a scale no survey program could match. The payoff can be large. A separate large Australian retailer attributed $4.8M in incremental income to customer-driven store-level decisions unlocked from a single pilot.

Common mistakes to avoid

  • Rebuilding your survey taxonomy on unstructured data. Preset categories throw away the specificity that makes tickets and reviews valuable. Let themes emerge instead.
  • Stopping at sentiment. A positive-negative split is not a metric leaders can track. Convert themes into scored outcomes with units and movement over time.
  • Treating the synthetic score as unauditable. If a score cannot be traced back to the verbatims behind it, no skeptical stakeholder will trust it. Keep every metric linked to its source comments.
  • Measuring without routing. A score nobody owns changes nothing. Send each metric, and the driver behind it, to the team that can act.
  • Throwing surveys out entirely. Surveys are a weakening instrument, not a useless one. Keep a lean survey as a benchmark to validate the signals doing the real measurement.
1. Guide Analysis
Guides

Build, Buy or Partner? A Layered Guide to AI Feedback Analytics

Transforming customer feedback with AI holds immense potential, but many organizations stumble into unexpected challenges.

/* Top banner pulsating dot */