You've successfully subscribed to Thematic
Great! Next, complete checkout for full access to Thematic
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info is updated.
Billing info update failed.

Sarcasm in customer feedback – how common is it?

Most people believe that text analytics solutions fail because sarcasm in customer feedback is very common. Somebody writes “Great service, yeah right!” and the dumb algorithm tags it as positive. So, whenever I speak on text analytics, someone in the audience will always ask:

But how do you deal with sarcasm?

People tried writing complex algorithms to detect sarcasm and irony. But are there practical solutions that work? And, more importantly, do we actually need them? How common is sarcasm in customer feedback?

What is sarcasm?

First of all, what is sarcasm? Sarcasm, in simple terms, is when someone speaks bitterly or contradictory to the true intention. Turns out, there are seven types of sarcasm. Sarcasm is also an example of a verbal irony, something that occurs when the underlying meaning is contradictory to what is being said. There are other examples of verbal irony such as overstatement and understatement.

A classic example, at least here in New Zealand, take the Tui Brewery’s Yeah Right billboards (see the image above). Sarcasm can be challenging, misleading and annoying. I am an immigrant myself, submerged in the British culture. So, I can attest that it will take me many more years before “I get it”.

So, my British business partner sent me this

twitter image.jpg

Sarcasm, Algorithms, Customer Feedback, and Your Brand

Can algorithms detect sarcasm?

Understanding sarcasm and irony is hard. Just like understanding humor, it requires not just language proficiency, but also knowledge of common sense, cultural knowledge, and context. These can be quite intricate.

Unfortunately, we have not yet succeeded at teaching computers common sense, nor cultural knowledge. Capturing context is another hard nut to crack. This means that unless there are clear clues of irony such as emoji or heavy punctuation, today’s text analytics will struggle with sarcasm.

But how common are sarcasm and irony in customer feedback? And how useful is it for brands being able to deal with them correctly?

How common is sarcasm in customer feedback?

I recently spoke to people at two large Australasian retailers. Both stated that Kiwis and Australians love sarcasm. They were convinced that their customers leave plenty of sarcastic comments, both in surveys and on social media. I was curious if that’s true.

I reviewed two datasets with customer feedback by hand to figure out how common sarcastic sentences were:

  • 500 survey responses
  • 500 posts on social media

Turns out, even in retail, less than 1% of survey comments and less than 5% of social media comments contained sarcasm. This corresponds to findings in other datasets. The training dataset for sarcasm detection competition at PAKDD 2016 also contained only 3% of positive examples.

But the fact that sarcasm is rare is just one part of the story. Let’s look at the examples of sarcasm I found.

When is sarcasm useful to brands?

Somebody wrote on Facebook: “Very helpful answer. Troll.” Troll, in the context of social media, is a derogatory term, so the comment is sarcastic. The answer wasn’t helpful at all. Most algorithms would fail to recognize this.

But so what? It’s just some banter on social media. Half of the sarcasm examples I found were of this type: meaningless back and forth in comments to posts unrelated to the actual topic of the conversation.

What brands want to know is what people are specifically saying about the product, the brand, competition, or people in customer-facing roles. Detecting sarcasm in people’s reaction to each other’s opinion is not of much value. This brings down the percentage of meaningful sarcasm even lower.

Most other examples of sarcasm were part of a longer comment. For example, “I’m so disappointed! What a great customer service you have!”. It’s impossible for a person to be disappointed at and praise something at the same time. Hence, we can conclude that it’s a sarcastic comment. The service wasn’t great at all. This is an example of where the brand will be interested in this information

The good news is that most algorithms looking at the whole comment will mark the general sentiment as either negative or neutral. A useful algorithm will also detect “customer service” as the topic of this comment. Following that, people’s sentiment will be aggregated across all comments correctly.

Conclusion: Remember this about sarcasm in customer comments

  • Sarcasm in customer feedback is difficult to understand for an algorithm. But it is incredibly rare! Rarer than you think!
  • Most examples of sarcasm on social media aren’t tied to any specific topic. Therefore, they are meaningless to brands.
  • Sarcasm in relation to a specific theme in customer comment is likely to appear alongside other indicators of the author’s true sentiment. Algorithms are likely to identify correctly those.
Try Thematic on your data

Thematic is the easiest way to discover the best insights in feedback. Act on what matters to your customers and make an impact.

Keep reading
How to theme qualitative data using thematic analysis software
How to theme qualitative data using thematic analysis software

If you ever had to analyze customer feedback, you will know that the most difficult part is to create a perfect code frame. You need to understand the dataset, the stakeholders involved and the ideal outcomes of the analysis. You will have to iterate before settling on a solution, which

Alyona Medelyan PhD
Alyona Medelyan PhD
How to super-charge your Qualtrics setup with Thematic & Power BI
How to super-charge your Qualtrics setup with Thematic & Power BI

Qualtrics is one of the most well-known and powerful Customer Feedback Management platforms. But even so, it has limitations. We recently hosted a live panel where data analysts from two well-known brands shared their experiences with Qualtrics, and how they extended this platform’s capabilities. Below, we’ll share the

Alyona Medelyan PhD
Alyona Medelyan PhD
Customer Experience
How we use our own platform and Chrome extension to centralize & analyze feedback
How we use our own platform and Chrome extension to centralize & analyze feedback

Customer feedback doesn't have all the answers. But it has critical insights for strategy and prioritization. Thematic is a B2B SaaS company. We aren't swimming in feedback. Every piece of feedback counts. Collecting and analyzing this feedback requires a different approach. We receive feedback from many places: our in-product NPS

Alyona Medelyan PhD
Alyona Medelyan PhD
Feedback Analysis