Finding Product Insights in App Reviews

I rarely visit a bank in person, but I use my bank’s website every week. I’m sure I’m not alone, and often wonder why every banking website I’ve used feels like the equivalent of walking into a building with torn carpet, flickering lighting, and a couple of leaky pipes overhead.

I always look around and think, “this should be a whole lot better.” And I’ve long been interested in using a bank built around online services.

I wanted to find out if a digital-first bank provides a better digital experience than a traditional bank.

To do this I decided to use Thematic to answer my question by discovering product insights in banking app reviews.

In the wake of the Apple Card debut, I thought I’d check in on Simple; one of the first online-only banks launched in the iPhone era. I decided to compare Simple against a big traditional bank, Chase.

To compare the two, I used Thematic’s AI to analyze publicly available customer feedback from the Android and iOS app stores.

While the apps had a comparable overall ratings of 3.8, Thematic uncovered key differences by identifying themes in the 15.3k comments that went with with the ratings.

Read on to see how they compare.

Chase

Chase is competitive in an area I assumed would be a core advantage for Simple: ease of use. 10.4% of Chase reviews commented that the app was easy to use, compared with 13.5% of Simple reviews.

Simple has the edge, but not an overwhelming one.

Customer support is also a strong point for Simple. Roughly 3x as many Simple customers talk about great customer service as compared to Chase. Ironically. 10x more Simple customers mention poor customer service.

Although customers generally rate Chase’s mobile app as easy to use, Thematic’s theme impact calculations showed that bugs, updates causing problems, and simply being unable to login to their account cause a much larger negative impact to Chase’s app rating than Simple’s.

Now let’s compare this to similar product insights we discovered in Simple’s app reviews.

Simple

I was surprised to find that one of Simple’s greatest advantages over Chase comes from its low fees, rather than a better app.

Customers love Simple’s low fees, with one saying, “No fees, no risk, what else could you ask for?” People mentioning fees left an average rating of 4 stars compared to 2 stars with Chase.

On the other hand, being online-only had an interesting downside: customers reported more issues getting started with Simple.

Unlike Chase, Simple doesn’t have physical locations to guide the process of opening an account, so users encounter issues like this:

“Spent an hour trying to open an account. Online, the site just hung up and I got the never-ending blue spinning circle of death. When I tried to submit the application via the app, it says my email is unavailable.”

One particularly interesting product insight came from looking at what was impacting Simple’s score using Thematic’s “Over Time” feature. The Over Time feature identifies themes that impact a score between two points in time.

Their score looked stable from April to May 2019, but our tool revealed that it would have risen significantly…

Were it not for the fact that comments about their billpay feature decreased their overall rating by almost half a star that month.

As it turns out, they discontinued their billpay service that month, and customers were quick to vent their unhappiness.

Other problems involved customers detesting having to wait for their deposits and transfers to clear.

“To have to wait 3 days for a bank transfer to even get started is ridiculous.”
said one.

This issue showed up 10x more frequently than in feedback about Chase.

Lastly, one customer commented:

“ever since Simple got swallowed up by another bank, their customer service has gone way downhill.”

It turns out this anecdotal feedback is borne out by the data. Simple was acquired by BBVA in 2014, and as you can see, customers mentioning customer service have dropped their ratings significantly. From close to 5 stars at the time of the acquisition, to around 3 stars today.

Both banks

Customers from both banks reported a variety of issues related to logging into their accounts. Including having to type passwords because fingerprint/face login is often unavailable, being forced to reset passwords frequently, and crashes on login.

It’s surprising to see the critical first step every user has to take when opening the app riddled with this many problems.

In addition to this, a significant number of people also reported problems making mobile check deposits.

These issues range from bugs like, in the succinct words of one customer; “mobile check deposit doesn’t work”, to usability issues like this:

“Check deposits harder than they need to be. Once I tell you I want to deposit, open my keyboard so I can enter the deposit amount. Then once I choose “auto” for the photo, remember that for the photo of the back of the check and for the session. When I have ten checks to deposit, that time adds up.”

If I were working on these apps as a product manager, quantifying problems like this would help me make the case for prioritizing bugfixes and polish.

When you begin to hunt for product insights in your review or customer feedback data, you find issues that you could easily otherwise overlook.

And these issues are often simple fixes that can have big impacts to your user experience.

The bottom line

Simple edges out Chase in terms of app quality, with better ease of use and fewer bugs.

Although this advantage is countered by issues with their core banking service. Ranging from customer onboarding to delays in deposits clearing to customer service.

Seeing the feedback data summarized and quantified reminded me that a well-designed app is only one of many important factors to consider when choosing a bank.

Even Apple is learning some hard lessons here. As for me, I’ll stick with my current bank, even if the app is MacGyver’d together.

Interested in turning your survey, chat, review and complaint data into product insights?

Discover how Thematic finds insights in your feedback data.