Why US airlines rank best or worst, according to passengers

Last week, ThePointsGuy published the 2018 “The Best And Worst Airlines In America”. According to Forbes’ interview with Brian Kelly, the author of the report, 9 airlines were reviewed using 10 objective criteria. Here, we extend this report by adding the element of customer perception according to online reviews.

To rank the airlines, ThePointsGuy used 10 criteria such as pricing, lounge quality, cabin comfort and frequent flyer program and ranked the airlines as follows:

1. Alaska Airlines
2. Southwest Airlines
3. Delta Air Lines
4. United Airlines
5. Frontier Airlines
6. American Airlines
7. Spirit Airlines
8. JetBlue
9. Hawaiian Airlines

According to Kelly, the winner, Alaska Airlines “does a lot of things well”, whereas the loser, Hawaiian, scored lowest on the size of its airline route network and airfares.

We analyzed over 44,000 publicly available passenger reviews for these airlines in 2017, and found the following insights that complement the original criteria.

Insight 1. It’s all subjective!

Here are all the general themes found in public reviews of these US airlines. Arrows going upwards indicate that people care more about these themes in 2017, whereas down arrows mean that people cared more about these themes in 2016.

Interestingly, the 10 themes most frequently mentioned by passengers in the online reviews are actually different to the 10 criteria that ThePointsGuy used in his report. The most common themes we found in online reviews of the same 9 airlines were (listed in order of frequency):

1. Crew (cabin crew, flight attendants and their friendliness)
2. Pricing (expensive vs. good price, extra charges, and changes in fees)
3. Good experience (great airline, smooth flight, reliable)
4. Flight service (entertainment, wifi, blankets, great service in general)
5. Flight timeliness (delays and waiting time)
6. Seat comfort (good seats, aisle seats, window seats, changing seats)
7. Great staff (pleasant, professional, helpful)
8. Food and drink service (meal service, alcohol, coffee and tea, snacks)
9. Aircraft quality (bathrooms, air-conditioning, cleanliness, old vs. new aircraft)
10. Checked bags (complimentary bags, bags and fees)

Most of these criteria are very subjective. Whether or not someone liked the food or found the attendant friendly depends greatly on their personal opinion, and their previous experiences that set the service standards.

What really matters to passengers

According to ThePointsGuy, their results are based “entirely on quantifiable numbers and hard data”. This makes sense, because hard facts can be quantified and without traditional extensive qualitative research it’s different to reconcile many differing opinions. However, with tens of thousands of reviews available online for these airlines, and a tool like Thematic, we can aggregate thousands of subjective opinions into a single clear picture of what matters to people.

Ultimately, what customers think is what matters and what is going to impact on whether they choose an airline or not. People are emotional beings. They may know and value that Alaska has the largest route network, but it’s more likely that a memorable customer experience of an exceptional service, without any headaches such as lost baggage, will influence their decision when choosing an airline. Personally, I avoid Lufthansa, after a particularly stressful experience two years ago.

Insight 2. Communication is the key driver of negative scores

To understand which themes actually drive the score up or down vs. which themes are simply the most frequently mentioned, we ran a little data experiment, where we calculated an average satisfaction score for each theme based on the number of stars in reviews that contain those themes.

The themes with the highest average satisfaction scores were:

1. Great staff
2. Good experience
3. Early bird boarding

The themes with the lowest satisfaction scores were:

1. Missed flight
2. Communication
3. Poor experience (which includes the mentions of a missed or an uncomfortable flight).

Which themes are driving the scores

Our next experiment was to calculate the impact score by excluding reviews mentioning the above themes, and here is why. Even though some themes are more likely to be negative, such as a missed flight, not many people overall will be affected by this, as the incident is isolated.

Therefore, if we exclude reviews with a negative but rare theme, the score won’t change. But if we exclude reviews with a negative and frequent theme, the score should significantly improve. So, which themes are actually driving the Airlines average satisfaction scores up or down, because they affect many people?

A great flight (smooth and comfortable) with great staff (pleasant and professional) is what makes the greatest positive impact. Communication, which includes needing to call or email the airline, or needing to wait on the phone, is what makes the greatest negative impact on the score.

Insight 3. Entertainment and wifi — quick wins for Hawaiian

Hawaiian Airlines, which came last in ThePointsGuy’s report, seems to indeed have the worst flight experience compared to its competitors, as you can see in the below chart.

But what really stands out is that where passengers give kudos to competitors for certain attributes of the flight service, they tend to penalize Hawaiian Airlines, as it doesn’t meet their expectations. Hawaiian clearly fails in the areas where its competitors excel:

Specifically, Hawaiian’s entertainment system and in-flight wifi aren’t up to standards. According to passengers, they don’t offer free movies and the movie selection isn’t as good as on competing airlines servicing similar long-distance routes such as Air New Zealand and Qantas Airlines. This is a quick win which is reasonably easy to implement that could see Hawaiian’s overall score improve.

Insight 4. Hawaiian needs to invest in its people

Comparing Alaska Airlines’ 1 and 2-star reviews (the winner in ThePointsGuy’s report), with Hawaiian (the loser), we can see that the number one issue, mentioned by almost 40% of respondents, is the crew and the flight attendants. When people aren’t happy, they are more likely to mention the airline’s staff.

However, it’s not all bad news for Hawaiian. In fact, their meal and beverage service trumps Alaskan’s. It’s one of the top 3 themes mentioned in the 4 and 5-star reviews of the airline.

The moral of the story: while it’s easy to focus on the tangible performance metrics such as pricing or on-time arrivals, what drives satisfaction is inherently subjective and is based on passenger’s growing expectations of what an amazing airline should deliver. Looking deeper into airline reviews for specific themes to see what works and what doesn’t, can give these airlines a roadmap for where to prioritize their time and resource.