Welcome, Guest Login Register
loading...
Net Promoter Community > Richard's and Laura's Blog > Tags > benchmark
 

Richard's and Laura's Blog

2 Posts tagged with the benchmark tag

"All the women are strong, all the men are good looking, and all the children are above average."

Garrison Keillor, Lake Wobegone Days

 

Most performance measure in business are relative. Market share, growth rates, earnings. We benchmark against others all the time. But with NPS, many companies don't really know where they stand, and where they stand could be the ultimate measure of performance.

 

Several years ago, Bain and Company did some great research to understand how profit pools got divided up by industry, and how NPS played a role in that. They found that every industry had a “bright line”, an NPS score which separated winners from the pack, and that those who entered the winners circle (so to speak) enjoyed a disproportionate share of the profits in their industry. This shouldn't surprise anyone; in most industries profits are not linearly correlated to size or even market share. It's not a fair game - it turns out it's rigged in favor of NPS leaders.

 

What's really interesting to me, however, was that the bright line that separated the leaders was not uniform across all industries or geographies. Rather, it varied significantly by industry. This should come as no surprise: we usually see that Business to Business NPS results are often significantly lower and less variable than Business to Consumer. We also know that customers compare and formulate perspectives based on their expectations, which can vary according to prior experience and price. We teach customers what to expect in our industry, then we give them relative pricing to help set their expectations around our role as discounters or premium players.

 

Companies spend precious little time thinking about issues of comparable performance, which seems odd given its importance. After all, knowing what your score is only matters if you have some sense as to what it should be. This disconnect ripples through the corporation in multiple ways. Companies without a clear sense of NPS “situational awareness” will struggle to set appropriate goals, compensation metrics or process. I'd even go as far as to say that measuring NPS without a clear sense of target is worse than not measuring at all.

 

Part of the problem is the challenge around getting good benchmarks. We do publish benchmark data, so for those industries we cover, you can get an accurate sense of where you should be (and an independent view of where you are trending). But that data will never be detailed enough for some, so they often find themselves doing specific benchmarking studies. Others tell me that they make a benchmark of the data that is published in the book. While I agree it's aspirational to try and reach USAA's lofty NPS goals, it's probably neither feasible nor even desirable, assuming you are not a major insurance provider. Nor is it practical to simply target the top quartile and straight line trend your own NPS towards it as a goal setting technique. You are pretty much assured of falling short initially as NPS just doesn't improve in a straight line.

 

If you are determined to avoid detailed target setting and outside benchmarks, there is still hope. Stack ranked employee, or region or segment performance provides you with the opportunity to challenge the lower performing segments to raise their game to the average. That alone improves your score and starts moving you in the right direction; although it doesn't help you understand if your entire organization is on track for financial success.

 

One other thought. Absolute NPS does matter in one important way. Industries with low "threshold NPS" - a low target that gets you in the winner's circle - have fewer absolute promoters than those where only a high score wins. The absolute number of promoters can be thought of as word of mouth capacity, so their absence reduces the overall positive effect for industry participants and the industry in general. By all means, out-run the other guy for success, but to get real organic growth you still need an army of promoters.

1 Comments Permalink

Everyone's a Winner!

Posted by RichardOwen Sep 7, 2010

If you, like me, track companies who publish their NPS you probably are developing an inferiority complex about now. Every day I log in to a constant stream of announcements about companies posting 50, 60, even 90% NPS. Precious few of these fall in the camp of public companies who are informing the street - the majority of comments seem to come from either smaller companies, or large companies through "unofficial" channels.

Oh, and there are fair number of "benchmarks" and "awards" starting to come out from companies who sell services into the NPS space. After all, nothing picks up search engine hits better than "Large Company X gets our award for the highest NPS of 70% in their industry!" Corporate PR departments are not in a rush to disclaim such announcements, and these scorecard publishers are wisely more careful about publishing the bottom rung of their study.

So there you are, running your NPS program and sweating to create a 20% NPS. Your boss now wants to know why everyone else is at 50. You must be doing something seriously wrong! Or maybe not.

We like to think we know a thing or two about NPS scores. With apologies for the risk of hubris, we don't see the evidence, at a market level that these very high numbers are typical, nor do we see the anecdotal evidence amongst the large numbers of companies we discuss NPS with each year. So what gives?

Pants on Fire!

It's generally thought that Charles Wentworth Dike, who died in 1911, coined the term "lies, damned lies, and statistics" as a practice of using statistics to bolster a weak argument. However, I would argue this is an unlikely explanation and we should start with giving the benefit of the doubt to these publishers. Deliberate misrepresentation of data seems pretty rare.

Adverse Samples

It's quite possible that the companies publishing their NPS do not represent a good sample of the overall population. This topic is rich with irony; one of the classic tenants of NPS is that it should avoid heroic extrapolation from small data (especially in B2B) and act in favor of census. It's very plausible that only companies with very high NPS would seek to publish them. Of course, this risks looking silly, or at the least disingenuous should those scores decline and you - ahem - forget to update your readers. But this theory does hold water as long as you have a good sense of what is good in your own industry. The PR value of a 15% score in the cable television industry might actually be lost on the public at large but be relatively better than a 60% NPS in the luxury hospitality game. All these factors create upwards bias.

Equally, it's possible that the publishing of data is more likely by small, private companies rather than large public firms. Publically traded companies always worry more about lawsuits and the defensibility of their claims, so perhaps that's it. Or maybe, just maybe, a lot of small nimble companies tend to have higher NPS after all!

Getting the math wrong

We see enough sophisticated, large companies get the math wrong to suggest that any published score has no better than a 50/50 chance of being correct (meaning an accurate representation of the business) in any case. Sure, there are the eye-popping errors (0-11 scale, multi $bn publically traded firm) but the more likely error is around sampling or segmentation introducing bias into their numbers. It's hard to get a trustworthy data set, as you probably already know. Small error? Not necessarily, our benchmark data can differ by 20% or more from publically reported numbers of the same firm. Of course, our benchmark has built in errors also for absolute score reports.

Of course, you could ignore all the noise. My advice: focus on improving your own score. If you really want a comparison point, organize a comprehensive benchmarking study and draw your own conclusions.


[Editor's note: Download Satmetrix White Paper on Creating a Sampling Strategy for your Business]

0 Comments Permalink

Actions