If you, like me, track companies who publish their NPS you probably are developing an inferiority complex about now. Every day I log in to a constant stream of announcements about companies posting 50, 60, even 90% NPS. Precious few of these fall in the camp of public companies who are informing the street - the majority of comments seem to come from either smaller companies, or large companies through "unofficial" channels.
Oh, and there are fair number of "benchmarks" and "awards" starting to come out from companies who sell services into the NPS space. After all, nothing picks up search engine hits better than "Large Company X gets our award for the highest NPS of 70% in their industry!" Corporate PR departments are not in a rush to disclaim such announcements, and these scorecard publishers are wisely more careful about publishing the bottom rung of their study.
So there you are, running your NPS program and sweating to create a 20% NPS. Your boss now wants to know why everyone else is at 50. You must be doing something seriously wrong! Or maybe not.
We like to think we know a thing or two about NPS scores. With apologies for the risk of hubris, we don't see the evidence, at a market level that these very high numbers are typical, nor do we see the anecdotal evidence amongst the large numbers of companies we discuss NPS with each year. So what gives?
Pants on Fire!
It's generally thought that Charles Wentworth Dike, who died in 1911, coined the term "lies, damned lies, and statistics" as a practice of using statistics to bolster a weak argument. However, I would argue this is an unlikely explanation and we should start with giving the benefit of the doubt to these publishers. Deliberate misrepresentation of data seems pretty rare.
Adverse Samples
It's quite possible that the companies publishing their NPS do not represent a good sample of the overall population. This topic is rich with irony; one of the classic tenants of NPS is that it should avoid heroic extrapolation from small data (especially in B2B) and act in favor of census. It's very plausible that only companies with very high NPS would seek to publish them. Of course, this risks looking silly, or at the least disingenuous should those scores decline and you - ahem - forget to update your readers. But this theory does hold water as long as you have a good sense of what is good in your own industry. The PR value of a 15% score in the cable television industry might actually be lost on the public at large but be relatively better than a 60% NPS in the luxury hospitality game. All these factors create upwards bias.
Equally, it's possible that the publishing of data is more likely by small, private companies rather than large public firms. Publically traded companies always worry more about lawsuits and the defensibility of their claims, so perhaps that's it. Or maybe, just maybe, a lot of small nimble companies tend to have higher NPS after all!
Getting the math wrong
We see enough sophisticated, large companies get the math wrong to suggest that any published score has no better than a 50/50 chance of being correct (meaning an accurate representation of the business) in any case. Sure, there are the eye-popping errors (0-11 scale, multi $bn publically traded firm) but the more likely error is around sampling or segmentation introducing bias into their numbers. It's hard to get a trustworthy data set, as you probably already know. Small error? Not necessarily, our benchmark data can differ by 20% or more from publically reported numbers of the same firm. Of course, our benchmark has built in errors also for absolute score reports.
Of course, you could ignore all the noise. My advice: focus on improving your own score. If you really want a comparison point, organize a comprehensive benchmarking study and draw your own conclusions.
[Editor's note: Download Satmetrix White Paper on Creating a Sampling Strategy for your Business]
