Many of you, no doubt would have already read the article in the Sloan Management Review by Tim Keiningham et al on the topic of Net Promoter. You missed it? Not a complete surprise. Much to my own chagrin as a Sloan Alumni, the SMR is not exactly mainstream reading. However, it is the absolutely perfect place for Tim to make his arguments against Net Promoter as a methodology. It's a great location for academic arguments.
As I have argued before, detractors of Net Promoter usually start from a position of self interest, but position themselves as academics in search of a universal truth. Traditional market research firms have made a lot of money developing complex methodologies for measuring satisfaction and loyalty, and in doing so positioning themselves as essential advisors to their clients. In many instances, they have done great work and created value. Regardless, the widespread adoption of Net Promoter represents a simple business challenge that must be met head on. The risks come in three parts.
First, standardized open metrics threaten a lucrative business stream in metrics design. A client who adopts Net Promoter won't be spending big dollars custom creating a perfect metric. This doesn't mean that they might not benefit from a custom metric, but it changes the value equation and certainly creates enormous price pressure.
Secondly, Net Promoter, with it's appeal to senior executives and line managers, threatens their client relationships and budgets. In many cases, Net Promoter has been driven by executives outside the market research function, such as in marketing. This is not the client relationship that the market research firm has spent 20 years cultivating. There are many cases also where market researchers have let the Net Promoter initiative, but in those cases the shift in methodology still imperils business relationships. Given Tim's strong stance against NPS, what if a major Ipsos client adopted that methodology? Do you think that might have a relationship impact?
Finally, a simple metric shifts the nature of competence in service providers. If you are selecting a vendor for a Net Promoter program, what skills do you look for? They might look very different from the competency set that the traditional market research vendor has developed. Process design, systems competency, strategy... all tools that may trump the deep research and statistical expertise that has powered this industry traditionally. Not a positive trend if you are in that business.
Of course, having a business motive to attack a methodology, doesn't mean you aren't correct in your assertions. Most arguments against NPS generally take one of two forms.
First, the argument is with the original correlation research. Generally these take the form of a study that shows that NPS may not be the absolutely best metric to correlate to growth in every scenario. Often, the favorite metric of the company doing the study turns out on top. It's hard to make the case, however, that there is no bias whatsoever in these studies.
When Laura Brooks and Fred Reichheld did the original research on NPS, they were not looking to prove that NPS was a better metric. They simply followed where the research data led them. That's a fundamentally different approach than a study designed to promote one view over another. Put another way, if they hadn't discovered a good story, there simply would have been no outcome published. Satmetrix would continue with it's business, as would Bain.
By contrast, studies designed to prove another methodology wrong will simply keep seeking data until the particular circumstances prove their point. Put another way, could you imagine that if the data pointed in favor of NPS, the article would be published?
The second argument is, in my opinion, more condescending. It usually takes the form of a veiled attack on the ability of senior execs or line managers to "understand" this business of customer loyalty. NPS is simple, therefore it appeals to people who "don't understand" the implications of their choice or who are not "trained in research". CEO claims that NPS is an effective business tool are dismissed as somehow based on imcomplete understanding of the situation.
This is supremely arrogant. If the ultimate goal of all our efforts is business outcomes, not academic purity, these people are our customers. If they are getting results they are happy with, isn't that the ultimate goal? The fact that we don't like the way they got their results doesn't entitle us to run them down.
It also misses the point as to why executives do adopt Net Promoter. Hint: It's not just the metric.
The fixation of the anti-NPS vendors is around the metric, not the system, because they frequently don't value the system. Yet the business owners usually place much higher value on the complete Net Promoter system than just the metric -- the ability to execute closed loop processes, engage frontline employees etc.
Missing the system is simply missing the point.
Timothy Keiningham
says:
Richard --
You have every right to disagree with our findings. You do not, however, have any right to imply that the research I co-authored with esteemed academic colleagues was in any way tainted by my employment. To suggest that my co-authors and I would bias our analysis to discredit Net Promoter is ridiculous, and the highest insult to any serious scientific researcher. We did not hunt out data designed to discredit the metric, and your assertion that that is the case is unfortunate. If Net Promoter had shown itself to be a strong predictor of growth, then that is exactly what we would have reported. Finally, our research has past peer review in the most prestigious scientific journals, and won best paper awards in these journals, which is a testament to the quality of our research.
Sincerely,
Tim Keiningham