David Hankin, Director of Business Consulting for Satmetrix, discussed how and why organizations should benchmark their Net Promoter Scores against their competition and account for cultural differences. Competitive benchmarking helps executives answer the question, "How do I know if we have a good NPS?" There are three approaches to competitive benchmarking that can answer this question:

1) add competitor questions to your own loyalty survey,
2) use an external industry benchmark and
3) build your own panel of customers across your industry.

Each option has different costs, benefits and complexities.

Option 1
The first, and simplest, approach is to include some questions about competitors in your loyalty survey. This is easy to deploy quickly, and can be done at a low cost. One key consideration is to carefully word the questions to make sure the respondent has had recent experience with your competitor. The other is to realize that your customers may not represent the total universe of your competitors' customers. However, the approach can provide directional, or relative, performance of your company in the market compared to your competitors. Clearly, this can be valuable information.

Option 2
The second approach is to work with an external industry benchmark provided by a third party. The way this works is that competitors in a market space agree to provide their customer loyalty data, confidentially and in aggregate, to the third party so that all of the market players can view low, high and average Net Promoter Scores for their industry. Competitive gaps also can be included, which enables the participants to view the effectiveness of their differentiation strategies, or perhaps, develop new strategies. The third party ensures confidentiality. Satmetrix provides several of these types of benchmarks, most notably in Software and Telecommunications industries.

It is important in this approach to identify the companies included in the benchmark. In some industries, one or more large companies refuse to participate in the benchmark and thus lower the value of the benchmark itself.

Option 3
The third option is rent or build your own competitive panel, but be warned, costs are high. Costs per completed interview can run over $50 each to get the right sample base. The key is to make sure that a representative sample base can be acquired, because without the right sample, your insights from the competitive assessment may be off the mark. If done correctly, building a panel of your competitor's customers can be an incredibly powerful method to compare Net Promoter Scores and identify opportunities to differentiate.David also discussed the impact of cultural differences on survey scores, and their implications.

Cultural bias can be easily understood as the different way a respondent may score the same survey based on where they live. According to the Satmetrix cross-cultural benchmark, people in Latin-America typically will report higher Net Promoter Scores than people located in some areas of Asia. However, organizations should be very careful of overstating cross-cultural biases. Adjusting the Net Promoter model is not recommended; i.e., avoid the temptation to make 8's promoters, or 6's passive in certain regions. In line with the general theme we have heard during the Net Promoter conference, David described how important it is to track the rate of improvements to NPS, rather than comparing the baseline Net Promoter Scores across regions.

David's final topic discussed how to set NPS targets and goals. The first step is to identify achievable targets based on your internal NPS and external industry benchmark. Not all industries are the same. For example, the Satmetrix benchmark shows that the average NPS in Telecommunications is lower than the lowest NPS in Financial Services. Customer expectations and competitive pressures vary by industry, and this is reflected in NPS ranges.

There is also a law of diminishing returns that shows the higher your NPS, the harder it is to make large percentage improvements. It is important to understand where your NPS baseline exists today, and what a good improvement in NPS would like. Targets can then be set which are achievable and will lead to significant growth.

Any questions regarding this blog can be e-mailed to paulp@satmetrix.com.

Click Not authorized to view the specified document 1090 to download the presentation.

Audience expectations seemed quite high as it was standing room only. Taking a practical approach as a result of his practitioner experience, Dr. Vince Nowinski, Principal Methodologist of Satmetrix, began by addressing the who, what, and when of data collection. He explained how the underlying validity and content of the data collected is what will ultimately impact the "actionability" of the information in your organization. (Yes, that is a made up word, btw.)

In addressing the "who should be surveyed," Vince pointed out that it varies greatly in B2B environments from B2C environments. A key element of measuring what matters is "Voice according to value." Vince went on to contrast B2B and B2C approaches. B2C segmentation is used to inform the organization as to the nature of the core customers, those using the strategic elements of what is delivered to customers. In most organizations these are the customers who drive the bottom line. The goal is to optimize loyalty, not to just drive it up.

In B2C settings, it's important to segment by value too. The wrinkle in B2B is that there are multiple decision making influences who can be fundamentally characterized as end users, influences and decision makers. It's important to realize that all voices are not equal and to recognize the relative importance of these various roles in the decision making syndicate.

Now on to the issue of sample versus census. Often times, the driver of the sampling strategy is statistical significance. In fact it's as important that the right targeting and recruiting of respondents is key to making the results believable and usable in the organization. In a B2B setting, census is the preferred approach, whereas collecting a statistically significant sample is the right approach. But what does this mean in practice? In order to deliver meaningful comparisons between customer groups and other segmentation variables, these must be decided first to drive the correct sampling so the resultant analysis can withstand the questions and criticisms that will come about in most organizations.

Another consideration is to put in the right recruitment and communications approach to prevent potential gaming by the generation of a skewed sample. What data to you collect? There is a controversy regarding the number of questions to ask? In order to answer the questions there a couple of key considerations. One is the impact of length of survey on response rates and the second is how to quantify the drivers of Net Promoter scores.

A two question survey will reduce survey fatigue and tend to be more exploratory in nature. Response rates will be higher. Multiple question surveys will deliver more underlying driver data, but response rates will drop. With a two question survey, the analysis of comments is time intensive and category based to perform root cause analysis. Multiple question surveys are less resource intensive to analyze and yields readily to statistical analysis. However, the development of the additional questions needs to sound in order to generate valid results.

How does one select the appropriate method for your firm? It turns out that while survey length does impact response rate. But after digging into the data, the data revealed there was a large difference in response rates with a two question survey from one to thirty eight percent. What are the steps that impact response rate? Engagement is the underlying driver of response rates, much more than survey length. The comparison was a ten to twelve question survey to a two question survey.

The underlying factors that developed included:

• Auditing the contact list
• Communications to clients around the program
• A relatively short survey that is focused and personalized
• Use standard text to avoid spam filters

To continue that comparison, the question of determining drivers and the categorization of comments versus the collection of driver questions yield different results. There were common themes but the ranking of the longer survey yielded more statistically valid results. Comments are frequently what's top of mind, but it may not reflect actual importance. Promoters tend to make less comments then detractors, skewing the result towards the negative and as a result, driving less information about what delights customers.

Ultimately, your firm's change strategy should inform your approach. Two questions are useful for customer recovery. A multiple question survey will reveal the customer recovery issues and support valid strategic analysis.

Vince is a great presenter. He was peppered with questions for 45 minutes afterwards in a very lively session.