Welcome, Guest Login Register
loading...
Net Promoter Community > New York Conference Blog 2010 > Tags > nowinski
 

New York Conference Blog 2010

3 Posts tagged with the nowinski tag

Once a customer tells you he or she is a detractor, It's obviously important to understand why--and to find out in a reasonably efficient, cost effective way. How much of that information can be extracted from the open ended "why" answers customers provide on NPS surveys? It's a daunting challenge. Customers aren't always precise, or complete in their responses, for example.

 

001.JPG

 

Satmetrix Director of  Methodology Dr. Vince Nowinski provided some in-depth insights.  The Stanford PhD described how "comment intelligence analysis" methods allow you to make conceptual sense of such data, to see with greater specififity where problems are actually occuring and what the underlying issue may really be.

 

But users of NPS much be careful not to look at verbatim analyis as the end all.  Rather, my understanding is that it helps you formulate intelligent hypotheses about where the critical issues with detractors lie. Further analysis will typically be required, perhaps in the form of focus groups, follow on surveys to targeted groups, individual interviews etc before you actually get at root causes of the issues detractors have.

 

Think of it this way. Without verbatims, you could only guess as to what's bothering detractors.  With verbatims and the analytical tools described by Vince, you can formulate a very good set of hypotheses for further testing.

 

Download presentation

... more >>
0 Comments Permalink

Dr. Vince Nowinski, Director of Methodology for Satmetrix, discussed a newly completed assessment of how organizations are doing when it comes to Net Promoter best practices. Dr. Laura Brooks was originally scheduled to present this topic, but fell ill with the flu…we were lucky to have Vince already with us in New York, since he worked directly with Laura on development of the assessment.

vince_nowinski_small.JPG

From Vince’s perspective, many companies excel in one area or another. But they are asking, “How do I know where the gaps are?” That’s the question he set out answer today.

Vince described why Satmetrix calls the assessment Net Promoter 360. The feedback on internal practices includes viewpoints from the Executive Sponsor of NPS within the organization, the Senior Executive Team, and Program Sponsors who manage the feedback operations. Think of this as a 360 degree view of what’s going on with your Net Promoter program.

Who Participated?

 

Vince presented data from a group of 22 different companies from 12 different industries, with an even split of B2B and B2C business models. Most of the companies (about two-thirds) had over $1 billion in revenue.

The assessment itself was designed jointly by Satmetrix and Bain & Company.  Questions on the assessment capture a mix of internal attitudes, behaviors, and business results to assess a company’s maturity level in adopting the Net Promoter discipline. It includes questions like, “Does the organization invest resources? Do executives believe the data they have is reliable? Do they understand what the top customer priorities are? Do they have closed loop processes in place to address individual customer issues?”

How Mature Are You?

 

Vince separated the 22 companies into 3 groups, to understand different levels of maturity.

 

Level 1: Developing

Adoption of Net Promoter is limited to some parts of the organization. Data was not yet trustworthy, or fully used operationally within the business.

Level 2: Progressing

The biggest difference here was that NPS had wider adoption, and companies were regularly taking action on the data. This group also had a big focus on root cause analysis…indicative of a phase of adoption where tactical insights are being developed.

Level 3: Advanced

These companies had Net Promoter fully integrated with the decision making culture. They were focused on understanding relative competitive position, and had processes in place to focus both on limiting Detractors and, more importantly, generating more Promoters.

Some Key Differences

 

Ultimately, Vince pointed to the measures of customer centricity as the most linked to overall level of adoption. Many people describe this as “culture.” I like to think of it as passion for the mission of the company.

What really struck me was the internal NPS for the Net Promoter program itself. Advanced companies had 96% Promoters of their NPS efforts, with no internal Detractors. These are organizations where all the key executives and core team members are “on the bus.” In contrast, those at the developing stage had closer to 50-60% Promoters and a group of Detractors within the core circle that should normally be leading the rest of the organization.

Advanced companies (100% of them) felt that NPS was involved in the day-to-day operations of the business, while none of the advanced companies described it as a “research initiative.” I found this point interesting. What language does your organization use to discuss Net Promoter internally? I’m sure the advanced companies have “research” resources involved with the program, but they understand that the ultimate value of the research is its use in strategic and tactical decision making. The other word that jumped out at me was “initiative.” The advanced companies don’t think of it as a one-time initiative. It is part of the way they do business.

Finally, Vince turned to the big question: What’s more important, eliminating Detractors or creating more Promoters? Ultimately, companies need to do both, but we found that the more advanced companies were in their adoption of NPS, the more likely they were to be focused on creating more Promoters. That’s really the end goal, isn’t it? You may indeed need to focus on eliminating Detractors, but if that’s all you ever focus on, it can be hard to define a winning vision and sustain the motivation of your employees.

Download presentation

... more >>
0 Comments Permalink

As you may have already figured out, the “Ultimate Question” that is the foundation of Net Promoter is not actually a question, but questions. Just like Batman cannot exist without Robin or Peanut Butter without Jelly  <insert cheesy analogy here>, likelihood to recommend has limited business value if there is not some way of tying context to the response that was given. The numerical values captured from clients is extremely useful for quantitative analysis, including correlation and regression analysis to identify drivers, while the open ended question is extremely useful for qualitative analysis.


Is too much data a bad thing? Not at all if you have the means to process large quantities of data. With numerical data, it is quite easy to ingest volumes of data into a database and crunch numbers with little to no regard for the volume that is being processed. However with unstructured data, like verbatim comments, we face a much more formidable challenge.


As Vince Nowinski, Director of Methodology, Satmetrix, shared during his session today, there are multiple ways of managing qualitative data in the form of customer verbatims.

 

  1. Single Punch Categorization – This is where most comment analysis occurs. Verbatim comments are “bucketed” into a single themes or category.  Categorization may be more subjective due to interpretation or bias from the person categorizing the comment.
  2. Multi-Punch Categorization – Similar to the above, however comments are bucketed into multiple themes or categories.
  3. Comment Intelligence - Using software algorithms, open ended feedback are processed analysis is performed by the incidence of particular words and their relationships with other words within the comment.  In contrast to the single-punch and multi-punch categorization techniques which require categories to be created by the person (or people) performing the categorization, comment intelligence uses technology that analyses the patterns which exist due to interreliability of the keywords in a data set.  Objectivity is not an issue since software performs the analysis.


The output of comment intelligence allows an organization to uncover relationships amongst different concepts and themes, along with the frequency and likelihood of association of keywords within themes.

 

For organizations with very large volumes of open ended feedback, comment intelligence solutions may be a good investment simply due to the time and effort required to manually process verbatim feedback.
While verbatim feedback is very useful to dovetail into quantitative analysis, Vince shared the following insights to take into account when using qualitative feedback:

 

  • Qualitative feedback diminishes as surveys lengthen
  • Open ended comments require more thought and effort on behalf of the client
  • Motivated customers share more and their sentiment influences responses
  • Tailored/targeted questions are best for asking verbatim
  • Customers are unreliable narrators and raters due to halo effects, fatigue, satisficing, ballistic responding and both cultural and response biases (Is it root cause or top of mind?)
  • Don’t’rely just on quantitiative feedback because you’re bound to miss what you don’t measure!

Used wisely, however, verbatim comments can inform thinking about the customer experience, shed light on how customer segments differ and prove better insights into root cause of customer experience issues.

 

Download presentation

... more >>
0 Comments Permalink