The first presentation is by Dr. Vince Nowinski, the Principal Methodologist at Satmetrix (co-developer of Net Promoter). He is the guy who works to maintain sound methodological standards with Satmetrix and based on my observations, it appears he is succeeding. His presentation today is entitled "Measuring What Matters - Capturing the Voice of Your Most Important Customers." Vince's approach to the discussion is to cover the "who," "what," and "when," as it relates to sampling strategies, response rates and general survey design. This is a hotly debated subject and has brought much discussion thus far at the conference.

One of the underlying messages from Vince was that getting 'statistical representativeness' of your sample may not be the ultimate goal, especially when considering the differences between B2B versus B2C businesses models. The main takeaway I heard was that when in a B2B environment, particularly when focusing on an enterprise account structure, a census approach is preferred when as compared to a more traditional sampling methodology.  Vince was recommending that you identify your key decision makers and with assistance from your field sales organization, obtain survey responses from 100% of these folks if possible.

Vince then discussed the considerations around survey length, and raised the debate between survey design using "the one number..." versus the more traditional approach.  Data was shown around survey length and response rates, and I was very surprised that the difference in response rates between a very short survey and the longer survey was actually very small. Vince's point was that response rates were influenced more strongly by ensuring that the contacts were the most relevant in determining your relationship and the possibility of future business.

One last point that I took from Vince's presentation was how each of these 2 survey design philosophies could the impact the ability to determine key drivers. My key takeaway is that using the shorter survey, with perhaps the Recommend question and 1 open ended question, the ability to identify accurate drivers of loyalty becomes difficult. This is because the short survey provides you very few options for analysis.  The only tool you have available is to dive into the verbatim comments and determine qualitatively the key drivers. Using a more in-depth survey design will allow you to use statistical techniques to determine drivers, which to me seems like a more sound approach.

I will be interesting to see how this subject evolves over time as we begin seeing some long-standing practices being challenged.