Survey Analysis Grows Up with SurveyVisualizer

Luc Girardin of Macrofocus contacted us in response to our post "When Will Survey Analysis Grow Up?" to point us to their SurveyVisualizer analysis tool. I had a little time this weekend to download and play with this application. There is a lot to like.

SurveyVisualizer is designed for surveys that have a hierachical or tree structure. Luc describes the relevant data structure in a background paper about the product:

The questions—also called quality criteria—are then aggregated into 23 quality dimensions (e.g. network quality, ticketing, cleanliness, security, reliability). They represent the level of satisfaction with a whole group of questions pertaining to a particular issue. The quality dimensions themselves are further aggregated into three different customer satisfaction indices, reflecting the different areas of responsibility.

The free download has multiple satisfaction "criteria" (e.g. friendliness of crew) roll up to "dimensions" (e.g. cabin crew) which fall under "indices" (e.g. index of flight services). This may be an appropriate structure for a satisfaction survey—but it isn’t one I’ve encountered before.

Despite this limitation, the analysis capabilities delivered by SurveyVisualizer are intuitive and innovative. For example, all your survey data is displayed at once in a kind of relational map. This lets users visually identify patterns in the full set of results. Each of the vertical hashes represents a question or roll-up of questions. Clicking on any one of these hashes highlights the hierarchical relationships. The "ghost" lines represent the results across questions for a multitude of dimensions or respondent types.

Users have the ability to select specific dimensions to identify patterns in the corresponding results. An easy-to-use interface lets you choose a dimension then apply a color to the line within the relational map.

Also, users can click on individual display lines to investigate the results (e.g. I wonder who had that particularly crappy score for flight delays?)

If your analysis requirements don’t fit this particular structure, Macrofocus has a more general-purpose tool called InfoScope.

When Will Survey Analysis Grow Up?

At the 2004TED conference, Malcolm Gladwell tells the story of Dr. Howard Moskowitz, a man who revolutionized the prepared food industry through a new kind of analytical thinking. Long story short: Dr. Moskowitz was one of the first people to argue that companies should pursue multiple products targeted at customer subsegments rather than try to create the perfect product for all customers. He realized that an attempt to create a "platonic ideal" —whether it was pickles, mustard, or pasta sauce—would be a suboptimal result for most consumers. Consumers are individuals with preferences that are better clustered than averaged. Mr. Gladwell states that this change in business thinking (spurred by Moskowitz’s study of pasta sauce) mirrors a more general scientific shift from a focus on universal truths to the study of variation.

The prepared food industry gets it—as evidenced by nine variations of Ragu sauce on the grocery shelves—but I’m not convinced that these lessons have permeated the rest of the business analytics landscape. In particular, I am struck by the inability of most survey analyses to reveal insights about respondents.

The tools may be part of the problem. Here’s an example of what WebSurveyor provides its users to help them analyze online surveys:

Their site tells us:

"Each question is graphed independently allowing you greater flexibility in customizing the layout of reporting for each question...Filter results based on specific responses or cross-tabulate results from two different questions, giving you powerful tools for detailed analysis."

Powerful? Flexible? More like barebones. WebSurveyor is putting the analyst in a very constrained box that won’t help deliver an better understanding of respondents. WebSurveyor’s tool demands "question-centric" not "customer-centric" analysis.

Consider how this typical survey approach would serve you in an effort to understand the passengers of Noah’s Ark. A surveyor would ask each animal to fill out basic information about their height, weight, number of legs, food preference, etc. The results would then let us know that the average animal weights 23 pounds, has a height of 1.2 feet, 5.6 legs, 30% omnivore and so on. All of which would miss the essential insight about the animals on board: there are two of each.

Unfortunately, the kind of analysis needed to reveal personality / needs / behavior clusters in your respondent population isn’t well supported by out-of-the-box analytical tools. One approach is factor analysis—a statistical technique that is used in marketing to "identify the salient attributes consumers use to evaluate products in a category" (Wikipedia). Another approach is to examine individual visual representations of individual respondents—a technique that we term (rather clumsily): customer flashcards.