A little knowledge is a dangerous thing

I was working with a client recently who had sketched out a fairly standard marketing dashboard. The company has multiple marketing channels (online, direct mail, phone, etc.) and wanted a simple report that would track leads, conversion rates, and cost per acquisition by channel. These were seemly valid and important things to know... but surprisingly slippery information the more we discussed it.

In their business, many potential customers are likely to have multiple interactions across marketing channels as they explore their options. One of these leads might first be drawn in through online search and later receive a piece of direct mail or a phone call. As a result, evaluating individual marketing channels in isolation delivers deceptive results. Which channel gets credit for the new customers? The better unit of measurement may be unique channel combinations.

It gets messier: cost per acquisition suggests to most dashboard readers that another dollar spent on that marketing channel delivers a specific return on investment. In reality, marketing channels are prone to diminishing returns as you dig deeper and push harder to find the next lead. In search engine marketing, spending more can drive competition for limited space and lead directly to a higher cost per lead. Conversion rates might better be understood as a curve rather than a single data point.

This a common situation that reinforces a few of the themes we like to harp on:

  • Analytics is a journey. A simple dashboard is a fine place to start, as long as the users commit to learning from the results and push for a more accurate representation of reality.
  • Ask the next question. Good analytics should stoke curiosity and push people to think more deeply about their situation.
  • Analysis before reporting. Premature reporting can lock down a mistaken version of reality.