Juice Analytics

Scott Maxwell sums up 10 ways to lie with metrics. This is great advice for the political strivers and schemey backstabbers and a great antipattern for the readership of this blog. To summarize, here are his ten ways to lie when presenting data:

1. Only present metrics that are positive. That’s why you collect all those metrics.
2. Only present metrics that are easy to manage.
3. Use many metrics.
4. Be extremely precise with your numbers.
5. Present quickly, drown ’em with data.
6. Say “you don’t break down metrics” if they aren’t flattering to you.
7. Put lipstick on that pig—apply lots of gloss to your charts. Hello, Crystal XCelsius!
8. Show off your bona fides by sharing some metrics “off the cuff”.
9. Prep your team by feeding them lines.
10. Your job isn’t to educate your audience about your metrics. If people don’t know what you’re talking about, it’s because they’re stupid.

This is a great list, and it’s hard to avoid committing some of this sins from time-to-time. I think the best tool to improve your honesty when presenting numbers is to respect the intelligence and good judgment of your audience.

This isn’t easy; we all have people who can drive us crazy, who can derail a presentation with niggling questions or who ask for information they’ll never use.

There is no magic bullet when presenting numbers. Your job is not merely to show a few columns of numbers, but to teach your colleagues what those numbers mean.

[Editors note: Read the comments! David has some timely additions to this list.]

Dear PPT,

I have a feeling you knew this time would come. And I have to say that it’s not you, it’s all me.

You and I have been through hundreds of presentations together and our relationship has been pretty good. Up until now. You never demanded exclusivity and I didn’t imagine I’d need to go anywhere else for my presentation needs.

Sadly, it has come to this: I need to see other applications.

I’ve started to look around and I’m finding that there are many different ways that I can share information without your homeliness (mostly in the sagging visual and functional constraints areas). I know that sounds unfair—I have my own constraints. Face it, PPT, the world has changed and you’re just not the stunner you used to be.

Why, just the other day I had to share a lot of data with a client and wanted to flip through different scenarios. I admit that I have been spending a lot of time with your sister application Excel. But it’s not what you’re thinking. Once I took out the gridlines, labeled the tabs, jacked-up the typeface, and went to full-screen, I was practically in the world of slide-ware. And Excel offers built-in tables, calculations, nice visual alignment, more flexible charting, easy interactivity, and many of your drawing features. Excel’s been around the block but still managed to keep up with the times.

You might argue that we are talking about a non-traditional presentation format. But here’s the really bad news: I’ve found myself dazzled by Keynote. That’s probably the last thing you want to hear, I know. Unspoken, I knew I should have kept away, but once I had the Mac, that little temptress consumed my thoughts.

Oh, the shocking out-of-the-box design appeal is just too much for a presenter not to fiddle with. Check out a few of your default templates:

Now take a look at what Keynote will do for me:

Oh, PPT, why did you let yourself go like this? I know, it shouldn’t be all about sizzle. And to be fair, I’m not so sure that Keynote provides the range of functionality that I’ve been accustomed to with you, PPT. But it sure does push me toward better looking presentations. You know how visual we MBAs are.

I thought back to the times I used you to merely to compile and organize information. You know I dumped your other sister Word for this task (I hope Thanksgiving wasn’t too awkward for you that year). But then I ran into OmniOutliner. What a great tool for capturing information, organizing my thoughts, and structuring a story. Here’s a look at how it can integrate different file types into one information document:

PPT, you know I used to try and juice up my presentation while I was thinking through my message. But now I know that it is far better to separate data from presentation—two distinct steps. You may have noticed that when we spend less time together, I’m off writing in a plain text editor. Massaging my thoughts before I use your presentation capabilities.

We need to redefine our relationship. I still love you for your strengths but I need more than this. I have to go elsewhere for those things that you don’t do as well.

Don’t hold this against me. Particularly in the middle of an important presentation.

Regretfully yours,
Zach

P.S. See you Wednesday if it isn’t raining.

A Friend of Juice pointed me at a recent dispatch from the Change This web site entitled Elegant Solutions. If you aren’t familiar with Change This, it is worth a look. It is an outlet for big thinkers to publish “manifestos” on (mostly) business ideas ranging from marketing and customer service to strategy and innovation.

This particular manifesto by Matthew E. May (a trimmed down version of his book by the same name) reveals the secrets behind innovation at Toyota. It offers a number of themes that resonate with the discussions on this blog. A few of the high points:

“Toyota is in pursuit of ’elegant solutions to real world problems.’ Not grand slam homeruns, but groundball singles implemented all across the company by associates that view their role not to be simply doing the work, but taking it to the next level… An elegant solution is one in which the optimal outcome is achieved with the minimal expenditure of effort and expense… [and is] is recognized by its juxtaposition of simplicity and power.”

“Great innovation requires understanding and appreciating the concept of elegance as it relates to solving important problems. Oliver Wendell Holmes once said: ’I would not give a fig for the simplicity this side of complexity, but I would give my life for the simplicity on the other side of complexity.’ This is one of my favorite quotes—certainly my favorite by OWH.

Elegant solutions avoid the traps of: 1) Swinging for the fences; 2) Getting too clever—i.e. too many bells and whistles; 3) Solving problems frivolously.

“[Innovation] requires that we work the way artists or scientists do: accept the limitations, use them to our advantage, and pursue the simple question that drives the thinking behind every breakthrough, big or small: Is there a better way?” This idea of embracing constraints, which we wrote about a while back, is becoming increasingly embedded in business thinking.

“Artists and scientists own their work and sculpt their job. That’s new school. It’s a different mindset, and anything different is risky… New-schoolers know they’ll get pushback, but they trust their abilities, and continue to employ their ingenuity to explore and experiment with new ways of doing things within the confines of the organization.” Chris and I are sons of an artist who quit his job to pursue his dream of painting. That lesson stuck.

“Real learning is a cycle of questioning, experimenting and reflecting. It’s how we convert curiosity into an innovative solution.” This is a theme we constantly re-enforce with our clients—analytics is a journey of discovery. The end goal isn’t a report or analysis, it is a step that will reveal new understanding and help you ask a better question next time.

“Elegant solutions often come from customers—get out more and live in their world…. go look and see to fully grasp the situation; then, and only then, define the problem and design the appropriate solution.” We have been a proponent of looking at the raw data that describes individual customer behaviors–alternatively, there are companies like Lextant that specialize in helping businesses get closer to understanding customer needs.

“Focus on clear and present needs, or your great ideas remain just that. Make sure you’re concentrating on a real need. Don’t confuse an unarticulated need with a non-existent one. Don’t attempt to manufacture a need.” This is the fundamental problem with many data warehouse and business intelligence projects – in their attempt to be comprehensive, they minimize current needs and frequently miss the mark on future needs.

“Pictures and images connect people to thoughts and goals and help turn valuable ideas into action. So get graphic. Whenever you can, wherever you can, start building a visual element into your thinking… Digging into relevant data helps fight the dangers of bias, convention and instinct. There’s nothing better to help make the break with comfortable patterns than solid evidence. ” Preaching to the choir.

“Be-all, end-all, feature-rich solutions almost always miss the mark. Because they’re over-scoped and too complex. They’re usually proof that we lack real insight into our customer’sdesires. Complexity destroys value, which is what matters most to the customer. The most elegant solutions always seem blazingly simple.” Isn’t that just the way it is with many BI solutions?

The last mile of business analytics is poorly paved. Most of the effort and investment has been put into gathering, centralizing and warehousing data; relatively little time is spent on thoughtful, creative analysis and ensuring that results are communicated into the minds of the decision-makers. It is this last piece that may matter the most.

We can construct engaging stories from the data and put together winning PowerPoint decks, but the window of opportunity to communicate our results always seems frighteningly narrow. Weeks of analysis and synthesis can get crammed into a single 30-minute agenda-item. The cramming part isn’t so much the problem (it demands focus on the key results—I am more concerned about the fleeting attention of an audience that has a dozen other priorities, is awash in information, and may be data-phobic. Not to mention the risk of getting derailed by conversations about the data sources and statistical significance.

How can we share our analytical results in a way that will stick to the distracted mind of an executive? Moreover, is our obsession with top management misplaced when insights about the business should be spread to all levels of an organization?

One place to start is to consider how to break through the cluster of information with creative communication techniques. Here are a few ideas—I’d be interested in hearing your ideas or success stories:

• Catch them in their downtime. At one client, we created flyers that showed the results from a customer survey and posted them in the bathroom stalls under the title “Learn as you go.” We gave a captured audience something to read. Maybe we were too timid—why not go all the way with custom-printed toilet paper?
• A new format. When traditional slides seem to numb your audience, maybe a new information format is in order. Try a science-fair type poster, a web page, or a short book (check out self-publishing with Lulu). We once created a movie (Windows Media only) to show customer behaviors; a year later I got a request to show the movie in order to re-establish the key message. That’s sticky.
• One-page summary. Provide your audience with something to take away that summarizes your key messages. You might hand out (or stuff mail boxes with) a laminated one-pager with your most important framework and results. A colorful summary that begs to be thumbtacked to an office wall is better than a 40-page black-and-white deck that begs to be thrown out.
• 10’ display. It seems to be in vogue for companies to have big TVs in the lobby to stream corporate propaganda to the minions. Reach out to corporate communications to see if you can get into the program.

I’ve been trying to become better informed about the state of the business analytics industry. This has meant wading into the dreary swamp of sales deck webcasts. Here’s are a few thoughts on how to drain Sales Deck Swamp:

Respect your audience

Too many presentations ask that you attend a LIVE webcast at a specific time on a specific day. What’s more, they require pre-registration. Not all of us are willing to stick our heads that close to the maw of the sales-lion.

You can respect your audience by letting people watch on their own time on their own machine. The archive of the presentation IS the presentation. Make the archives of your webcasts prominently available, searchable, and historically accessible. Give people an e-mail address, or better yet, a open forum or comment system where they can ask questions.

Many webcasts are glorified sales presentations. If you accept that a webcast is a cheaper way of doing an initial sales meeting then the wisdom of “respect your audience” is clear. You wouldn’t tell a prospective client that if they want to see your sales presentation, they need to clear time on Monday at 1:00 pm. No, you’d let them find a time that worked for them. And you wouldn’t require that everyone who attended the meeting “register” and give you their name, address, phone number, and role at their company. You’d be happy if a few extra people dropped by. If the SVP of Marketing happens to drop in, so much the better.

Show a tool, not a sales presentation

If you can, show the product in use solving a real problem, rather than a canned series of screenshots. Jon Udell’s Screening Room is a great example of how to this can work.

Showing your product being used by a skilled user in a real situation helps me imagine how it could solve my problems. It also allows you to convey tacit knowledge—ways of working with your product that advanced users know but they can’t really transmit in words.

Show your face

Google has made available a series of in-house lectures on various technologies at Google University. These lectures aren’t presented in a sophisticated way, but to an analytics geek they’re pure gold. Do I want to hear Guido Van Rossum talk about the next generation of Python or Richard Hipp break down why he built SQLite? I do.

The videos are simple, presenter at a lectern, slides on the wall behind them. Yet they’ve been viewed thousands of times.

Would I like to hear a down to earth discussion about Cognos’ report builder or Business Objects infrastructure plans or Crystal XCelsius talking about integration with Excel 2007? It would be a pleasure next to some of the webinars I’ve seen.

If you work with data for a living, the following sites are worth a visit (or a subscription) to learn from some of the best, most passionate practitioners. While we’re at it, we are handing out our first annual Juicy Awards celebrating contributions to the analytics community. The categories are:

1. Charts and graphs

2. Information visualization

3. Presentations

4. Excel

5. Data analysis

6. Business intelligence

(4 through 6 are covered in part 2)

Here is an OPML file containing the finalists if you want to subscribe to these feeds with a single click.

Juice Resources for Analysts.opml

***

Charts and Graphs

These sites are dedicated to teaching the best (and pointing out the worst) approaches for displaying data in business environments. And the Juicy goes to (darn… this… envelope) Junk Charts for “recycling chart junk as junk art.”

Best of the rest:

• Stephen Few: “Thoughts about how visual representations of data and visual interaction techniques can be used in practical ways to analyze and communicate business information.” We like Steve for his tireless campaign to improve data visualization in a business environment as well as hard-hitting critiques of the established business intelligence vendors.
• Edward Tufte’s Ask E.T. Forum: The Don of Data Display. The Patriarch of Presentation. The Vicar of Visualization. The forum has a lot of great discussion about data presentation and charting.
• Communication Nation: “I believe we will be happier and more productive if we can improve our ability to communicate. This blog is dedicated to that effort”
• Stat. Graphics and Data Vis.: Straddling the fence between our charting and info visualization categories, this blog appears to have fallen fallow. On the plus side, it offers one of my favorite examples of a terrible 3D chart.
• Indexed: Sometimes a hand-drawn chart is just what is needed to tell the story.

Information visualization

Thse sites show off the best of the world of “info vis.” This is where art and design take starring roles, pushing aside the straightforward data display of traditional charts and graphs. Winner of the Juicy: Information Aesthetics for Form Follows Data. A daily dose of information visualization to spark your creativity.

Best of the rest:

• Info Vis Wiki: Community platform with a plethora of links and resources. As is typical of this category: a ton of information presented beautifully is still a ton of information.
• Visual Complexity: A gallery of visualization examples with particular focus on ways to display complex, multi-dimensional networks.
• NiXLOG Infographics: “A running collection of links to infographics found on the web.”
• Datapacifica: Tuned in to the latest examples of data visualization. [Edit: Jay Jakosky, author of Who Was Where, writes in to tell us that the blog has changed names and locations to Datapacifica from Who Was Where. The link and OMPL have been changed.]
• Visuale: Pointing readers to examples of cool infovis. Bonus points for using the famous Napoleon goes to Moscow graphic in the blog logo.

Presentation

Now you’ve got to put all your data together to tell a full story. The sites below give advice, tips, and guidelines for constructing and delivering your presentation. (We left out the bevy of PowerPoint tips blogs in part because of the ugliness of their presentation…the irony does not escape us).

The Juicy goes to Presentation Zen in a runaway victory. Garr Reynolds offers tips, tricks, and examples for making great presentations. An excerpt: “Edward Tufte says: “PowerPoint is Evil.” This got me thinking… What if Darth Vader — my favorite fictional bad guy — gave a formal presentation? How would it look? How would it compare to the presentation style of Yoda, the wise Jedi master?”

Best of the rest:

• MasterViews: Lively blog focused on PowerPoint.
• Signum sine tinnitu: Guy Kawaski, author of The Art of the Start, frequently writes about presenting and presentations.
• Maniactive: A fun blog that touches on both PowerPoint and approaches to presenting.
• Tony Ramos: A rambling list of links and thoughts that will keep you up on all that is new in PowerPoint, presentations, and presentation design.

Data analysts always face the same white-knuckle fear when they present: Will someone derail my presentation by questioning the source of a data point, the quality of the data, statistical significance, or why two numbers don’t align?

These types of inquires may appear innocent and within the field of play. I find them counter-productive. They implicitly undermine the analyst’s credibility and worse, deny the rest of the audience the opportunity to hear the full analysis narrative.

I’ve suffered through enough of these train wrecks to wonder about the underlying cause of this disruptive behavior. Here’s my theory:

People who have experience and comfort with numbers have the ability to abstract meaning from analysis for themselves—even when numbers don’t line up, the data is unclear, or the analysis has minor flaws. They can ask themselves higher-level questions: What does this mean? What are the implications for the business? How else could these results be interpreted?

In contrast, people who are uncomfortable with analytics treat numbers literally. They are disturbed by surface level inconsistencies. They expect—even need&mdas;hthe numbers to line up in straightforward ways. The medium simply isn’t familiar enough to abstract their own meaning.

Combine this uncertainty with the omnipresent pressure in business to express an opinion (any opinion) to appear smart—and you end up with wasted time spend discussing superficial irregularities.

Does this sound like the rantings of a data elitist? Fair enough, but I’d suggest that a similar phenomena is common across any field of expertise. Take art, for example. People who have little exposure to fine art are bound to ask for help in explaining the underlying meaning, remark on the superficial beauty, and find themselves attracted to the most obvious reflections of reality. Meanwhile, those who have experience can gather their own meaning, think creatively about what they are seeing and understand how it fits in context with other examples.

If this notion is accurate, where does that leave us? On the one hand, it highlights the need to understand your audience and anticipate and educate those people who are inclined to be disruptive. It also hints at some of the poorly understood realities of analysis:

• Analysis is as much art as science. Analysis relies on the perspective and skill of the analyst. Managers might want an objective reality, but they should recognize that the “truth” won’t tell an actionable story.
• Analysis should strive for directionally-correct results, not precision and comprehensiveness. Good analysis impacts decisions—therefore speed matters.
• Give people something tangible and incontrovertible to hang onto. This is one reason we really like the bottom’s-up analysis approach. When you create visualizations of individual customers, people can’t argue with this granular data—and it gives them something that they can fully understand and appreciate.
• Finally, there is nothing quite as valuable as personal credibility to make an analysis valuable. If people trust the work you have done in the past, they are likely to avoid arguing the small stuff.

To paraphrase from Really Bad Metaphors:

“Presentations can be as lame as a duck. Not the metaphorical lame duck,

either, but a real duck that was actually lame, maybe from stepping

on a land mine or something.”

Here are a few ideas to create spicier presentations:

1. Get thematic. Chose a theme that drives home a general concept in your presentation, then sprinkle it throughout your presentation. For one client, we presented an analysis that ended with a movie. Throughout the presentation we offered subtle hints in anticipation of our grand finale. Avoid themes that could also be used for a high school prom (e.g. A Winter Wonderland, Magical Memories).

2. Change of pace. Break up the action with picture-only slides (see Beyond Bullets, Presentation Zen) or multimedia (i.e. audio or video clip). A little fun in the middle of a bunch of dry slides can help wake up the audience.

3. Use a human voice. Surprise your audience by departing from the traditional business-speak. Try writing slides that don’t over-qualify the message or attempt to be comprehensive in the descriptions. Lawyers need to cover all their bases with that kind of language; you need to convey a simple message that will stick.

Instead of “IT organization has little understanding of business units’ long term strategic direction and impact on technology needs”, say “IT and business aren’t on the same page.”

Instead of “Disparate and redundant technology solutions leads to increased operating costs”, say “Technology tangle is taxing.”

4. Just the answer. Present your conclusion, then ask the audience if they they need to know more detail. If not, meeting over. I haven’t tried this, but surely a five minute presentation with a simple take-away will be as memorable as the typical presentation death-march.

5. Something to remember. Provide a simple, memorable acronym or metaphor so they will at least walk away with your key point. For example: Good presentation = f(S,E,C) where S = coherent Story; E = credible Evidence; C = Creatively told.

6. “And one more thing.” Steve Jobs is well-known for delivering one final surprise at the end of his presentation. Is there one last blinding insight that you’d want your audience to walk away with?

7. A new format. Consider stepping out of the traditional slide presentation format. Maybe you can convey your results with a science-fair type poster or with a web page or in a short book (check out self-publishing with Lulu). How about with charades or a Broadway tune? Too much?

8. Give them a souvenir. Provide your audience with something to take away that summarizes your key messages. For example, you might hand out a laminated one-pager with your most important framework and results. A colorful summary that begs to be thumbtacked to an office wall is better than a 40-page black-and-white deck that begs to be thrown out.

Bonus post: 4 Ideas from Microsoft for Presentations with More Zing (But Are More Distracting Than Useful)

1. Animation. Animation can be useful (e.g. “building” the content of a complex slide) but is overrated for livening up a presentation. The first slide animation is novel; the second is overdone.

2. Fancy template. Microsoft has been kind enough to provide a variety of dazzling slide templates—almost all of which distract from the content.

3. Clipart. A while back, we railed against Screenbeans, the little ant-like people that visually depict activities or moods. Those little buggers continue to be a bad idea.

4. Fonts. Especially Comic Sans. Won’t you join the Ban Comic Sans movement with me?

I spent the first half of our weekend morning stroll blissfully prattling on about the technical aspects of photography. I was attempted to explain to my wife, drawing on a high-school class and my flawed memory, the relationships between lens aperture, shutter speed, film sensitivity, focal distance, and light.

In the middle of this revery, my wife turned to me and explained how little she cared—and would it be too much trouble for me to stop talking about it? My semi-educational chatter was ruining the peacefulness of the walk.

I was confused. I know she loves to take pictures, and we’ve been having real problems with indoor pictures coming out blurry. Didn’t she want to know why this was happening and what it would take to fix it? Turns out she’d prefer if I’d just adjust the settings so the camera would work. Just the answer, please.

My wife is an attorney and has an amazing mind for the intricacies of legal problems. I don’t. Or more accurately, I have a visceral reaction when the conversation turns to torts, habeaus corpos, and subject-matter jurisdiction. I debate in my mind whether to try to follow the logic or simply space out. Like her, I just want the answer—not the journey.

All of which raises a question for business analysts: Is some of the resistance we encounter to data-driven decisions perhaps just a general queasiness with the detail? Perhaps the culprit isn’t some organizational culture for gut-instinct decisions or distrust of the data. It is just the journey (i.e. the presentations filled with charts and graphs and numbers) that causes people to disengage from the discussion.

On this blog, we’ve often argued that better data presentation can make your analysis more accessible and impactful. But maybe the answer can be less data presentation—until it becomes absolutely necessary.

There are many legitimate reasons for presenting the details of an analysis; there are also some poor ones. Consider whether you have explained too much because…

* you are self-conscious about your credibility?

* you want to showcase all the hard work?

* you assume the audience is like you?

* you want our presentation to lead up to a dramatic conclusion?

Analytics truism: everyone wants a dashboard (a.k.a. key performance indicators (a.k.a KPIs), success metrics, scorecards). Managers want a barometer of performance, a hammer to use on their subordinates, and a straightforward quantification of their business. Below are a few of the guidelines we use when we take on this task:

1. Actionable metrics. Ask yourself: what would I do if the metric is out of line? Do I have the levers that can impact it? Measures that track final outcomes like revenue or total customers don’t give you much time to react or guidance about what to do next.

2. Less than five. When I first started at AOL, a friend of mine pointed to the dozens of reports flying around the organization and remarked (I paraphrase): “This many ’important’ metrics just indicates that nobody really understands this business.” If you struggle to boil down, you should spend more time defining success and understanding the factors that drive performance.

3. Simplicity over comprehensiveness. We don’t agree with Thomas Davenport’s call for more proprietary metrics:

You know you compete on analytics when…You not only are expert at number crunching but also invent proprietary metrics for use in key business processes.

In our experience, you’re better off if you choose metrics that can be understood outside your corner of the world. One common trap we’ve seen is a desire to create a single comprehensive metric; this metric is often an index that combines a number of factors into an overall measure of performance. The result: numbers that are meaningless without a lot of context and difficulty in interpreting deltas.

4. Presentation matters. Your dashboard should be easy to understand and provide enough data to give your audience context. I’ve seen many dashboards that stubbornly show only the current state of a metric and the change from the previous week. Why so stingy with historical data? At Juice, we always show trending and try to give users a means to “cut” the data – by business line, customer type, month, etc. Check out our template for creating a success metric dashboard (more info below).

5. Evolve to goals. Metrics without goals can be a waste. Unfortunately, getting people to agree to specific targets can be painful. After all, goals start us down a slippery slope toward clear accountability. Here’s what I’ve found works: start by focusing your energy on getting people to buy-in to the success metrics. Get clarity on definitions, show trending, and incorporate them into the organization’s vernacular. Be patient: one day someone will raise their hand in a meeting and ask if there are targets for the metrics. Pretend to act surprised by the cleverness of this suggestion.

The success metrics template makes it easy to quickly put together a top-line report for your organization. The ’data’ worksheet gives you a place to put in your weekly (or daily, monthly, etc.) metrics. Add in dimensions where appropriate. Saving and re-opening the spreadsheet will refresh the pivot tables. Shoot me an e-mail if you have questions.

The Dashboard Spy has a great blog about business dashboards.