Open up your CRM reporting tool. If it looks anything like the tools I've seen, it's all about filters and metrics and bland line graphs. Now imagine it worked a little more like the new Google Finance dashboard:

or like the Baby Name Wizard:

or had an interface like News Map by Marumushi:

Why is it that consumer data visualization tools are so much cleaner, easier to understand, more polished, and engaging than enterprise reporting/analysis tools? I have a few theories:

1. Reporting comes last. Reporting is typically the last thing considered and created in development projects. Visibility into the data becomes less important that ensuring data entry, integrity, management, and access. Alternatively, it is a matter of the developers playing to their strengths. Design and data visualization clearly aren't passions - or they'd be unable to launch with these tools.
2. Reporting trumps analysis. There is too much emphasis on delivering raw numbers, and not enough emphasis on presenting information in ways that allow users to understand what is going on. Put another way: success is defined as delivering accurate data filtered as the user has defined it. What if success was about delivering new insights about what's going on in the enterprise?
3. Comprehensiveness is king. I can just imagine the list of requirements stacked on the desk of the team tasked to build the reporting interface. It's a long list and shoehorning all of it into an interface is hard. The irony is, from what I've seen, really useful data views are left out (like time series and animation).

The examples above offer smooth-animation, mouse-over discovery and highlighting, and a consistent color palette. The baby name explorer and NewsMap are also specific tools that address one specific need very well.

In a previous post, we described how NFL coaches and players use film study as their approach to analysis. We argue that "slicing and dicing" statistics doesn’t help much when deciding on a game plan. Business intelligence tools can explain the size of the problem (how good is the opponent?) and trends (what are their preferred offensive weapons?). These same tools do not, however, provide real perspective on customer behaviors or insights that give your organization data-driven direction.

The question remains: How do we bring the value of film study to business intelligence? The solution we've used is inspired in equal parts by Edward Tufte and Malcolm Gladwell.

Tufte is a well-known expert in presentation of informational graphics. Among numerous concepts, he popularized the idea of sparklines: "data-intense, design-simple, word-sized graphics.” Here is a sparkline showing deficit spending from 1983 to 2003: What if there was a way to create data-intensive pictures that represent customer behaviors? They could draw on customer usage of a product, marketing touch-points, service calls, and any number of other relevant interactions. The goal: create a simple representation that quickly shows customer behaviors that matter to your business. Here are a few examples from our work:

These pictures are intriguing, but can they be useful? In his book Blink, Malcolm Gladwell introduces the idea of thin slicing: "the act of relegating the decision-making process to the adaptive unconscious by focusing on a small set of pertinent key variables, as opposed to consciously considering the situations as wholes over much longer periods of time." He explains how people become experts at quickly evaluating the relevant data and arrive at a rapid understanding of a situation.

We want to give business people a sense of their customers in a blink of an eye. To do so, customer sparklines need to be intuitive and easy to learn. Success is the ability to show these pictures to anyone in the organization—from senior executives to front-line customer service reps—and have them grasp what they are seeing with just a few minutes of explanation.

The next version of Microsoft Office boasts many graphical improvements. Like some people, Microsoft seems to have confounded decoration with communication.

Productive environments separate content from presentation (consider Textile and/or Markdown) and let people concentrate on communicating.

Instead, Microsoft's SmartArt gives you lots of choices...

...to let you do this...

Users are going to burn tons of hours twiddling all these different options - I'm sure I won't be able to help myself, for that matter. Result: organizations full of users concentrating on their own self-expression, not on communicating clearly with each other.

The situation with Excel charting is even worse, as Stephen Few has recently pointed out.

Poor Justin Gatlin. The guy sprints to a world record 100 meters then sees his record taken away due to rounding. His 9.766 time got rounded up to 9.77 -- leaving him tied with Asafa Powell. You might think hundredths of a second offers enough precision to distinguish winners from losers. As sprinters cross the finish line they're traveling about 15 centimeters each hundreth of a second. Does recording 100 meter times in hundreths of a second provide enough detail to record the desired outcome: determining who is the fastest man in the world?

In Justin's case, he was running a race that was measured with insufficient precision. He may be the world's fastest man by a few inches--enough that an attentive spectator would notice.

In business, however, we often err in the opposite direction. We measure and argue about precise numbers when directional accuracy is all that's needed. If I were to run a race against Justin Gatlin, a sundial would provide enough detail to show the faster runner.

Ask yourself, how much precision do you need? In the case of Zillow, the online home valuation database, too much precision can be a problem. It can mislead users to think the precise numbers are also accurate. Precision has costs. Not always in money, but often in attention. Have you found yourself derailed in a presentation because the Marketing's numbers don't exactly square with the numbers from Operations? Does it matter? Probably not.

Like our confused feelings for Tom Cruise, our love-hate relationship with Excel rages on.

We've railed against Excel's ugly charts, inactive user community, and plans for the Excel 2007 user interface. Now here I am doing an ankle-snapping reverse to share some cool stuff we've done recently to create quick, easy-to-use applications for list management. First, we created an interface for a searchable database of people, like an alumni directory or yellow pages. More recently, we put together a tool to keep track of lists of people - like a classroom attendance log or mailing list manager. It's all possible in Excel.

First a disclaimer: this isn't a claim that Excel has proven to be an ideal tool. We ran into all sorts of limitations when we asked Excel to act like a database, work across multiple lists, and input new data. To a certain degree, MS Access may have been a better option.

That said, there are situations where Excel's interface features (e.g. check boxes, drop-down lists), user familiarity, data manipulation, and VBA fit the bill. For instance, you may find yourself...

• Wanting to give users a tool they are familiar with
• Needing a solution that can be thrown together in weeks, not months
• Facing users who don't always have Internet access
• Independent of a central database
• Wanting to easily report and analyze your data

Here's a taste of the two applications we put together recently:

Member Directory

* Search using the first letters of last name

* Filter search by various attributes

* Find members within a specified distance

* Map results in Yahoo! Maps

(Here, your selected list shows up in a Yahoo! map)

Attendance Tracker

* Search for individuals by first letters of last name

* Add one person at a time to the master list

* Create filtered lists and select which people you want to add to your master list

* Save, open, delete lists

* Export lists to a separate data file

* Report on attendance participation by event, participant, date, etc.

The Google Earth User Interface is controlled by a mysterious cabal directed by the Vatican. Whoops, check that - Da Vinci Code marketing cross-talk. The Google Earth User Interface is really controlled by a simple, easy-to-read, easy-to-modify XML file. This means it's easy to show, hide, or modify elements of the user interface. Here's an example.

Observe the floating navigation panel with more room for the Earth in Google Earth. The navigation panel can also be hidden, or dragged to a different screen. Convenient.

This change is simple to make by editing the Google Earth "kvw" (Keyhole View?) file. On my system, this is found at C:\Program Files\Google\Google Earth\kvw\default_lt.kvw. This is a simple text file. Open the file in a text editor and find the Navigation Panel windowStack. Change the location attribute to "float" as you see below.

Make a backup before you start making changes. More extreme changes are not kosher, resulting in Google Earth failing to start. Restoring the kvw file from backup will fix the problem.

[Added] OgleEarth provides directions on using this hack on Mac OS X.

For the Mac, right-click on the Google Earth application (when it is not running), select "Show package contents", then navigate on over to:

/Applications/Google Earth.app/Contents/MacOS/kvw/default_lt.kvw

Open it in a text editor and apply Juice Analytics' hack. 

Incidentally, the Google Earth map you see above is graphing hurricane paths of the last 5 years with color coded intensities using a Python KML library. Just trying to keep up with the Joneses (the Joneses, in this case, being the Timoney group with their excellent start to Google Earth based analytics).

Insightful analysis of data is important whether you are in business or sports. However, the approaches used in these arenas couldn't be more different.

Take the NFL, for example. Coaches and players spend hours analyzing film to identify the strengths and weaknesses of opposing teams. As much as any professional league, game planning can make the difference in setting-up a team for success. For players, film study is the mental foundation that lets instinct take over on the field. Given the importance of this raw data analysis, it is worth tapping into my ESPN Sunday morning education (thank you, EA Sports NFL Matchup) to considering the techniques involved:

Get granular: Examine the raw data like where players are positioned, who gets the ball from different formations, what plays are called at different field positions, and even what techniques are used by individual players.

Use your eyes: Rely on your brain's powerful ability to recognize patterns (or "tendencies" in football-speak). Record these patterns by player, by formation, by down and distance.

Create a common context: Given the volume of data, it is important to focus on the differences while holding everything else constant. For the Tampa Buccaneers (and presumably most NFL teams): "every game and practice session - every step, block, throw, kick, zig and zag - is captured on film from two bird's-eye views: sideline and end zone. The tapes then are intercut so each play can be seen from both angles."

Group common patterns; highlight anomalies: The building blocks of analysis become the common patterns. How frequently does the team behave in a certain way? Does this occur more often in the red zone? Also, after watching enough film, unusual actions on the field pop-out quickly.

Bottom-up strategy: Finally, use this deep understanding of all the opposing team, players, and patterns as the foundation for the game plan.

This type of approach is a far cry from the types of drill-down analyses most common in businesses. Imagine if a NFL team depended on business-type statistics and OLAP reporting. How valuable would it be to look at:

• Average yards per carry,
• Trends in passing versus rushing yards,
• Distribution of touches by player,
• Individual player statistics?

This type of "slicing and dicing" of the statistics can show the size of the problem (how good is the opponent?) and trends (what are their preferred offensive weapons?), but it wouldn't get a coach much closer to figuring out what to do about it.

And that typifies the stuck state of analytics in business today. Business intelligence tools are good at reporting and showing trends. These same tools are not good at understanding customer behaviors or complex processes--the types of understanding that provides a solid foundation for marketing, operations, and even corporate strategy. The tools don't help with the one question that matters whether you are in business or in sports: what should I do based on the evidence?

In Part 2, we'd like to share one idea of how to get deeper into the data so you can get smarter: an approach we call "customer sparklines."

Brian Timoney of The Timoney Group just launched a great web site that gives a feel for the types of analysis that can be done with Google Earth. The site, entitled Energy Impact of Rita and Katrina, lays out a multitude of geographic views (available as .kml files in Google Earth) that show the hurricanes' devastation in the Gulf region. For example, here's a look at the Gas Production in 2004 vs. 2005:

That's just the eye candy. Even better, Brian has developed a series of Google Earth tools that let you analyze the actual geographic data. Below is the description of the a tool that lets you choose a buffer distance around a path.

Last week I attended the 2006 Excel User Conference. A few immediate reactions to the conference:

• Roughly 50 people attended, which made for a nice, close-knit setting that encouraged questions and discourse between the participants. But seriously, 50 people? I know 50 people myself that could have gone to this conference and saved themselves endless hours of frustration. The cost of conference fees and taking 2 days off work would probably be paid back by increased productivity within months.
• Overall, the conference was very comprehensive and every question I could think of was easily answered by the presenters. The speed at which I learned new techniques for manipulating Excel's functions or charts was amazing once I had someone to show me visually while explaining it. I guess it just goes to show how simple peer-to-peer learning can be the best way to quickly learn.
• A good craftsman never blames his or her tools, but even good analysts many times find it hard to deal with raw data or graphing their analysis when using Excel. One major theme I took from the conference is that Excel is all about details. All the formulas, formatting, and open room to do whatever you want is often overkill when someone is looking for a particular solution. Many questions people have about doing something in Excel is usually one formula or right click away. Learning Excel requires attention to detail.

Next week I will share some of what I’ve learned about Excel through dealing with Chris and his nonstop Excel wizardry and from this conference including:

1. Methods for finding answers to your Excel questions – Tutorials, tips, and forums where you can go to find the answer to your specific excel questions.
2. 10 things I wish someone had told me 5 years ago about Excel – Procedures and tricks I use every day in Excel.

Perhaps the main reason for the lack of Excel power users lie in the fact that many people don’t realize what they don’t know about Excel. With just a general understanding of Excel’s potential, most people have the ability to take the extra steps needed to be their own source for analytical understanding.

For our European readers, here's a company that does a better job articulating what we feel about business analytics than we do.  It is like they are in my head.

On the increasing amount of data...

The amount of complex data is growing faster than our capabilities to analyze it. Large investments are put into collecting data and building data warehouses. How to get something out, and use the potential of the data collection often comes as an after thought only.

...and the ineffectual tools for understanding this data...

The methods and the technology to keep an overview of this flood of data, to put it in context, and turn the resulting information into useful knowledge are still few and far. This distilled knowledge however is what will give a competitive advantage to companies trying to stay afloat in the emerging knowledge economy.

The tools that do exist are linear and machine-centered, and force people to formalize their thoughts and adapt their way of working to the rigidity of machines. It is in this gap that we see our opportunity to make a difference and be successful.

What is needed are tools that fit the fluid way in which we think and work and that are intuitive to use.

There are surprisingly few companies that really understand these problems and have started to chart a better path.  We've spoken to a few - DecisionStudio, Perceptual Edge, and even Anant Dhingran at IBM - but we'd love to hear from more.

Here's a screencast showing the Excel Chart Cleaner tool being used.

Unlike previous screencasts, this one is not inline with the blog, but is on a separate page as I needed more elbow room to show the charting process.

The screencast is less than 5 minutes in length, so there is no real discussion of Excel charting problems. You can find those here or here. The video is safe for work.