Paul Graham, from The Power of the Marginal

"Rising up through the hierarchy of the average big company demands an attention to politics few thoughtful people could spare... I think that's one reason big companies are so often blindsided by startups. People at big companies don't realize the extent to which they live in an environment that is one large, ongoing test for the wrong qualities."

A while back, we got all over Thomas Davenport for his checklist of ways to tell if your organization is an "Analytics Competitor." To me, he had posed the wrong question. It asks too much and reveals too little.

I don't need to know, for example, if I can chip like Phil Mickelson and finish off a tournament like Tiger Woods. I just want to be a decent golfer who isn't embarrassed during a corporate outing. Organizations don't need to know if they can go head-to-head with Harrah's (the new popular case study, displacing Capital One); they just want to be smarter about their decision-making using the data they have.

Creating a useful analytics capability requires a number of pieces to come together. Here's our take on the important things to evaluate as you consider: Where do I stand with my analytics and how can I get better?

Strategy

• For each business function or line, how will analytics impact decisions?
• What decisions will not be impacted?

People and tools

• Do I have people who understand the dynamics of the business (i.e. can pull their head of the data and see the context for an analysis)?
• Do I have people who are skilled in basic analysis approaches and tools? (e.g. PivotTables, simple modeling, basic statistics)
• Do I have people who can effectively communicate the results of their work through simple, attractive data presentation?
• Are there analyses or reports that I cannot accomplish with my current toolset because the data sets are too large or the statistical requirements to heavy?

Process

• Do I have a process for working with the business lines and functions to understand their needs?
• Do I have a process for defining, developing, producing, and delivering reports?
• Do I have a process for managing the ever-expanding queue of requests?
• Do I have templates for data presentation, reports, common analyses, models, etc. that will make my work more repeatable and efficient?

Raw materials

• Do I have a low-friction means to access the data that is the raw materials for my work?
• Do I understand the issues and intricacies of my organization's data? Have I documented it?

Integrate into business

• Have I proven the value of analytics through visible “wins”, i.e. real, live (and successful) cases where reporting and analysis is driving business decisions?
• Have I achieved a seat at the table, i.e. genuine involvement in decision-making process?

If I could influence the future of business intelligence tools (wait, maybe I can), I would put a premium on "tangible" data manipulation. I'd design interfaces that let users touch, play with, and sculpt data as an object.

Many data crunching applications, particularly those focused on statistics (e.g. SAS), tend to separate the user from the act of data manipulation. The user defines a set of scripts or formulas, points to a data set, and let's the application take over. For a programmer, this type of abstraction works. For non-technical business folk, it limits our ability to understand what is happening and why the result turned out differently than we imagined.

Here are a couple interesting examples of computer interfaces that attempt to merge real-world touch and feel with digital-world manipulation of information:

Via Information Aesthetics

What if BI interfaces brought an artisan's mentality (I'm imagining glassblower for some reason) to data manipulation? Data is the tangible raw material. When there was something odd or imperfect in the raw material, it would be obvious on visual inspection. We'd have access to a variety of tools, some for broad and crude actions, others for a more delicate and subtle actions. These tools would be put in physical contact with the data to shape it. Finally, we could add a final aesthetic finish to our creation. Analysts could take pride in creating digital objects that could move and influence others.

Related thought: can we blame the poor visualization of analytical results on the lack of visualization in the data analysis and manipulation process?

Business intelligence expert Claudia Imhoff of Intelligent Solutions describes the end-game for business intelligence in something she calls the "Corporate Information Factory" (plus an "e" for Extended). CIFE is a comprehensive ecosystems of people, processes, systems and applications to deliver all the promise of business intelligence for an organization.

Here's the problem: most businesses looking at this picture are going to feel intimidated and inadequate, like presenting to Steve Jobs (entertaining for Martha Stewart?). Below is my view of the common reality of business intelligence—it is a far simpler picture, lacking the governance, data management processes, multiple data marts, and overall data discipline.

Of course, CIFE probably wasn't meant for comparison; it is a long-term vision. Even so, the gap between reality and this beau ideal of BI may be too vast to be useful for many organizations.

The real questions: How do I get started in the right direction? How can I make better use of my business data without all the large investments implied by the CIFE model?

Over the next couple of weeks, we'll share our perspective on low-risk, practical actions with immediate impact. Here's some of what you can expect...

Step 1: Where do I stand?—Asking the right questions to start the journey

Step 2: Love the One You're With—Making the most of Excel

Step 3: The Learning Journey—Investing in analysis before reporting

Step 4: Just Say No—Reducing the pain and increasing the value of reporting

Step 5: Small Victories—Getting the organization to care about analytics

Here's the third (and hopefully final) in my series on interesting podcasts. This one comes from the folks at IT Conversations. It is a recorded presentation by Ray Lane from the Software 2006 Conference discussing the shifting landscape of the software industry. (Ray's slides are here.) Ray is the former president of Oracle and current partner at venture capital firm Kleiner Perkins.

About 26 minutes in, he gets to the part that intrigued me. He provides a description of what it is going to take to develop the successful and valuable enterprise software of the future. His thoughts resonated with ideas we've had about the weakness of existing business intelligence solutions and what we'd do to fix it. Here is my favorite quote:

Enterprise software industry made one big, huge mistake in the late nineties. It focused on buyers and forgot the users. [They need to ask] How are the users really using the software?

When it comes to new software opportunities, he recommends that new entrants:

1. Target the white space. "Find the white space, the open space, the stuff that hasn't been done. Don't try to do it better than Oracle, Microsoft, SAP. Enterprise software hasn't done everything. There is still a lot of manual decision making, a lot of manual effort, a lot of cost that goes in."

2. Low effort improvement. "Improve what they have today but do it low effort." He mentioned an example of an innovative company that could install its software in a week.

3. Free now, pay later. "Free so i can try it and actually see the value very quickly, then pay for it later. Trust that the customer will see enough value that they will pay for it later."

4. Generate individual value. "We make different technology decisions at home that at work. Why? because there is something that says: the enterprise is bigger than us. Take the [Amazon.com, Google] mindset into the enterprise."

I'd add one more item to his list:

5. Solve specific problems. Too often, enterprise software is built to be comprehensive and generic so it can marginally solve any problem. Better to flexible and modular to allow rapid implementations that target the biggest pain points or opportunities for individual client situations.

Yesterday, Google rolled out new mapping features for Google Earth and Google Maps. Many of these features are behind the scenes in the APIs, but there are great new capabilities that you will start to see. One thing I'm excited about is that KML—Google Earth's format for building sophisticated map overlays—has come to Google Maps.

Google demoed this at their Geo Developer day yesterday using one of our Google Earth overlays that shows US census bureau data by county mapped as a heatmap. It looks like this.

Counties are displayed in a list on the left. When you click on a county, you get a nice popup showing statistics for that county.

There are a few limitations. Large KML files don't load in Google Maps, medium-sized files load very slowly—it seems Google is parsing the KML using Javascript.

The mapping toolkits provided by Google Maps, Google Earth, and Yahoo Maps beta are well on their way to becoming important business tools once developers figure out how to wire in your enterprise data.

Without further ado, here are some US census data maps for you to explore in Google Maps.

Population Density

Lighter is higher population density (white is 800+ people per square mile), Dark is lower population density (black is 2 or fewer people per square mile)

Median Age

Lighter is older median age (white is 46.0 years median age), Dark is younger median age (black is 29.0 years median age)

Male/Female Ratio

Lighter means more men than women (white is 55% men), Dark means more women than men (black is 45% men)

Science and art have in common intense seeing, the wide-eyed observing that generates empirical information. Beautiful Evidence is about how seeing turns into showing, how emipirical evidence turns into explanations and evidence presentations. The book identifies excellent and effective methods of presenting information, suggests new designs, and provides tools for assessing the credibility of evidence presentations.

Edward Tufte's Beautiful Evidence, is now available for purchase. This is the labor of years of writing, thinking, and discussion. The book is physically beautiful--cloth-bound, five-color printing--and has the physical and intellectual heft to beat down opponents of good design.

Tufte covers the following topics.