via Dashboard Spy

Real-time dashboards — the kind that show up on a big screen in a call center — are an entirely different beast than your standard management dashboard. Their job is to support immediate decision-making. As a result, the information must be easy to interpret, alert users to problems, and make the next action obvious. In addition to key success metrics, real-time dashboards may show detailed data about the action “on the ground.” Here are eight characteristics that can make a real-time dashboard effective:

1. A summary status that indicates how things stand overall. Users need to be able to tell at a glance whether they should worry or not. Here’s a great example from the folks at Superblock. The “Is it going to rain?” site tells you the single most important thing you need from a weather report.



2. Reflect a well-understood structure of the business. By the time you design a real-time dashboard, you should have a strong theory for how the pieces of the business fit together (i.e. the relationships between key measures, drivers, and available actions). For example, in the call center business, there are clearly defined success measures (e.g. wait time), a mathematical relationship between these measures and their underlying drivers (e.g. call volume), and known levers to address problems (e.g. staffing levels).

3. Support quick diagnosis of problems. The data presentation should point directly to the likely source of the problem. Real-time dashboards aren’t the place for deep analysis or introspection into the drivers of the business.

4. Simple data presentation. In my view, real-time dashbaord’s aren’t the place for complex or advanced data visualizations. Imagine you were Napoleon and you had to use a half-completed version of this chart to make a battlefield decision in the next 5 minutes.



5. Granular view of the “unit of action.” Real-time dashboards are often about tracking activity. It may be useful to show the raw data around these events in the form of a ticker, scroll or RSS feed. We use at a real-time tracker for our website called Sitemeter. It does a nice job of tracking the basic unit of action — visitors.



6. Appropriate time window. Getting time right on an operational dashboard is critical. If the measures and trends represent too long a time period, users may not react to changes quickly enough. On the other hand, very small time windows encourage frantic reactions to changes that may not represent real trends. Ideally, the dashboard should offer the ability to configure this time range and “freeze” a moment in time.

7. Prominent but balanced alerts. Naturally, alerting users to problems is a central mission for real-time dashboards. The challenge (as always with alerts) is to balance between “the sky is falling” hysteria and “don’t worry, be happy” apathy. I’ve written before about alerts, but one item to emphasize is the need to show a sense of relative importance. Not all problems have the same impact on the business, and finding a way to communicate this relative importance is valuable.

8. Point to specific action. If real-time dashboards are about identifying and responding to issues, the tool should point users to what they can do about a problem. This can be as simple as displaying the phone number of the right person to call.

Real-time dashboards can be ignorable, create mayhem, or drive great behavior in an organization. Thinking carefully about the design and functionality will make a huge difference.

This post is the 2nd part in a 2 part series. Last time we talked about how organizations use Tribal Elders and Static Reports to find answers to questions that they already know. Today we'll talk about the other three phases of Data Analytics Maturation.

Data Analytics Maturation Phase 3: Bigger Static Reports

Answers to questions you don't know

Once the organization realizes that they need answers to questions that they don't yet know, they start to extract all sorts of permutations on all of the data that they have and distribute those reports to the "need to knows" on a regularly scheduled basis. In most cases, an analytics team is set up to manage the requests from the business for more or different information. Sometimes the reports are modified, but many times new reports are created because the users already know how to use the old reports. The analytics team works hard to maintain the information flow to the individual requests with the intent to provide all the information that would ever be needed by the consumers.

On the other hand: Page 73, Row 14, Column G

The down side is that this typically manifests itself in the form of the dreaded 124 page monthly report. So, the reporting "Oracle of Delphi" shows up in the inbox. For a little while there's some excitement along the lines of "I never knew we could get all this information." However, soon folks realize that interpreting the data for "questions you don't know" turns out to be pretty difficult and once they figure out where are the answers to the questions they know, they just look at those few rows of the report and leave the rest for analysis "later" (which probably means it ends up in the recycle bin... if we're lucky).

Data Analytics Maturation Phase 4: Ad-hoc reports

Answer your own questions

Phase 4 begins when a few folks who get the 124 page data dump realize "if I could just filter the data down a little I could much better understand the answers to this specific question". So the organization provides the ability for end users create ad-hoc reports. Now the user has the ability to construct their own custom reports to answer the specific and unique questions they have about their data.

On the other hand: Water, water everywhere...

Sadly enough, however, most people who need to know the answers get stuck in any of a few traps down in the weeds. The first trap is that they may be sure they know what questions to ask, but even in spite of their confidence, they're really asking the wrong ones. Secondly, most people in this situation are more business oriented and less technical (presumably the more technical ones have already figured out how to query the data directly). In all but a few cases, the tool that is provided requires too much technical expertise for most business people to overcome in order to be really productive. Thirdly, even if they can actually get to the data that really does help them to be more productive, they lack the analytical expertise to interpret the data and turn it into usable information. The end result of these three hurdles is that the users end up either in analysis paralysis, or just plain giving up.

Data Analytics Maturation Phase 5: Experienced Guide

Answers to questions you should know

To solve the barriers presented by having a lot of data available only to technical users, maturing organizations provide solutions targeted at specific business areas that make exploration accessible to those who can impact business performance (in other words, everyone involved in the workflow). These solutions are not about the technology or even the data, but rather about providing information that translates easily in to getting stuff done.

The results are provided in a fashion that makes access to the right information easy by guiding the user through a process to help them answer the known questions, discover new questions to ask, and explore answers to these questions. It's sort of like the guide you might hire on a photo safari. The experienced guide will make sure you find the animals that you came to see in the first place, but will also point out really interesting things along the way that you had never thought of. And you might even discover something amazing and exciting that you didn't even know existed. Good information tools are just like an experienced safari guide.

On the other hand: Few and far between

The sad part about "experienced guide" information tools is that there are so few that exist. The good news is that we see more and more information workers and decision makers "seeing the light" when it comes to understanding their need for these sorts of tools. And, we believe that as more and more organizations mature and experience the challenges of the first 4 Phases of Analytics Maturation that more and more will see the benefits of Phase 5, and implement solutions that help us all be more effective and efficient users of information.

Key takeaways for the 5 Data Analytics Maturation Phases:

Recently, while meeting with one of our clients, they mentioned their desire to provide their customer's business team with the ability to run ad-hoc reports. This notion spurred me to think about whether or not I thought this was a plan for success. Would having this additional analytics ability help the non-analyst be more effective at getting their job done?

Over the next few days, we'll be exploring the different stages of maturity that information workers go through as they try to become more effective and efficient at consuming and acting on information. By our reckoning, we figure there are 5 Phases in the maturation cycle:

Phase 1: Tribal Elders
Phase 2: Static Reports
Phase 3: Bigger Static Reports
Phase 4: Ad-hoc reports
Phase 5: Experienced Guide

As we go through the different stages, we'll discuss the breadth (how wide is coverage of all available information), depth (how deep is the understanding about covered information), reach (how easy is the access to the covered information), the typical user of the analytics method, and the signs that the organization is outgrowing each phase in the model. So, without further ado, let's get started.

Data Analytics Maturation Phase 1: Tribal Elders

Answers from the Experts

The earliest stage of analytics maturity is one in which the organization relies entirely on the expertise of one or two individuals who use their business savvy to provide analytics. These folks, we'll call them Tribal Elders, have been around the company for a long time and have "seen it all." Just like the those "elders" in the movies, they're wizened leaders who can mash all the data in their head and join it with their experiences to make good decisions. I guess you would say that technically, there are no formal analytics that are performed during this stage. However, everyday, the expert is using their training in the school of hard knocks to observe, analyze, act and advise on what they know to be the best for the business.

On the other hand: No rest for the weary

An organization outgrows this phase when the business becomes complex either through growth or through changing environment (such as variance in market conditions, or the expert leaving the business). All of a sudden, the leaders find themselves in a situation where they can't scale the decision making quickly enough to continue to drive the business. The huge asset of the expert's experience has turned into a liability that acts like an anchor on the organization's maneuverability.

Data Analytics Maturation Phase 2: Static Reports

Answers to questions you know

An organization has reached the second phase when they have realized that they have outgrown their ability to rely wholly on what they can get out of the Tribal Elders to run the company. So they start to write down all the questions they normally ask. They use this list to start to build reports that that can provide answers to those questions that they know. Once completed, the organization now has the ability to enable a broad audience to answer the questions that have been asked on a regular basis.

On the other hand: Surprise!

The limitation of this approach is that the Tribal Elders are still needed to answer the questions that fall out side of the standard "what I know to ask" category. The beginning of the end of this phase happens when an event that was unforeseen occurs that dramatically and negatively impacts performance. The logical question arises "why didn't we see this coming?" followed by the answer "we didn't have that data." The organization then begins the transition to Phase 3.

Key takeaways for analytics Phases 1 & 2:

Next time we'll discuss the remaining three phases of maturation.