Measures and Averages and Indexes, Oh My!
By Ken Hilburn
June 25, 2010
Find more about:
Ah... Summer in Atlanta. Sunshine, green trees, Peachtree Road Race... and pollution. I love living in Atlanta, I really do, but one thing that most non-Atlantans don't know is that we have a real problem with poor air quality. As someone who really enjoys biking, hiking, and running, I pay a lot of attention to the Georgia Department of Natural Resources Air Quality Index. So far this year we've had a handful of "orange" days, but no "red" or "purple" ones - which sounds anticlimactic, but is a real improvement over a few years ago (we haven't registered a "code purple" day since 2002).
Anyway, as I was checking the air quality forecast this week, it occurred to me that the green/yellow/orange/red/purple/maroon categorization is based on an index. This started some thinking about measures, averages and indexes.
If you've never thought much about indexes, they are calculated by dividing the measured value by a base or expected value and then (usually) multiplying by 100. The result is that the target value is "100".
The great thing about indexes is... they are super easy for casual users to interpret. This is the case because they remove the dependency on the user to understand and keep absolute values in their head. In the case of the Air Quality Index, it's based on the national air quality standard for the pollutant measured. Since air quality in Georgia is primarily composed of 7 measures, it can get pretty confusing if you want to know what's going on. To demonstrate, here's a table of the national air quality standards that the Georgia Department of Natural Resources monitors:
Get the point? As an environmental layperson, it's much easier for me to interpret current measurements if they are expressed with respect to the national index (100) as opposed to ppb, ug/m^3, etc. Most people don't have a clue (nor do they care) about what a mg/m^3 or a ppb is, but they do care about their respiratory health. So, providing a common, simplified measure makes complex data oh so much more accessible to the populous. This is the power of indexes over absolute measures and average values. In Atlanta the use of indexes has unified nearly every Atlantan's practical understanding of air quality.
What it means for you
So, when it comes to creating information applications and dashboards, if you're presenting complex values, consider using indexes to reduce the barriers to entry for non-domain-expert users. Here are a few tips to keep in mind:
- Just as with any new metric, it needs to follow certain guidelines for good metrics.
- Get buy-in with your user group so they don't feel like you're pushing yet another meaningless value down their throat.
- Don't fall into the temptation to swap indexes with historical average values. Averages represent historical measures; indexes represent performance compared to a group.
Oh yeah, lest I forget, there's always a tradeoff. As with most things, when you simplify, you lose some resolution. The draw back is that for those who want to delve deeper into the meaning of the measure, they now have to do some researching (just as I did) to understand how the actual metrics are measured and how the index is calculated. Take this into consideration in your information design and make the absolute measures readily available through alternate views, mouse overs, or similar.
Can familiarity trump usability?
By Zach Gemignani
June 1, 2010
Find more about:
dashboard
Grocery shopping at a new store is a drag, no matter how thoughtful the supermarket layout or how clear the signage or how wide the aisles. I have a mental model of my local supermarket that makes my trip efficient and helps me avoid that frustrating "double back" to search for the peanut butter.
This thought made me wonder about the importance of familiarity in dashboards. We spend a lot of time at Juice designing intuitive, simple-to-use dashboards. We want to create a logic and cohesiveness that ensures the right things are placed in the right proximity and order; sales leads should connect to prospects in the same way as the peanut butter and jelly is shelved near the bread.
If you are starting from scratch, this internal logic and consistency is paramount. But how about a dashboard that is already familiar to the target audience? Does it makes sense to redesign a dashboard for usability if it is already heavily used and understood?
For many dashboards, the purpose is simply to convey a few key nuggets of information. Without a series of interactions or tasks, the user's only need is to locate and absorb data. In these cases, the measure of success is whether the user can find what they are looking for quickly.
I can appreciate the value of familiarity over usability. When the new Microsoft Office "menu ribbon" was introduced, it was described as a convenience to new users because it displays the most relevant features for any given context. For power-users it broke the experience; all the effort I'd put into memorizing static menus was lost.
For all our concerns about poorly-designed dashboards, it may be familiarity that explains why it can makes sense to keep the status quo.
9 comments | Show all comments only the last 5 are shown
Clint said:
Zach, what's with the apologist post? A poorly constructed product is a poorly constructed product - no matter how many people use it.
brad said:
Take a look at any three or four good field guides to the birds, and you'll see that they're all organized the same way: evolutionarily, with the "oldest" birds like loons at the beginning and the "newest" birds like sparrows at the end. That may not be intuitive, but because that's the way Roger Tory Peterson organized his original Field Guide to the Birds, every bird guide that tried to break that organizational model met with howls of protest from birders who attempted to use it.
As an aside, one of the reasons Peterson's book was so popular was because he deliberately left out all but the most important details in his paintings of the birds; when the Audubon Society came out with their guide, which used photographs, birders found it less useful because the photographs contained too much information and you couldn't hone in on the key features that distinguish one species from another.
Clint said:
Brad,
Not being a birder I lack a relevant context. However, based only on your description, it sounds like Peterson hit the right usability factors for his field guide right out of the gate (paintings that highlighted distinguishing features instead of reproducing exacting and confusing detail). Rather than an argument for the status quo (regardless of quality) it reads like an argument for usability because readers revolted against less usable versions of the information.
In the specific case of Audubon that you highlight, more information doesn't necessarily equal better usability or even more useful. In fact, dashboards - more often than not - are about distilling a large amount of detailed information down to those features that can be used to most easily identify the current state of a business.
brad said:
@Clint: the bit about detail was an aside; the "status quo" argument is in my first paragraph. There are probably much more "usable" approaches to organizing a bird guide; for example: put all red birds together, all black birds together, all shorebirds together, etc., but he didn't do that. He used a taxonomic/evolutionary approach, which isn't really intuitive to anyone but an ornithologist. But because his guide became the standard, subsequent guides that deviated from his approach were less useable to experienced birders because they didn't know where to find anything.
I have the same problem with Mark Bittman's "How to Cook Everything" book. He completely revised and reorganized it a year or two ago, and he had the bright idea of putting all the "essential" recipes together at the beginning of each section. Makes sense on paper, but in practice it's frustrating because now some of the recipes for roasted chicken are in one place (at the beginning of the Poultry chapter) and others are in another place (deeper in the chapter in the section on roasted chicken). I find this separation a bit maddening as it takes more effort to review the related recipes and figure out which one I want to cook from. The old approach was actually more useable.
Jon Peltier said:
"Familiarity" is the rationale for using chart types that according to first principles are not effective. Charts like Marimekkos, cascade charts, stacked and clustered columns, and tornado charts are familiar to people in certain fields.
It might not make sense to completely change the existing reports, so keep some of the familiar while cleaning up the most glaring problems. You can revisit the argument if/when Phase II is discussed.
Joe Mako said:
I agree there are pitfalls to redesigning an in use dashboard, and I think the key is understanding the audience, and the questions they are using it to answer. If you had to redesign a dashboard that was previously good enough, but needed the colors changed to enable color blind users, many other users would have issues adjusting to the new colors, because they stopped using the legend to see what a color stands for, and changing the colors on them would disrupt their understanding of the data presented because they would need to relearn the colors.
I agree with Clint, changing something like a speedometer gauge to a bullet graph, is always a good choice, but a good enough dashboard can harder to redesign than a bad dashboard.
Mark said:
This reminds me of something I happened to read today at http://www.jnd.org/dn.mss/affordances_and.html: "Convention severely constrains creativity. ... On the whole, however, unless we follow the major conventions, we are doomed to fail. Those who violate conventions, even when they are convinced that their new method is superior, are doomed to fail. (You cannot successfully introduce a non-qwerty keyboard today, or reverse the window scroll bar convention, or suddenly require double-clicking on web links. For better or for worse, human culture changes slowly, if at all.)"
brad said:
@Mark: I actually think convention fosters creativity: constraints effectively force you to be creative in coming up with fresh approaches and solutions. Take a look through any compilation of the haiku masters Basho or Issa and you'll see how creative they could be within the tight confines of the rules of haiku -- not just the length of each line but also the requirement for a seasonal reference, etc.
When there are no rules or conventions it's too easy to be lured into creativity for creativity's sake, which too often leads to marvelous but useless results.
James said:
As to the point about the Ribbon in Office 2007, it wasn't just that the menus had been rearranged, wasting the learned effort we'd had before. It was actually harder for an experienced user to get things done: if I wanted to sort a list, apply formatting to it and then create a graph, that meant at least 3 more mouse clicks than under Office 2003. The Ribbon still seems more about putting more eye candy onto the screen than actually improving usability for anyone other than somebody who doesn't know what a spreadsheet is for.
Add a comment
Subscribe & Browse
- Visitors Guide Our most popular posts and tools
- Subscribe via RSS or Email
- Browse by Topic, Date or Author
Explore our world
Recent Posts
- Juice Takes Message to the Capital City
- Chart Makeovers, Fed IT Dashboard edition
- Dashboard Design - Flow
- Juice Boston Tea Party
- Tableau and Juice together with You
- Designing a Better 'Federal IT Dashboard'
- 30 Great Visualization Resources in 30 Days
- Come Have Breakfast with Juice on July 15th
- Measures and Averages and Indexes, Oh My!
- Can familiarity trump usability?
3 comments
John B. said:
And please don't sum or average your index, I see people doing this so often it's saddening. Aggregate and recalculate.
Dave Marcus said:
I think that two additional steps are important.
The first is to make sure that all deviations from the standard that are bad move in the same direction (greater or less than 100). For all the measures above, a value >100 is bad. But what if there was a measure called "clarity of air" with a standard of 1 mile visibility = 100. For that measure a value of 200 is very good. For ozone, a value of 200 is very bad. For most of us, that would be confusing.
The second step is scaling. Is a value of 200 for ozone as bad as a value of 200 for small particulate matter? Assume first that "bad" can be defined, perhaps in terms of millions of dollars of additional healthcare costs. The next step then is weight the deviation for each measure so that a deviation of X points in any measure means that the same amount of added healthcare spend is likely.
Ken Hilburn said:
@John and @Dave, great and insightful comments. Thanks for contributing. Certainly this topic would probably warrant effort in kind to a mathematics graduate thesis, but in lieu of that, thanks for raising the bar.
said:
Add a comment