Juice Analytics

Juice’s Jon Buffington took center stage at the National Capital Area Google Technology user’s group in the D.C. area recently and busted some fancy moves.  And, you should have seen him once his presentation started.

Taking the group through the process of designing and implementing an interactive data visualization using Google Web Toolkit (GWT), Jon also incorporated DOM, Canvas, SVG, ReST and Scala browser and server technologies to complete the information experience.

GWT is an open source development toolkit for building and optimizing complex browser-based applications, and is used by many products at Google, including Google Adwords and Orkut.

Armed with a tutorial, Jon compared browser graphical techniques and their respective technologies compatible with GWT. As exhibited in this little gem, Jon simplified the visualization down to a basic bar chart, making the similarities and differences between the technologies amazingly clear. (Yo, Jon.)

Download the presentation, and adopt some formidable moves of your own.

Jon Buffington Shares Insight on Building High Performance Data Visualizations

Jon leads our product development team here at Juice, and crafts ingenious software technology that transforms data into information experiences. You can check out more of his work, specifically, here.

We’ll let you guys know where we’ll be next. Or, if you really want to keep up, sign up for our RSS feed and/or follow us on Twitter. (Hint: We share lots of little tidbits on Twitter that we don’t share a-n-y-w-h-e-r-e else.)

Juice’s own Ken Hilburn brings it home at the Strata conference. He sits down with Mac Slocum of O’Reilly for a few data softballs. Here’s a second-by-second account.

[0:03] Q: What are the most common visualization mistakes that people are making?

[0:03] A: The bottom line is usability. Stay focused on your purpose, making decisions, taking action. Stay simple, if you have to explain you’re failing.

[0:15] Brett Favre endorses Wranglers, but Ken wears Data Wanglers [not shown].

[0:59] Dropping names, and twisting the knife on usability.

[1:25]? What better time to confront a little gap in your knowledge than when you’re being filmed? It’s Antoine de Saint Exupéry.

[1:48] Q: Do we need different tools to create simple visualizations?

[2:05] Plentiful shout outs to friends in the industry. Even Business Objects gets a friendly mention, is Ken getting soft?

[2:53] A difficult point to cover in a short time. We speak of data journalism and telling stories with data, but there are really no great tools that allow this in an everyday business environment. There is a lot of attention, not much progress.

[3:15] Q: What makes a great dashboard?

[3:25] A: Zach and Ken just delivered a 3 hour tutorial on the subject earlier in the week. Can Ken cover it in 30 seconds? Attention, context, and data drilling are keys but there’s precious little time to do more than mention big concepts.

[4:20] Q: Are dashboards too complicated?

[4:22] A major softball to close the session. Cheshire cat grin from Ken.

[4:54] “Getting your brain around it.” What Ken isn’t saying is that we know today’s businessfolk aren’t just looking at one dashboard, they have to look at many. They have to integrate info from lots of disparate systems into a picture of how their business is doing. If your info is harder, more complex, presents a bigger cognitive load, or is slower to load, then it’s going to be less valuable.

If you haven’t had the pleasure of getting to know Ken, stay tuned. We’ll be doing more Viva Visualization events in ’11 with more detailed prescriptions for great dashboards and there’s no better way to spend your morning than eating a nice warm bagel and listening to Ken.

The 12 Days of Christmas, as referenced in the song, starts in most traditions on Christmas Day and ends on January 6. Our version, the “12 Days of Data Visualization”, allows you to have all of your gifts in one day — bonus. Each “day” links to data visualization resources that we’ve shared with you over the years or find particularly enlightening, and are well worth repeating — the proverbial “gifts that keep on giving”.

On the first day of Christmas, my true love gave to me, a white paper to set my data free.

On the second day of Christmas, my true love gave to me, Two Hans
Rosling presentations.

On the third day of Christmas, my true love gave to me, Three tree-maps.

On the fourth day of Christmas, my true love gave to me, Four font rules.

On the fifth day of Christmas, my true love gave to me, Five filter features.

On the sixth day of Christmas, my true love gave to me, Six principles of context.

On the seventh day of Christmas, my true love gave to me, Seven data viz galleries for inspiration.

On the eighth day of Christmas, my true love gave to me, Eight better dashboards.

On the ninth day of Christmas, my true love gave to me, Nine online resources (and a great Stanford data viz video).

On the tenth day of Christmas, my true love gave to me, Ten metrics no-no’s.

On the eleventh day of Christmas, my true love gave to me, Eleven lines of code.

On the twelfth day of Christmas, my true love gave to me, “12-step” pie chart treatment.

From all of us to you and yours, have a merry Christmas, and a happy New Year! Wishing you many successful visualizations in 2011!

The Juice Analytics Team

Geeking out on NFL data shouldn’t be left only to the stats junkies at Football Outsiders. Our NFL stats “spike chart” is an easy way to see who’s leading the league in passing, rushing, receiving, tackles, team offense, and team defense. By showing key metrics side by side, you get the full picture of a player or team performance–not just the highlights. Click on the image to see the live visualization.

Choosing the right chart for data presentation isn’t easy — even if you do it for a living. For those with less practice, it may resembles the flash of confusion I experience when my wife asks “Which of these outfits looks best on me?”

“…uhhhhhhh, both?”

And like that answer, there isn’t any safety in sitting on the fence.

Wouldn’t it be nice if there was a formula for choosing the right chart? The fact that there isn’t suggests it is a mix of art and science. There are plenty of examples of people who have taken a crack at this problem:

• Andrew Abela created a diagram that categorizes chart types.
• In Stephen Few’s book Show Me the Numbers, Chapter 5 provides an overview of graph fundamentals. Bonus: I received the following Graph Selection Matrix (PDF) from Steve.
• In Stephen Kosslyn’s book Graph Design for the Eye and Mind, Chapter 2 is entitled “Choosing a Graph Format”
• Sanket Nadhani shared this short tutorial which tackles the basic choices.
• From NC State, a flow diagram  for chart selection
• An Oracle-financed white paper entitled: “Selecting the Best Graph Based on Data, Tasks, and User Roles” (PDF)
• BonaVista Systems has an Excel add-in for choosing the right chart.

(If you know of any others, put them in the comments and I’ll add to this list.)

While these are all great resources, I thought it could be instructive to walk through a sample chart selection process, starting simple then gradually adding more complex requirements. The focus of this post is on ’wireframing’ the correct presentation techniques; in a follow-up we’ll replicate these same charts noting best practices with refined aesthetics and layout.

I typically ask four questions in choosing how to present data:

1. What data is important to show? Specifically, which dimensions and metrics need to be shown at the same time.

2. What do I want to emphasize in the data? For example, do I want to compare different values, show relationships, or present changes over time? What story am I trying to tell?

3. What options do I have for displaying this data? Your Excel chart menu is a start, but don’t forget options such as tables, sparklines, small multiples, and advanced visualizations like treemaps. Many Eyes’ list of visualizations can spark additional ideas.

4. Which option is most effective at communicating the data? Which chart or visualization emphasizes what’s important in the most direct and readable way?

Imagine a sales organization where two metrics matter most: activity (as measured by call volume) and sales (as measured by dollars sold). The simplest place to start with this data is to present aggregate performance for those two measures. Even with this most basic situation, you have a few options:

Conclusion:Data doesn’t always need visualizing. The common and dreadful example of this mistake is when people use a speedometer-style gauge to show a single number (option 3). It is a lot of work, pixels, and distraction for no user value. In this example, we have just a single data point for each measure and no comparisons (e.g. to goals, to last year’s performance, the values against each other), so it’s best to keep things clean with option 1.

Next, let’s look at options for showing activity and sales data by product. In this case, the emphasis should be on the relative performance of each product.

Conclusion: Option 1 is the winner. We prefer a vertical layout of labels (bar chart) to a horizontal (i.e. column chart – not shown) because the labels are more readable and the horizontal layout can suggests a time element in the graph. As has been thoroughly documented, a pie chart doesn’t allow you to see differences in values as effectively as a bar chart.

What if we wanted to understand these two metrics by time?

Time needs to be displayed horizontally. We’ve seen ambitious examples from Trend.ly and Axiis that attempt to break this mold, but they more often confuse than enlighten.

Conclusion:I’ve backed away from using dual axis charts after experiencing too many situations where people are confused by which line goes with which axis, no matter how clearly labeled. Because the emphasis for the data needs to be the trend over time, I would recommend option 2 over option 3’s sparklines.

Now it gets interesting: What if we wanted to understand these two metrics by product and by time?

Conclusion: The best option for this case depends on the importance of clearly communicating the detailed trend for each product. In most cases, the “essence” of the trend is good enough, i.e. Is the trend up? Down? Erratic? Smooth? Under that assumption, option 3 provides a nice comparison of the relative product performance and trend.

• Labeling matters. How labels are laid out in a chart can be a big difference in readability. It is almost always better if the label text can be written horizontally and be closely tied to the value (rather than in a disconnected legend).
• Multiple areas of emphasis. There will be compromises when you need to emphasize two things simultaneously (trend, relative values). Pick which one matters most.
• Know your options. the more types of charts you know of and understand how to apply, the better set of options you’ll be able to come up with.

To fly is to be frustrated. If you’ve been traveling for long, you no doubt have your opinions about what airlines and airports are the biggest sources of suffering. Whether it is weather delays, getting stuck on the tarmac due to air traffic, maintenance problems, or missing a connection, it all feels outside of your control.

But a little knowledge can help. The Bureau of Transportations has maintained a giant database of air traffic information for decades of flights — point of origin, flight times, flight delays, type of delay, etc. It is 72 gigabytes of data…just the type of data that needs some visualization. JuiceKit™ to the rescue.

We’ve put together a pair of visualizations that can make this data accessible to your average non-data-monkey traveler:

• Treemap uses size to represent the number of flights by airline and by point of origin. The color is used to show delay time — we’ve got all sorts of delay metrics, each of which tells an interesting story.

• US map uses size to represent the number of flights and the color to display delay time. Filtering by airline yields additional details.

There are some interesting insights that pop-out when you build a visualization this data.

• The different airline strategies are quickly apparent in the treemap. Hub-and-spoke airlines (Delta, Continental) have one or two dominant boxes (origin location), surrounded by lots of small locations. A point-to-point airline like Southwest looks entirely different with lots of similarly sized boxes.

• Flipping between delay types uncovers some unexpected results. For example, you might expect weather delays to be heavily correlated by airport. The data shows something a little different: Comair appears to be abnormally impacted by weather delays — as if a dark cloud chases around their airplanes. While Comair might be overstating weather delay data to prevent paying for meal vouchers, a more reasonable Wikipedia investigation suggests that Comair flies smaller weather-susceptible Bombardier airplanes.

A few details about this demo for our technical audience:

For those of you following JuiceKit™ development, this is a demo of some of the newer features available in our open source Juicekit™ 1.2 distribution, and some of the features that will be coming to the 1.3 version. Treemap styling is now elegant, crisp, and allows for white borders, fixing a couple rendering bugs. There is a new tree-level depth feature that can make it easier to navigate treemaps with lots of layers. The airports map demonstrates a geographic layout built using GeoLayout JuiceKit™ and Flare components. A major improvement demonstrated by the airline-selector dropdown is the ability to keep nodes consistent between data reloads. This allows us to animate the nodes even though they are generated by our new LiveQuery component.

At the recent Turning Statistics into Knowledge conference (here’s a synopsis), I saw The New York Times’ Amanda Cox present on how their 25-person design team designs and builds infographics. In my opinion, The New York Times sets the bar for telling stories with data. Amanda, I later found out, is sometimes referred to as the Michael Phelps of Infographics — presumably for her tendency to win infographics awards, not for getting photographed with a bong.

Here’s a infographic from the presentation that I particularly liked:

This chart is a re-examination of the OECD Business Cycle Clock which:

has been designed to better visualize business cycles – fluctuations of economic activity around their long term potential level – and how some key economic indicators interact with the business cycle.

(Flowing Data also took a look at this chart and the other approaches to presenting the same data.)

Amanda’s version of this chart is great because it demonstrates what can be done with the under-used scatterplot chart. Scatterplots are effective at presenting the relative performance of a set of things (e.g. product portfolio). Typically they show a snapshot in time; Amanda has added a time dimension without visually overwhelming the user.

As we’ve done in the past, we wanted to try to recreate a New York Times-style graphic in Excel. Here’s how it came out:

We have a few tricks in here to make this Excel chart possible:

• The chart is a scatterplot with smoothed connector lines. A second highlight series displays just values based on the time selection at the top.
• The line chart at the bottom contains a bar chart that keys off of the time selector to help visually display the time range selected.
• All the labels on the chart are extra data series with data labels rather than adding text labels onto the chart. This approach makes it easier to place the points in the appropriate spot and not worry about problems on resizing.
• A simple macro on the “animate” button walks through the data.

You can download the Excel spreadsheet here

Analytics can be all about having the right tool for the job. When your data is text, traditional analysis tools (e.g. Excel, OLAP tools) are like peeling a mango with a chainsaw.

There are a number of visual exploration tools specifically designed for text data, including:

• Word clouds like Wordle (fun but superficial);
• Network diagrams like Visual Thesaurus (good for individual words, not text);
• Trend graphs like Baby Name Voyager or Google Trends;
• Granular presentations for interacting and exploring individual phrases, e.g. We Feel Fine and Twistori
• “Word trees” that let you navigate through lines of text to understand the most frequent words, relationships between words, and common phase and sentence structures.

It is quite difficult to find a Word Tree in the wild. The brilliant team at IBM’s Many Eyes were the first to make Word Tree’s generally available. The same ManyEyes team have also created an alternative approach for visual text exploration with a tool called Phrase Net.

Recently, we built a slightly different take on the Word Tree in Concentrate, our tool which allows users to explore huge search query lists to see how people use search keywords. For geeky entertainment, we created a special Concentrate demo account with the lyrics of songs from Rolling Stone’s 500 Greatest Songs of All Time. Click here to sign-in to the demo (Press submit and then choose WordTree at the top).

Here’s how our Word Tree works:

• The box at the center is your starting point. When you open a Word Tree, it will contain the most common word in the text data. You can edit this box to “re-center” the wordtree (name that tune):

• Stretched out on either side are words and phrases that are tied to that center word. The size of the words represents their relative frequency.

• Rolling over the words/phrases will highlight the connections to your center word and on the other side. You’ll also see a pop-up box with examples of the phrases containing selected words.

• You can open or close branches by clicking on a word. Words with hidden branches are highlighted in orange. We also have an ability colorize the words based on a metric in your text data.

While these more advanced visualizations are a start, I suspect there is a lot of room for other tools and techniques to visually explore text data. I’d be curious to hear about other tools you’ve seen along these lines.

Now that I’ve got treemaps on the brain, I keep noticing how many things could be better understood using this visualization technique. A few examples:

We thought it would be a nice demonstration to use data from the 1997 and 2002 US Economic Census (unfortunately 2007 isn’t out yet) to see what kind of stories bubble forth. The demonstration was built using a component from JuiceKit™, our recently open sourced Software Development Kit (SDK) for building Information Experience™ applications. The SDK can be used by web designers and developers to build graphically rich and interactive information displays. JuiceKit™ currently integrates with Adobe Flex to create components that are easy to implement and aesthetically pleasing.

Check out the treemap here.

Here are a few of the macro-trends that I found:

• The rise of CostCo, Amazon, and Home Depot: This time period saw strong growth in warehouse clubs and superstores, online retailers (“electronic shopping”), and home centers.
• From manufacturing to services economy: Most of the growth was in service sectors (financial services, healthcare, professional services) while manufacturing was shrinking.
• Productivity gains, even in adversity: For struggling sectors, the employee declines almost always outpaced the sales declines — squeezing more sales per employee.
• Demographic shifts: Homes and services for the elderly were among the strongest areas of growth in the category of “healthcare and social assistance.”

And there were lots of little insights as well:

• No wonder hospital TV shows are so popular: Hospitals are the largest single employer as a business-type.
• Starbucks and Krispy Kreme steal the unhealthy food dollar: Cookies and frozen yogurt retail saw a rapid decline while coffee and donut shops flourished.
• Goodbye stand-alone pump: Gas stations with convenience stores overtook the just-plain gas station.
• It can’t last, can it?: Mortgage broker payroll up 177%.

Once you understand how to read treemaps, they are great for exploring data like this: hierarchical with both quantity and quality-type measures. In a true testament to their power, my wife admitted this visualization was “kinda interesting.”

In the information visualization world, treemaps are on the rise…and justifiably so. Treemaps simultaneously show the big picture, comparisons of related items, and allow easy navigation to the details.

However, treemaps aren’t easy to get right. In contrast to basic charts where Stephen Few, Edward Tufte, and the Chart Chooser have laid down the law, treemaps roam the Wild West of interface design, obeying few rules, breaking many, and contributing to much infovis lawlessness.

Over the last year or so we’ve been building treemaps for our clients using our (recently open-sourced) Flex-based JuiceKit™ SDK. Over the course of these projects, we’ve thought a lot about the best way to make treemaps easy to understand and use. I won’t claim we have “cracked the code,” but we have gotten a feel for what works and what doesn’t. I want to share some examples of the good and the bad in treemap design, and hopefully gather some feedback so we can continue to evolve our thinking.

1. Choose the right measures for size and color

Each box in a treemap can show two measures:

• Size of the boxes should be a quantity measure. The measures should sum up along the hierarchical structure of the data. The sum of all the elements in one branch need to sum to the value of the branch as a whole. Therefore, you can’t use ratios or dates or any other measure you wouldn’t use in a pie chart.
• Color of the boxes is best suited to a measure of performance or change such as growth over time, average conversion rate, or customer satisfaction.

The King of Treemaps — Smart Money’s Map of the Market — offers a classic set of measures: size represents market cap; color represents change in market cap.

2. Space matters

Like a pie chart, size represents value in a treemap. In the following example from LabEscape, the category labels use space — almost as if you added slices to a pie chart for labeling. This approach distorts the values by arbitrarily using space, making it harder for the viewer to visually compare sizes.

3. Labels should add value

Labels are hard to get right in a treemap. If you aren’t careful, labels can clutter up the treemap without adding useful information. This Macrofocus treemap wasn’t careful. Notice how the majority of labels get reduced to just a few letters or simply an ellipses (“…”). It would be better to show nothing until the user rolls over a box.

4. Labels must stand-out against treemap colors

One of the unique challenges of a treemap is that the labels need to stand out against a multicolored background. The ILOG Elixir treemap chooses to put the labels in a white text box. Unfortunately these text boxes look clunky, obscure some of the data, and don’t always fit into the allotted space.

To neutralize the contrast of the label to the background and ensure legibility, we created a “glow” around the text.

5. Explanatory legends

The New York Times folks know what they are doing when it comes to visualizations and the explanations around them. Below is the legend for a treemap about automobile sales. The meaning of size and color aspects are articulated in a small space.

6. Color ranges fit the data

The nature of your color measure should determine whether you need a one-sided or two-sided color range. In situations where the color measure has both negative and positive values (e.g. period over period growth), we typically use a two-sided color range with a light grey at the middle. A one-sided color range is a better fit when the measure starts at zero. The Hive Group treemap below offers an example where a two-sided color range (red to green) doesn’t make as much sense. This treemap is using color to show geographic area rank from 1 (largest) to 195 (smallest).

7. Show correlation by highlighting

One of the nice advanced features treemaps can offer is highlighting items that meet a user-specified criteria. In the Many Eyes treemap below, a search features identifies that companies that include the selected search term. Not only does this aid the navigational capabilities of the treemap, it allow allows you to see color, size, and location correlations for the selected items.

8. Show changes with animation

When you want to show variations in the data (e.g. changing time periods, filtering, changing measures), we’ve found that animation effects can help emphasize the differences. In our stimulus plan treemap, flipping between “cost” and “votes” to size the boxes results in an animated reorganization of the boxes. The boxes that get bigger move to the upper left and those that shrink move down and to the right. The effect helps the user track where things are moving and get an understanding of the overall differences in the treemap.

9. Simple presentation of node detail

When a user selects a node in a treemap, they should see the available detail either in a tooltip window or in the sidebar. If the detail is substantial in size, it is best to push it into a sidebar as we did with our Stimulus Plan Explorer. Simpler data can show up in a tooltip box like the beautifully designed tooltip created by MIX Online (notice how it flips around to stay within the borders of the treemap).

ILOG Elixir’s demo recognizes the need to see detail, but the execution is flawed. Selecting a box in their treemap highlights rows in a table, but the rows are not consolidated so you are lucky to see only one or two rows of highlighted data. Users need to scroll through a massive table to be able to see the complete details.

10. Gradually reveal detail

Panopticon has a powerful treemap offering, but their demo treemap has some missteps in showing the detail. In particular, they choose to show as much detail as possible, but in a faint grey text. When you roll-over a box, this text becomes legible just as a redundant pop-up box appears. Detail is shown before the user has even expressed any interest in the box. Better to wait until the user rolls over or clicks on a box, then show the details. In the meantime, let the size and color do the talking.

These are just a few of the design lessons we’ve considered in our work. Treemaps offer an opportunity to make vast and complex data accessible — but they depend on thoughtful, user-friendly design.

How about you? What are some of the design features you have seen in treemaps that you think are particularly effective in making the communication of information stronger?