Juice Analytics

There are two kinds of people in this world: those who put things into two categories and those who don’t. Maybe this isn’t the best representation of the complexities of the human race, but it does give me a cheap lead-in to compare two types of problem solutions: “high tech,” focused on tools, and “high touch,” focused on interpersonal communications.

I was reminded of these two approaches by a recent interesting article in Wired that expresses an opinion about why America’s performance in Iraq has been disappointing. The basic premise of this article is that America has entered into this engagement in a “technology networked” fashion, drowning it in technology; the more, the better.

The article suggests that the US forces would make more progress if they were to spend more time on a “socially networked” approach. For instance, instead of remote controlling a drone from 100 miles away, spend more time drinking chai with local leaders. Not the absence of technology, but the incorporation of technology into a socially based environment.

“If I know where the enemy is, I can kill it. My problem is I can’t connect with the local population.” This was a quote from one division commander. Change a couple of words and you end up with a statement that many of us would find all too familiar:

“If I know where the inefficiency is, I can fix it. My problem is I can’t connect with my data.”

Aren’t we witnessing this in spades right now in the BI space? There’s no lack of number of tools and number of features in these tools. The challenge is figuring out who the real insurgents are and how you deal with them. If you’ve been reading the Juice blog for very long, you have a pretty good feeling for how we approach what we believe is a social problem (high touch) and not a technical one (high tech).

The good news is that the US forces are changing their approach to socialize more with the Iraqi people—hopefully leading to a better Iraq. Is there good news for the BI space? We’d like to hear from you on how you’re making sure you focus enough on the social “high touch” aspects of our space. What’s your insurgent data? How can you get to know it better?

Mapping Philly

This geomapping project is digitizing and mapping historical images of Philadelphia, including meta data..

ICCARUS: Three Dimensional Data Visualization

3D is fun, but would you really be able to extract insights from this tool?

Convert your portrait to a character from “The Simpson’s”

Ever wonder what you would like like if you got one of those sweet cameo appearances on “The Simpson’s?” Now’s your chance to find out. Go to this site and follow the directions to upload your photo—and poof, you’re in Springfield.

Monte Carlo. It’s a car. It’s a song. It’s a casino. It’s a city in Monaco (near France, somewhere). It’s also a method of statistical simulation that is used to better understand the probabilistic distribution of known set of data. Great. So what?

I recently attended the Atlanta session of the FogBugz World Tour to hear from Joel Spolsky about the latest version of his bug tracking software, FogBugz 6.0. Joel Spolsky has been writing about software development, management, business, and the Internet at joelonsoftware.com since 2000. Three books have sprung from content on his site: User Interface Design for Programmers, Joel on Software, and Smart and Gets Things Done. They are good reads for software developers and business folks alike; smart, funny, and focused on the human face of software development.

Before starting his company, Joel was a Program Manager at Microsoft on the Excel team to provide programmability to Excel.

The new release of FogBugz has quite a few nice features to make bug tracking much easier, but I was most intrigued by the new capability called Evidence-Based Scheduling, or EBS for short. This new capability uses a Monte Carlo simulation approach to determine probabilities associated with the expected “ship dates” of the software release project.

The core premise behind this capability is that unlike in the financial realm, in the software world, future results can accurately be based on past performance. FogBugz remembers the original estimate and the actuals for each task for each developer, and for all the projects they’ve been assigned to. Here’s where it gets interesting. Based on this information, FogBugz runs a series of Monte Carlo scenarios where it randomly generates different plausible results for each member of the development team. It then assembles the results to create a distribution for probability of completion for each developer—and more importantly, for the entire project. The result of this analysis is a “Ship Date” probability curve that shows potential ship dates on the X-axis and probabilities of achieving those dates on the Y-axis. Ship Date depends not only on estimates for remaining components, but also on the probability that the individual developers will be able to individually meet their commitments. A steeper curve means the developer estimates are more confident and a flatter curve means estimates are less confident.

The classic software development process involves balancing three things: time, money, and features. Most projects of reasonable duration are not going to be able to effectively add staff mid-stream—at least not to accelerate delivery timeframes. If you assume the primary factor in determining “money” is tied to people, time and features are your only real variables. FogBugz 6.0 lets you experiment with changing these two factors. This is a useful tool. Consider a scenario where the business people come to the development lead because they need a project to be complete by a specific date. This tool gives the project lead enough information to understand the probability of making a specific date. Additionally, it lets you test out what will happen if you remove a particular feature. With this new information, the development lead has the visibility to provide back additional guidance to the project sponsor about the probability of making the requested date, as well as the effect that changing release date and scope have on the probability of on-time delivery.

So how does it know? The ship dates are based on each developer’s history of estimation accuracy supported by developer time sheets. Yes, that’s right. I said time sheets—the bane of all developers. But stick with me, it’s not as bad as you might think. Each developer (or responsible team lead) can turn on a timer that automatically tracks time spent on each of their cases.

Fogbugz plots a linear fit through the data points for each developer and then uses the calculated R² value to determine how consistent the developer’s estimates have historically been. Based on this calculation, probability distributions for remaining tasks for each team member are determined, which leads the ship date probabilities. It’s then easy to see the long pole in the project (Milton Ritchie in this example). This doesn’t necessarily mean the most work, but only reflects the probability of on-time completion—and correspondingly, the most risk to the project.

Anyone who has followed Joel’s writings knows that he is keen on the idea of improving the process of software development. And, we all know that estimating an accurate time to completion, or “ship date,” for any software project is a pain in the rear even after all these years of “practice.” The unique approach that Fog Creek takes is not to predict the specific completion time, but rather to predict the probability of completing the project across a range of dates.

So, project estimating will likely never be a simple turn and churn process. But with this release of FogBugz, Fog Creek Software has shown great outside-the-box-thinking that could significantly improve how we deliver software projects.

Congressional Earmarks in Google Earth

Brad Forrest from the O’Reilly Radar blog points out a new way to better understand where your tax money is actually being spent geographically.

Nielsen/NetRatings’ August social media numbers: Not much change

Interesting post I stumbled on related to Nielsen’s web analytics service. Several references to “juicy” or “juiciness”.

Inbox Zero

Merlin Mann on cleaning your e-mail inbox.

The New York Times > Home Prices Across the Nation

The most interesting / important part may be the talking head in the lower left, should you be annotating your reports with video?

Introduction to Statistical Thought—free ebook

1) explains how statisticians think about data

2) introduces modern statistical computing

3) as lots of real examples

Programming Collective Intelligence

Pulling information from community contributed data.

Videos that can change your organization

Top ten business videos on YouTube.

The Encyclopedia of Business Cliches

UC Berkeley CS160 User Interfaces Fall 06

Course readings and student notes.

Language Log: Chicken: the PowerPoint Presentation

The presentation you dare not give.

Prometheus Meets the Enterprise Management System

I laughed, I cried, I laughed again.

Diagrams: Tools and Tutorials

Data Visualization: Modern Approaches

A grab bag of ideas.

fontblog : Introducing Ambiguity

A typographic symbol to indicate ambiguity, compare to the typographic mark lol which indicates stupidity.

Whimsley: The Netflix Prize: 300 Days Later

Process Trends Website

Good excel charting and visualization tips.

BusinessWeek: Who Participates And What People Are Doing Online

A simple and fairly effective use of square pies.