innovation

In Fogbugz 6.0, Future Results CAN be Predicted by Past Performance

Monte Carlo. It’s a car. It’s a song. It’s a casino. It’s a city in Monaco (near France, somewhere). It’s also a method of statistical simulation that is used to better understand the probabilistic distribution of known set of data. Great. So what?

I recently attended the Atlanta session of the FogBugz World Tour to hear from Joel Spolsky about the latest version of his bug tracking software, FogBugz 6.0. Joel Spolsky has been writing about software development, management, business, and the Internet at joelonsoftware.com since 2000. Three books have sprung from content on his site: User Interface Design for Programmers,Joel on Software, and Smart and Gets Things Done. They are good reads for software developers and business folks alike; smart, funny, and focused on the human face of software development.

Before starting his company, Joel was a Program Manager at Microsoft on the Excel team to provide programmability to Excel.

The new release of FogBugz has quite a few nice features to make bug tracking much easier, but I was most intrigued by the new capability called Evidence-Based Scheduling, or EBS for short. This new capability uses a Monte Carlo simulation approach to determine probabilities associated with the expected “ship dates" of the software release project.

The core premise behind this capability is that unlike in the financial realm, in the software world, future results can accurately be based on past performance. FogBugz remembers the original estimate and the actuals for each task for each developer, and for all the projects they’ve been assigned to. Here’s where it gets interesting. Based on this information, FogBugz runs a series of Monte Carlo scenarios where it randomly generates different plausible results for each member of the development team. It then assembles the results to create a distribution for probability of completion for each developer—and more importantly, for the entire project. The result of this analysis is a "Ship Date" probability curve that shows potential ship dates on the X-axis and probabilities of achieving those dates on the Y-axis. Ship Date depends not only on estimates for remaining components, but also on the probability that the individual developers will be able to individually meet their commitments. A steeper curve means the developer estimates are more confident and a flatter curve means estimates are less confident.

The classic software development process involves balancing three things: time, money, and features. Most projects of reasonable duration are not going to be able to effectively add staff mid-stream—at least not to accelerate delivery timeframes. If you assume the primary factor in determining "money" is tied to people, time and features are your only real variables. FogBugz 6.0 lets you experiment with changing these two factors. This is a useful tool. Consider a scenario where the business people come to the development lead because they need a project to be complete by a specific date. This tool gives the project lead enough information to understand the probability of making a specific date. Additionally, it lets you test out what will happen if you remove a particular feature. With this new information, the development lead has the visibility to provide back additional guidance to the project sponsor about the probability of making the requested date, as well as the effect that changing release date and scope have on the probability of on-time delivery.

So how does it know? The ship dates are based on each developer’s history of estimation accuracy supported by developer time sheets. Yes, that’s right. I said time sheets—the bane of all developers. But stick with me, it’s not as bad as you might think. Each developer (or responsible team lead) can turn on a timer that automatically tracks time spent on each of their cases.

Fogbugz plots a linear fit through the data points for each developer and then uses the calculated R2 value to determine how consistent the developer’s estimates have historically been. Based on this calculation, probability distributions for remaining tasks for each team member are determined, which leads the ship date probabilities. It’s then easy to see the long pole in the project (Milton Ritchie in this example). This doesn’t necessarily mean the most work, but only reflects the probability of on-time completion—and correspondingly, the most risk to the project.

Anyone who has followed Joel’s writings knows that he is keen on the idea of improving the process of software development. And, we all know that estimating an accurate time to completion, or “ship date," for any software project is a pain in the rear even after all these years of "practice." The unique approach that Fog Creek takes is not to predict the specific completion time, but rather to predict the probability of completing the project across a range of dates.

So, project estimating will likely never be a simple turn and churn process. But with this release of FogBugz, Fog Creek Software has shown great outside-the-box-thinking that could significantly improve how we deliver software projects.

The Google Analytics relaunch

Google Analytics has been rebuilt and the result redefines the frontiers of doing analytics on the web. Avinash Kaushik has the definitive early review.

Google Analytics v2

I had the privilege of attending the launch and playing with the early release. Here are a few things I noticed.

  • Speak my language: Google has put a lot of effort into replacing specialized terms with everyday ones. This makes the application usable by a broad base of people and is one way to fight GUI Jock-itis.
  • Speed kills: The interface is easily reconfigurable and fast. I’ve long argued that interface speed is a substitute for configuration options. I’m curious to play with the tool and get a better sense if this is true.
  • Flex rules: Much of the componentry for viewing data in Google Analytics is built in Adobe Flex. This is similar to Google Finance, and not at all like GMail or Google Reader, which use the GWT. We believe this has profound implications for analytical tools on the web and will dig into this in later posts.

    Solving the Pie

    Last week I challenged the you to reproduce this alternative to pie charts in Excel. I promised a screencast to show how it’s done.

    Square Pie

    Eighteen people answered the call with nearly three dozen different solutions. Click here to watch the screencast showing how to accomplish the two most popular solutions; filling cells with conditional formatting and pushing the column chart to extremes.

    If you want to look at the source,Clint Ivy produced an excellent version of the cell filling approach.

    square_pie_clint_ivy.xls

    Dermot Balson submitted an terrific version of the column chart approach.

    square_pie_dermot_balson.xls

    Thank you to everyone who submitted a solution.

    8 Ideas for Presentations with More Zing

    To paraphrase from Really Bad Metaphors:

    "Presentations can be as lame as a duck. Not the metaphorical lame duck,

    either, but a real duck that was actually lame, maybe from stepping

    on a land mine or something."

    Here are a few ideas to create spicier presentations:

    1. Get thematic. Chose a theme that drives home a general concept in your presentation, then sprinkle it throughout your presentation. For one client, we presented an analysis that ended with a movie. Throughout the presentation we offered subtle hints in anticipation of our grand finale. Avoid themes that could also be used for a high school prom (e.g. A Winter Wonderland, Magical Memories).

    2. Change of pace. Break up the action with picture-only slides (see Beyond Bullets, Presentation Zen) or multimedia (i.e. audio or video clip). A little fun in the middle of a bunch of dry slides can help wake up the audience.

    3. Use a human voice. Surprise your audience by departing from the traditional business-speak. Try writing slides that don’t over-qualify the message or attempt to be comprehensive in the descriptions. Lawyers need to cover all their bases with that kind of language; you need to convey a simple message that will stick.

      Instead of "IT organization has little understanding of business units’ long term strategic direction and impact on technology needs", say "IT and business aren’t on the same page."

      Instead of "Disparate and redundant technology solutions leads to increased operating costs", say "Technology tangle is taxing."

    4. Just the answer. Present your conclusion, then ask the audience if they they need to know more detail. If not, meeting over. I haven’t tried this, but surely a five minute presentation with a simple take-away will be as memorable as the typical presentation death-march.

    5. Something to remember. Provide a simple, memorable acronym or metaphor so they will at least walk away with your key point. For example: Good presentation = f(S,E,C) where S = coherent Story; E = credible Evidence; C = Creatively told.

    6. "And one more thing." Steve Jobs is well-known for delivering one final surprise at the end of his presentation. Is there one last blinding insight that you’d want your audience to walk away with?

    7. A new format. Consider stepping out of the traditional slide presentation format. Maybe you can convey your results with a science-fair type poster or with a web page or in a short book (check out self-publishing with Lulu). How about with charades or a Broadway tune? Too much?

    8. Give them a souvenir. Provide your audience with something to take away that summarizes your key messages. For example, you might hand out a laminated one-pager with your most important framework and results. A colorful summary that begs to be thumbtacked to an office wall is better than a 40-page black-and-white deck that begs to be thrown out.

    Bonus post: 4 Ideas from Microsoft for Presentations with More Zing (But Are More Distracting Than Useful)

    1. Animation. Animation can be useful (e.g. "building" the content of a complex slide) but is overrated for livening up a presentation. The first slide animation is novel; the second is overdone.

    2. Fancy template. Microsoft has been kind enough to provide a variety of dazzling slide templates—almost all of which distract from the content.

    3. Clipart. A while back, we railed against Screenbeans, the little ant-like people that visually depict activities or moods. Those little buggers continue to be a bad idea.

    4. Fonts. Especially Comic Sans. Won’t you join the Ban Comic Sans movement with me?

    Google Earth Enhancements: What the birdie brought

    A little birdie told me that the Juice Analytics census data heatmaps were used at Google’s Developer Day to show how Google Maps can now load Google Earth KML files. Very cool.

    Google Earth KML files now have two important user interface features that I’m excited to try out. First up is progressive display of data. This means a KML file can show high level summary info when when a user is high above the earth and seamlessly show more detail as the user zooms in. This was only possible through network links in the current version of Google Earth and this will feel a lot more polished to users. The other important UI feature is folders can now support radio buttons (where only one thing can be selected at a time). The big deal here is it allows a user to explore points organized into multiple dimensions where you can only view a single dimension at a time. For instance, you might want to view your customers grouped by sales volume, types of products purchased, or industry. Choose which of these groupings you want to see and the others will be hidden.

    Finally, viewing KML files in Google Maps is a potential home run. This increases the sophistication of what Google Maps can display and simplifies rollout of geographic information to an organization. Bravo, GE folks.

    Re-thinking constraints

    500 words. That’s all I’m giving myself to make my point. Here it is: constraints can be your friend. Limits on time, money, people, resources can channel your creative energy, drive innovation and focus.

    The seed: I’ve been listening to podcasts about entrepreneurship (Venture Voice, Entrepreneurial Thought Leaders), and hearing a recurring thread: company finds itself in a terrible pinch, money is running out, strategic options disappear, employees leave -- suddenly the start-up turns a corner to success. Why would this happen with such frequency?

    There’s a "Theory of Constraints", originated from the Operations novel (there aren’t many of those) "The Goal" by Eliyahu Goldratt. The concept goes:

    "In any complex system at any point in time, there is most often only one aspect of that system that is limiting its ability to achieve more of its goal. For that system to attain any significant improvement, that constraint must be identified and the whole system must be managed with it in mind."

    Which is to say: constraints limit performance. I’m not so sure, especially in service-based businesses. Absence of constraints can be the problem. Here’s why:

    • More begets more...confusion, chaos, complexity. You’ve probably seen the inefficiencies of big teams and the lack of focus of big-company strategies (Microsoft strategy presentation). Lost in the complexity is attention to detail, clarity of mission, an appreciation of the value of resources.
    • More options lead to analysis paralysis. Did you know there are six kinds of Snickers now?! I am often too dazzed by my candy bar options to choose. A constraint-less world offers too many options -- and leads to a fear of sub-optimizing. So we fall back on...
    • Status quo decisions. When a manager’s marketing budget goes up, the tendency is to just increase spend in proven channels -- rather than experimenting with something new. More options pushes us toward our affinity to avoid risk -- at the cost of innovation.

    Which isn’t to say I won’t take more money or more help any day. My concern is about managing the downsides of more:

    • More waste
    • Less creativity
    • Less attention to detail and quality
    • Less focus and clarity
    • Less pride in accomplishment

    Isn’t it worth seeking out constraints in some situations -- even if imposed artifically? A few ideas that we are going to try:

    • Create artificial deadlines with teeth. Something real and bad has to happen when a project extends beyond a deadline. What if a team had to write a document describing why a deadline was missed?
    • Limit design freedom with less space, fewer colors, fewer tabs and buttons. At Juice, we recently found that we had some fairly radical limitations on the space available to create a web interface. What started as an annoyance helped us take some great steps forward.
    • Cap team size. What if you limited every team to five or fewer people? Just imagine the efficiencies and focus -- and all the people you could legitimately exclude!
    • Try without money. What if you had no marketing budget for a new product? I bet most of the companies that succeed with viral marketing are those that need to. Big companies admire the power of using customers as a salesforce -- but advertising is so much more well understood.
    • Fewer words. How about limiting blog posts to 500 words; PowerPoint lists to five items; and proposals to three pages? As Mark Twain said: "I didn’t have time to write you a short letter, so I wrote you a long one."

    There is pain in fitting into constraints. And it isn’t always worth it. But there can be pay-offs in innovation, efficiency and focus. (Darn, 618 words. I’m off to my 118 minutes of "Dancing with the Stars".)

    Others’ thoughts on this subject: