Analytics Roundup
By Chris Gemignani
March 31, 2007
Find more about:
design
infovis
juicesite
visualization
"Business Intelligence isn't a technical problem, it's a social problem"
By Zach Gemignani
March 23, 2007
Find more about:
screencast
Yesterday I presented to an B-eye-network audience our perspective on why business intelligence is broken and what can be done to fix it. The full PDF-version (4mb) of the presentation can be downloaded.
A sampling of the fun:
"Chart-based encryption -- data goes in, no information comes out"
On the excessive emphasis on reporting over analysis...
"Technologists are looking to build an atomic-baloney slicer"..."Nobody ever got fired for adding more requirements"
"Data analysis isn't just for the data analysts anymore"
"Have you ever working with a reporting tool that outputted to PDF?"
Hopefully we stirred the pot a little with this presentation. A recording of the B-eye-network event should be available soon.
16 comments | Show all comments only the last 5 are shown
Robert Kosara said:
That was a great talk! I really liked the slides, the pictures were great and there were some very quotable statements. I think especially the points about reporting vs. analysis and data/analysis vs. presentation were good and are often overlooked. The remarks about BI being a social rather than a technical problem, and about "chart-based encryption" were also spot on: it's unbelievable what visual nonsense is being produced.
Your examples were also interesting, because they were unexpected. NameVoyager and WeFeelFine.org aren't exactly what I would consider business visualizations. But they clearly illustrate what good visualization should do (though they are certainly more on the presentation than the analysis side, which also makes them so appealing: they tell a story, or at least give you the building blocks for one).
What I missed was some kind of big final example at the end that would apply what you said to some real business cases. I am all for visualizing blogs and baby names, but how do these things apply to the typical BI or CRM application? And what does analysis vs. presentation mean in a practical example?
David A. Heiser said:
I can't understand business these days.
Microsoft expanded the graphic "blings" in Excel, because business asked for all these realy wierd pyramid columns in pastels. Pastels!! Business asking for pastel colors for presentations??? I thought presentations were to upset the audience, not put them to sleep.
What's the relevance of cows? Is it related to a new advertisong campaign? Analysis is how you can distort the facts to pursuade the audience.If you have to wake up the audience with spectacular displays, why not just bring a wide screen HD tv set and do a surroung presentation with cows mooing in the background (7.1 sound).
You could perhaps have a parade around the perimeter walls of the conference room, with a real brass band. Get everybody to join in. Signifies agrrement with the latest management program.
DAHeiser
Michel Guillet said:
The slides and discussion really resonated with me. For 10 years I I labored with the presentation of 1,000s of slides to execs about customers and statistics at The Coca-Cola Company, Thursday's discussion finally answered my question, "what am I doing wrong?" The problem was or is a social one. My perspective is changed.
Thank you for providing the enlightenment.
Guillet
Brian Timoney said:
Zach:
Good job.
I'm shamelessly thieving your approach with your PowerPoints and dropped $11 on iStockPhoto to get the appropriately resonating photos for an upcoming conference presentation.
The "chart-based encryption" joke was made funnier since you had no way of knowing if any of the webinar audience "got it." As an East Coaster transplanted west, I've had my share of lines during talks that were pure comedic gold be met earnest non-comprehension . (Another reason we're so well liked...)
Great stuff; I've already dropped the herding cats vs. herding cows distinction on a co-worker.
BT
Jeff O'Connor said:
Please come and give this presentation at my company. I and my coworkers have given similar ones for years now, but they don't seem to carry the same weight as the ones that better-paid outside contractors give.
As an outsider you'll have instant credibility with my senior management, which I and my coworkers lack because we're part of - you know - a cost-center.
To ensure that everything you say to my superiors is taken as seriously as possible, I will disavow any knowledge of having made this post...
Methods In Excel » Reporting and Analysis said:
[...] I was interested to read what Zack at Juice Analytics had to say on the matter. To paraphrase, reporting is for things we know well and are predictable where as analysis if for things that are unknown and erratic. Now I’ve only looked at the slides (check them out there good!) so I may have missed the context, but that’s not really the point. [...]
Fraser Moffatt said:
interesting presentation. I am especially drawn to the Mindset|Skillset|Toolset paradigm and would be interested in the audio recording, an expansion of these ideas or maybe some reference to follow-up on.
I've always thought that if you provide the tools, the mindset will follow. Sometimes is does and and sometime there's a lot of kicking and screaming.
I'm in an org where the tools are there (multimillion dollar BI/Datamart/Reporting "solution", but the mindset is non-existent and I'm wondering how to overcome this.
The concept of BI as a social problem is compelling and I would like to know more about your thoughts on this.
Zach said:
Thank you all for the feedback on the presentation. If you are interested in continuing the conversation, please send me an e-mail at zach.gemignani@juiceanalytics.com and we can set up a time to discuss in more depth. We always find it helpful to hear more stories about the challenges people face in the BI trenches.
Jay Jakosky said:
Nice presentation and your example were excellent.
Yaju Arya said:
Excellent stuff, awesome!!
Emily Breed said:
Zach, the cows-vs.-cats comparison is a great one. Would it be all right to borrow that idea? (I work in risk management, and we deal with a lot of people who'd prefer to pay attention to the cows only, even though we have cats overrunning the place...)
Zach said:
Emily, feel free...and use the pictures too.
Drew said:
Nice work! Thanks for sharing this.
Charles Lumpkin said:
Interesting. Would have liked to have seen it live.
hugo koopmans said:
Hi Zack,
this becomes more true as time passes...
good work! thank you
hugo
Anon said:
There's a grammatical error on Slide (page) 16. It's fragmented.
It says:
"Reporting is useful it
measures things that are
well-understood and
predictable"
Correction:
Reporting is useful IF it
measures things that are
well-understood and
predictable
Add a comment
Analytics Roundup
By Chris Gemignani
March 22, 2007
Find more about:
gis
juicesite
mapping
An $80 Million Super-Mugging
By Zach Gemignani
March 14, 2007
Find more about:
bi
Ah, the sweet smell of a swindle. Don't you just hate it when consulting companies cajole deals with hand-wringing about technology and, especially, preying on clients' lack of expertise?
I've seen some of these situations up close but nothing so ugly as this story.
$80 million supercomputer to analyze NYC student achievement
March 6, 2007, 7:58 AM EST NEW YORK (AP) — To understand student performance, the city will spend $80 million on a massive supercomputer that will crunch huge amounts of data and offer up-to-the-minute reports to teachers, principals and eventually parents, the Daily News reported Tuesday.
One million students and no high-volume transactional data? That might be huge to Dr. Evil but even by late 90's standards that's not huge. You want to talk huge? Now these are huge. The system that was sold to New York is more along the lines of a CRM system for a medium-sized insurance company.
The "super" reference here is pure drive-through mentality. In the same way that we are a nation that's overfed and undernourished, this is about a super-sized services contract that sits atop something that could be handled by a regular-sized computer.
The information fed into the IBM-designed system called Aris, or "Achievement Reporting and Innovation System" could include existing data on students—such as gender, race and any disabilities—along with new data from incremental testing.
Some aren't so pleased with the system's price tag.
"You can lower a lot of class sizes with that money—or buy a lot of supplies," teachers union President Randi Weingarten said in a statement obtained by the Daily News.
Mayor Michael Bloomberg told the tabloid the cost was worth it.
"Every child in this city deserves a quality education and we will spare no expense," he said.
This is where the sweet smell of swindle comes in. There is a difference between being willing to make the investment and having a no-bid contract.
Jim Liebman, the Education Department's chief accountability officer, also lauded the system.
"Aris will bring together every bit of learning information that we have on every one of our 1.1 million students," Liebman said. "Now, school professionals will be able to slice and dice that data to see what's wrong."
Teachers are underpaid, hardly appreciated, and overworked. I can only wonder what the half-life is of a system that asks teachers to log on to get information delivered by the "chief accountability officer."
And from an article in InformationWeek, we're enthralled by a description of the system capabilities:
"Think of a teacher trying to help a student struggling with geometry," says Michael Littlejohn, VP of public sector for IBM global services. "The teacher could tap into the system and search for best practices on geometry instruction, and get contact information for teachers identified as having strong skills in that area."
Sometimes it's good to reinvent the wheel - usually when you're trying to learn about wheels. But not when you're drawing away cash from an entity that doesn't have it to spare. Something like this could be built with off-the-shelf, mature products for a fraction of this wasted time and effort.
Sure, a fully-integrated, one-stop solution is going to run up the price but the functionality doesn't sound particularly whiz-bang. Best practices for teaching geometry can be found at Curriki or Edutopia or Wikiversity or Openplanner.
The real shame is not allowing such a system to connect more than just the overworked NYC school system teachers. But what would we call such a thing? An inter-net, perhaps?
Nah, that would never catch on.
Related articles
18 comments | Show all comments only the last 5 are shown
Tony said:
Great article and I love the subtle humor along with the reference to Dr. Evil! The last paragraph made me lol.
It amazes me the price tag on this "super" computer. I mean, this could probably be done with MS Access...
On Deal or No Deal the other night, there was a teacher who made the comment that $40k was way more than what she made in a year. $80 million could be spent on underpaid teachers who we rely on to teach our kids who are the next generation's leaders!
Ian said:
Great Post!
This reminds me of an article I read a few years back on how the University I attended was spending millions on an anti-spam email system when most of the best products out there for this are free.
At a time when you can buy a dual processor (with dual cores) machine with 4+ Gigs of RAM and over a terabyte of disk for less than $10K, I wonder what the other $79,990,000 is being spent on. Licensing and Installation?
Brian Timoney said:
A couple of observations:
1) For that kind of money I would've demanded a much hipper acronym--Aris? At least military contractors tend to give very cool names to theWhite Elephant projects they fob off to the gov't.
2) And as usual, no one thinks through the anthropology. So I'm the geometry teacher who has my sh*t together and a solid track record that's now in the 'system'. How will I react when every other teacher in NYC makes their students struggling with geometry my problem????
BT
Gary Babad said:
One more related article of interest : "ARIS Critics: IBM Responds"
http://nycpublicschoolparents.blogspot.com/2007/03/aris-critics-ibm-responds-march-7-2007.html
Robin said:
hmm 80.000.000 dollar for a database with one million students.
"Will crunch huge amounts of data and offer up-to-the-minute reports"
What, do those boys and girls get grades every minute or so?
With one million students there must be a student with a talent for MS access that can build it for 100 bucks. Shoot, you can almost vlookup it together in excel 2007! (not that you would want that of course).
Someone over there must really like IBM.
Rob said:
There seems to be a trend of education companies/districts being overcharged for data ware house and analytics. I wonder if NYC is being charged by the record.
Sorry, if you only had a million records that would have been $40 million. However, those extra 100k records are going to double the cost. You know... servers and processing costs are expensive!
bee said:
This measurement and reporting system addresses a non issue in education.
The fundamental problem with education is that it fails to adopt what we have learned about quality control. Namely, you cannot inspect quality into a product. The emphasis on testing fails the to address that education is not simply a measured outcome, it is a process. I would suggest spending the money to address the questions of what the inputs (students) look like and a real focus on teaching (the process) and less on testing (measurement and reporting).
Toyota does not inspect quality into each car. Rather, they focus on ensuring the parts are at spec and that the process is rigorously followed (and continuously improved). In education we have hundreds of curriculum that are not really validated to produce the specified outcome. At best they are benchmarked against some control which itself is not linked to the targeted outcomes. We further aggregate the process by not taking care to know who we are really teaching. No, in the name of inclusiveness, we often seek a wide range of skills among students to make process management impossible. When will we learn?
HS said:
Very interesting blog. I like bee's comments--sometimes we get too much savvy about technology instead of focusing on the core of the problem. The Toyota example is apt--we need to build quality into the process (of education), and not spend too many $$ on measuring (inspecting) the product (student).
Bill Gates, in front of the committee of education, labor, etc. emphasized upon merit-based pay for teachers--we need to spend money there. If there was more money in teaching a lot of smart people would enroll in the teaching profession (I would!).
bb said:
Where did the idea that this was a no-bid contract come from? It was a multi-month open bidding process in which much of the education industry and enterprise technology industry competed. McKinsey (hardly a naive client and with their bill paid by a foundaiton) supported NYC in evaluating the bids. A supercomputer is not involved in any way.
Pete said:
I don't want to say anything I would regret on the Juiceblog, so I'll just say that its probably worse than you think, take a look at these powerpoint/pdfs I downloaded from http://schools.nyc.gov :
http://www.datawrangling.com/media/AchievementReportingandInnovationSystemARISPanelPP.ppt
http://www.datawrangling.com/media/001_092706OnlineDeck1.pdf
"ARIS COMBINES OLD SYSTEMS AND NEW CAPABILITIES"
"Stoplight" charts...oh boy.
Pete said:
Look at slide 18, "Overview of Required Functionality". They don't even ask for built-in data visualization (they will export to Excel), this is just an $80M content-management system. Everything they list as a requirement is 5 year old technology, which you could do with Drupal, Plone, or SugarCRM today out of the box. I get most of this with MoinMoin right now, or you could use Django if you want to make something really slick.
*Library of templates/idea bank for school-created tests
*Test creation and upload
*Assessment content upload into online testing, reporting and analysis
*Online testing and scoring
*Score data upload into reporting and analysis
*Canned and customized reports
*Drill-down, roll-up, filtering
*Ad-hoc queries
*User-generated databases
*Adding new data to central repository
*Data viewing in 3rd party tools (e.g., Excel)
Advanced analysis
(for selected users)
*Advanced statistics
*Pattern detection for target setting
*Library of templates for different document types (e.g., lesson plans, curriculums, )
*Document creation and upload to repository
*Document tagging
*Collaborative document creation
*Publishing approval
*Document linking to reporting (surveys)
*Searching documents in the repositories
*Document viewing and download
*Pattern analysis across text documents
*Posting announcements; setting alerts
*Online discussions; online messaging
*Custom workspaces, integration w/email
*Pre-defined home page based on user role
*Personalization of home page (HP) by user
*Adding new tools to HP from a tool catalog
Henk said:
Could it be the $80 Million should read $80 Thousand? Maybe it was reported as 80,000 and the reporter thought the figures were in x'000?
Pete said:
The Information Week article has the $80M number as well (keep in mind that amount is over 5 years):
"New York City's public school system -- the largest in the nation, and one wrought with problems common to intercity school districts -- will pay IBM $80 million over the next five years to develop a business intelligence system that tracks and analyzes student and school performance."
I hope IBM takes the scrutiny into account and does a good job on this, even with the hokey requirements...as a company they are relatively open-source friendly and produce some good R&D work.
The question I have is whether the people who work on innovative projects are involved, or just the bread & butter enterprise consulting division....I suspect this was originally planned as a "turn the crank" style project using existing IBM software. Still, no matter how you cut it 80 million is a lot given the requirements.
Here are some platforms they might be using (Sakai, rSmart, IBM Content Producer):
http://www-03.ibm.com/industries/education/doc/content/landingdtw/1601142110.html?g_type=pspot
Some related links:
http://www-03.ibm.com/industries/education/doc/content/landingdtw/1584488110.html?P_Campaign=6N2CAS47
http://www-03.ibm.com/industries/education/doc/content/news/pressrelease/1832371110.html
http://www-03.ibm.com/industries/education/doc/content/news/pressrelease/1748306110.html
john macinnes said:
I am a teacher in England and find it hard that there are fools still in high places. The demands for education over here are just as high as over there. We tried the QCA online test, a test that was to stay with the pupil and adapt to their learning style, for example, it would make questions easier for the less able. As the months rolled by the system failed to emerge until the multi billion pound project is now to be scrapped. Could it be suggested or rumoured that there is a golden hand shake or a deal in the back ground. Over here all of the ICT is talk, and talk is cheap.
What really winds me up is the government officials who tell us these stories and don't want/ expect to hear a comment like, 'well what about the last 2 times you have tried and failed. Shouldn't you be learning from mistakes instead of spending money, ' (on a company which a relative of his probably owns.)
I grumble but will it ever change?
john macinnes said:
I am a teacher in England and find it hard that there are fools still in high places. The demands for education over here are just as high as over there. We tried the QCA online test, a test that was to stay with the pupil and adapt to their learning style, for example, it would make questions easier for the less able. As the months rolled by the system failed to emerge until the multi billion pound project is now to be scrapped. Could it be suggested or rumoured that there is a golden hand shake or a deal in the back ground. Over here all of the ICT is talk, and talk is cheap.
What really winds me up is the government officials who tell us these stories and don't want/ expect to hear a comment like, 'well what about the last 2 times you have tried and failed. Shouldn't you be learning from mistakes instead of spending money, ' (on a company which a relative of his probably owns.)
I grumble but will it ever change?
Mary said:
Zach, Your thoughts on this subject make me think you know something about education. Hmmm... After 30 years in the field of education, I could give you another dozen examples of how educators (or, anyway, administrators) have been sold a bill of goods by big business. I agree with bee. There is no need for this. Good teachers already have this information about their students or know where to get it. The list of things that really would be useful for students and teachers for that amount of money is endless. Oh, well, just another example of a way to leave children behind.
David Andersen said:
<em>"Think of a teacher trying to help a student struggling with geometry," says Michael Littlejohn, VP of public sector for IBM global services. "The teacher could tap into the system and search for best practices on geometry instruction, and get contact information for teachers identified as having strong skills in that area."</em>
Incredible b*llsh*t. This will never be implemented in any useful or novel manner, if at all.
greedcontrol said:
We need Greed Control in NYC not Mayoral Control. This mayoral thing has turned out to be the best business (kick back or otherwise) that he has ever done. Children are being marginalized (to put it kindly) by this mayor while he robs the city $60 billion budget. Oops there goes another billion!
Add a comment
Esurance--Competing on Analytics
By Zach Gemignani
March 6, 2007
Find more about:
analytics
bi
casestudy
management
metrics
reporting
Recently I caught up with my college friend John Swigart who now runs the marketing organization at Esurance. When the conversation inevitably drifted to business, I asked about how Esurance was using data to make decisions. I was expected to hear the same old story—big failed data warehouse projects, piles of underutilized reports, frustration about not being able to understand how the business was performing. I was way off.
It seems that John works for the rare company that has managed to live the analytics dream. Esurance competes on analytics—not in the idealistic model highlighted by Tom Davenport, whose "full-bore" analytics competitors are defined by:
"Top management had announced that analytics was key to their strategies; they had multiple initiatives under way involving complex data and statistical analysis, and they managed analytical activity at the enterprise (not departmental) level...
...Employees hired for their expertise with numbers or trained to recognize their importance are armed with the best evidence and the best quantitative tools. As a result, they make the best decisions: big and small, every day, over and over and over."
That's window-dressing. John didn't make any grandiose pronouncements of Esurance's analytical achievement or talk of the best tools and most complicated models. He simply stated that data-based decision-making has been a part of the culture from the very beginning and he considers it essential to running a smart business. A few points that he emphasized:
- Clear linkages between metrics. There needs to be a well-understood hierarchy that has important financial measures at the top (i.e. revenue) and connects to the underlying drivers.
- Frequent reviews of reporting. Senior managers get together on a regular basis to look through the core reporting. These meetings are detailed, but somehow useful enough that people stay committed to the process.
- Learning takes time. John recognized that Esurance cound not be as evolved in their understanding of the business without a commitment to this approach from the very beginning.
After getting off the phone with John, I asked him to respond to a few questions so our readers could get a taste of their approach:
How has Esurance managed to develop a culture that embraces decisions using data?
We don't make decisions based "I think we should this." We look at data to find out what we know, then decide what to do based on the facts. We identify expected outcomes up front and determine how we are going to measure the change before we implement something. Also, a data-driven culture starts at the top of our organization.
What processes do you have in place to get the right data in front of the right people?
We have centralized data warehouse and reporting structure. Everyone gets their data from the same place and the metrics are universal. This took 3-4 years to get it right, and we built it from scratch. It takes a substantial commitment to pull off.
What is the role of the analyst in your organization? What tools do they use?
We have technical analysts and DBAs in our business intelligence group that deal with the more technical issues. In Marketing, then, we have analysts how are on the individual marketing teams that work closely with the business people. The use some basic tools, nothing terribly fancy.
From an analysis perspective, what do you do when you are testing new marketing opportunities?
All tests are done with as much of a controlled environment as possible. With so many moving parts, this can be difficult, but is important.
How has analytics contributed to the success of Esurance?
Truly one of our competitive advantages. We would not be where we are today without great data and a dedication to using it through all levels of the organization.
2 comments
James Taylor said:
I'm curious - how does this kind of analytics mesh with Esurance's risk modeling and other forms of predictive analytics? Same group, same process or something different?
Zach said:
I didn't get into that area specifically, but John did say that Esurance builds almost everything in-house -- including their semi-controversial ad campaign (Slate pans it here http://www.slate.com/id/2153173/ , but I personally find it memorable).
Add a comment
A Juice Web Event: Empowering the Analyst
By Zach Gemignani
March 6, 2007
Find more about:
analytics
presentations
screencast
Our friends at Tableau invited us to lead off a webinar about the broken bits of Business Intelligence and what is needed to fix it. With the provocative title "The Score: IT-centric BI — 5, Information Worker — 0", we intend to hit blog-themes such as the plight of the noble but beaten-down analyst, the misplaced emphasis on bulky technology solutions, and the false deification of the Executive Dashboard.We'd love to have you stop by on March 22 at 2:00 ET. Go here to register.
The session abstract is below:
Empowering the "Everyday Data" Analyst
Like it or not, we've all become "everyday data" analysts during the last decade. We became document specialists and spreadsheet experts ten years before that. We have standard tools for creating documents, spreadsheets, and presentations right on our desktops. These applications are familiar and easy to use – even if we only use them infrequently. Why don't we have the same for working with data?
Everyone agrees that we have plenty of data—it streams through our departments and across our desktops everyday. But despite the big, IT-centric BI solutions that exist in our organizations, it's the tools and skills for investigating and making sense of "everyday data" that we're missing. The people who have the most to gain from data analysis are often the least capable of doing so. Where's the BI equivalent of Word or Visio?
Join Zach Gemignani, co-founder of Juice Analytics for this free web seminar. Based on his years of experience with analytics client engagements, you will hear him present the real-world struggle of "everyday data" analysts. You will learn:
- How the IT-centric view of BI should change
- How do we empower our "everyday data" analysts in our organizations
- What shifts in approach and technology are necessary for effectively working with data
1 comment
Wade said:
Missed (unfortunately) the webinar. Will you be making any materials from the event available on your site?
Cheers,
Wade
Add a comment
Subscribe & Browse
- Visitors Guide Our most popular posts and tools
- Subscribe via RSS or Email
- Browse by Topic, Date or Author
Explore our world
Recent Posts
- Juice Takes Message to the Capital City
- Chart Makeovers, Fed IT Dashboard edition
- Dashboard Design - Flow
- Juice Boston Tea Party
- Tableau and Juice together with You
- Designing a Better 'Federal IT Dashboard'
- 30 Great Visualization Resources in 30 Days
- Come Have Breakfast with Juice on July 15th
- Measures and Averages and Indexes, Oh My!
- Can familiarity trump usability?
0 comments | Add a comment
said: