Don't Be Afraid to Supplement Analysis with Experimentation

Share it on Twitter  
Share it on Facebook  
Share it on Linked in  

When I interviewed Jeanne Harris and Robert Morison, co-authors along with Tom Davenport of "Analytics at Work: Smarter Decisions, Better Results," they emphasized the importance of employing data analysis to make better business decisions. And enterprise data analysis' star is rising. IBM, for one, appears to be staking a great deal on its future as a data analysis provider, acquiring a string of companies to beef up its expertise in this area.


But with all the focus on analysis, are companies missing something? Michael Schrage, a research fellow at the MIT Center for Digital Business and a visiting fellow at the Imperial College Business School's Innovation and Entrepreneurship Group in London, seems to think so. In an interview published in the MIT Sloan Management Review, he contends companies are shying away from experimentation at a time when they should be embracing it. He said:

The problem with people who come out of excellent MBA programs is that they're really, really, really smart people. They love to analyze data, they love to analyze information. And they would rather analyze data and information than do a simple experiment. But the same technologies that allow you to do multivariant regression analysis allow you to do simple, and even more complicated experiments on a factory floor, in a store, in a classroom, in the street, in a mall, in a car. It's really fantastic. But people have been trained not to do experiments in business school. They've been trained to study cases and perform calculations. We've been trained to ignore, oftentimes, the single most valuable thing these technologies can do for us in the workplace and in the marketplace.

Experimentation can help companies identify simple, yet compelling, uses of technologies that add business value. As an example, Schrage mentions Tesco, the UK supermarket chain. It counts the number of people coming into the store, and using algorithms and simple rules, it opens up additional cash registers when a certain number of customers are there. He said:

There are ways of making complex technologies more accessible -- cash registers, swipes, all these things that we know about. But again, it should come back to what you're essentially trying to do. The designed focus is, we want to make it easy for people to shop. Easy for them to come in, easy for them to stay, and easier for them to leave. Minimize frustration. Because even if we're selling the same stuff as Sainsbury's, if they have to wait five minutes in line there and only 30 seconds at Tesco, we minimize the frustration. Heck, they may even pay a slight premium because they don't waste their time. Nice, simple, easy example. It's a meaningless thing to discuss without the technology, but the technology isn't valuable unless you discuss what kind of impact you want to have on customers.

Google is one company that conducts frequent, public experiments through Google Labs, a website that serves as a "sandbox" for ideas developed by Google employees during their vaunted "20-percent time." Among the services to come out of Google Labs are Google Maps and iGoogle. On his The Next Big Thing blog, Don Dodge, a developer advocate at Google, described the company's culture as one that "seems to follow the Thomas Edison approach which paraphrased is 'I haven't failed, I've just found lots of approaches that don't work, and I am closer to the solution.'"