Advanced Analytics, Collaborative Approach Real Winners in Netflix Contest

Share it on Twitter  
Share it on Facebook  
Share it on Linked in  

There are several winners in Netflix's million-dollar competition to find an algorithm that will improve the accuracy of its recommendation system and, in theory, help it retain more customers. Obviously there's BellKors Pragmatic Chaos, the team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel that squeaked out a narrow victory over No. 2 finisher the Ensemble to earn the million bucks. And there's Netflix, which got access to some great minds for a fraction of what it normally spends on R&D and plenty of positive PR to boot.


The sophisticated data modeling and data analysis techniques that are increasingly in demand across a wide range of industries were also winners. Before Google, few people had ever heard of algorithms. A lot of people still have only the vaguest idea of what algorithms do, yet they expect the kind of personalized treatment from companies that is possible only through advanced data analytics. When folks sign onto Netflix, they want the site to suggest movies they want to see. They want Amazon to offer them shopping suggestions that make sense.


Tracy Hewitt, marketing director at Business Insight, a not-for-profit consulting organization specializing in the application of advanced analytics to real-world problems, challenges and opportunities, described advanced analytics this way when she spoke to IT Business Edge's Lora Bentley in June:

Advanced analytics is built on the basics of business intelligence. So companies that are doing trends and statistics and reporting, we help them take that data and take it to the next level. Help them understand what's really driving the things that are impacting their business. Where are the new opportunities for their business? We also provide them the ability to do the "if" analysis. The ability to say, "If x happens, how is that going to affect my plan? My staffing? My budget?" Business intelligence is a big, big area, and we're kind of at the far end of that in the advanced analytics world. Trends, statistics, reporting, all those things are our building blocks. Once you have those in place and you really want to ask ambitious questions about the future, about forecasting, that's when the application of advanced analytics comes into play.

IBM, for one, hopes to cash in on this trend by opening analytics centers across the globe and last month announced plans to purchase predictive analytics specialist SPSS, a move Forrester Research predicts will trigger similar acquisitions among large software companies like SAP.


Another winner is the kind of collaborative model created by the competition, which brought together diverse teams of researchers using a wide variety of approaches. A New York Times story quoted Chris Volinsky, a scientist at AT&T Research and a competitor, who said the contest "will be looked at for years by people studying how to do predictive modeling."


Other companies, including Nokia and Electrolux, have sponsored contests to garner ideas to improve their products. These kinds of contests may grow in popularity during this down economy, which is causing many companies to reconsider their innovation spending. According to Forbes, Netflix spent $89 million on R&D last year. Darren Vengroff, a former lead researcher for Amazon's recommendation engine, called the contest "a tremendous strategy." He said:

"Traditionally, $1 million would get you about five researchers for a year. Instead, they spent the same amount and got thousands, probably millions of engineer-years."

Even the losers were winners. Arnab Gupta, CEO of Opera Solutions, a data analytics company that participated in the contest, said it experienced "a $10 million payoff internally from what we've learned," by using improved modeling and analysis techniques with its paying clients in the marketing, retail and finance fields.


Netflix is launching a new contest that will focus on improving recommendations for the newest users to its site, who haven't generated much behavioral data.