Big Data May Save Your Life

One of my long-time friends mocks my love of meteorology. He ranks it right above tea leaves and palm readings as a “science” and he isn’t shy about telling me so when I’m warning him about a pending storm front.

I can see why he’d say that, although I do think meteorology has improved tremendously over the past decade.

But I can’t deny — and neither would any meteorologists who works in the Ohio Valley — that there’s a great deal of prediction to it. To me, that speaks more to the many variables at play than to a lack of scientific effort.

Not surprisingly, meteorologists tend to feel the same way. Their issue isn’t bad science: It’s data.

A CNBC article looks at who’s doing what when it comes to advancing weather modeling and prediction:

“‘Today’s traditional forecasting models tend to break down in a week to 10 days because they can’t input every observed data piece from around the world,’ John Plavan, co-founder and CEO of EarthRisk Technologies, told CNBC.com this week. ‘There’s too much chaos in the system.’”

This looks like a job for Big Data. And, it turns out, Big Business.

You probably won’t be surprised to hear that IBM is among the players, given its highly publicized work with Watson. IBM is really pushing Big Data innovation in other, less publicized ways, though, and weather is yet another example. The company launched a division devoted to using Big Data tools to improve weather forecast modeling, and nicknamed it — I kid you not — Deep Thunder.

While everyone hopes using Big Data technologies in meteorology will lead to fewer deaths and better storm preparation, it’s not charity work. Deep Thunder’s client list includes utility companies and other businesses whose day-to-day work is affected by weather and natural disasters. So that’s yet another example of how Big Data is not just changing how we do things, but actually creating new services and lines of revenue for companies that invest in it.

Big Data continues to amaze and, yes, frighten me. Information Week’s April issue focused on Future Cities and one of the stories it covered was Chicago’s predictive analytics system, which combines Big Datasets, mapping tools and open source tools like MongoDB.

The name of the system, WindyGrid, will send conspiracy theorists to the brink, I’m sure. But right now, the focus is on simple correlation questions, such as “Why are our garbage carts disappearing in some areas and not others?” Answer: Garbage cart theft corresponds to street lights going out.

I’ll grant you, Chicago has bigger problems but, hey, it’s a start, and stopping theft is a pretty solid ROI.

Better weather prediction. Fewer deaths. Cutting the costs of replacing dumpsters. This is why Big Data is news and receives so much attention, both in the trades and, increasingly, the national news.

I’ll grant you, it may not be as fun as having tea leaves read, but the results are so much more gratifying, particularly when I think about calling my friend up to say, “Wave’s meteorologist says it’s going to snow that day,” and knowing there’s Big Data to back that up.