IoT Data: Insights Are Hard to Achieve in an Ocean of Possibilities

The Challenges of Gaining Useful Insight into Data

The Internet of Things (IoT) will create oceans of new data — but finding meaning amid the swell will be challenging.

That’s what some companies are finding when they deal with unstructured data in general. Despite the success stories, a recent Forrester survey shows our ability to manage and use unstructured data significantly lags behind traditional, structured data. Forrester found that on average, enterprises use about 35 percent of their structured data for insights and decisions. By comparison, only 25 percent of unstructured data is leveraged for those tasks.

Perhaps that doesn’t seem so bad, given how long enterprises have invested in structured data. But focus on only the IoT data, and the picture becomes even murkier. A recent Dimensional Research study, sponsored by IoT analytics vendor ParStream, asked 203 IT professionals, including systems integrators and IoT stakeholders, about their IoT projects. They found that only 8 percent agreed that they’re fully capturing and analyzing IoT data in a timely manner — even though an incredible 86 percent said data is important to their IoT projects.

Despite the low number of IoT projects capturing data, 70 percent of respondents said improved data would lead them to better, more meaningful decisions. I have to wonder how that’s going to happen if only 8 percent are actually capturing and analyzing said data in a timely manner.

It seems the reality of collecting and using IoT data is much, much trickier than success stories would lead us to believe. CIO.com’s guest column, “Are you ready for the Data of Things,” explains why the Data of Things (DoT) is much harder than the Internet of Things.

“DoT will be disparate, messy, in constant flux and inherently incomplete in nature,” writes Kumar Srivastava, senior director of product management at cloud Big Data analytics company, ClearStoryData. “DoT will be sparse, compacted and compressed. It will require being contextualized, decorated and described with data and metadata from other data sources.”

On top of all that, the quality of the data will be “unpredictable,” due in part to variances caused by the IoT’s routing mechanisms, he adds.

Organizations will have to offset these problems by blending or harmonizing IoT data with other data sets. These additional data sets will provide the context — time, environment, location, or other related details — about the IoT data, Srivastava explains.  

Honestly, it sounds like a nightmare. IoT data might not be worth the effort if it weren’t for the capabilities it promises.

“DoT will enable micro segmentation, micro messaging and micro behavior management,” Srivastava writes. “That is, enterprises will be able to control the behavior of micro segments across their network of connected devices, apps and systems.”

Of course, these micro segments will also strain your existing data pipes and processing systems. But, at least you know what you’ll get in exchange.

It seems clear that organizations understand the value of IoT data, but they’re unclear how to actually get from point A (collect the data) to point C (derive value from the data). Until that’s resolved, IoT projects may prove more frustrating than useful.

“The intersection of IoT and Big Data is creating tremendous business opportunity, but the reality of today’s projects is that this potential has yet to be fully tapped,” Diane Hagglund, Dimensional Research principal, told ParStream.

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.