The Four Plagues of the Coming IoT Datapocalypse

Five Ways to Use Data to Be More Relevant to Customers

The Internet of Things (IoT) is attracting attention way beyond most technology trends, no doubt because of predictions that it could trigger an economic boom. Who doesn’t want that?

That’s leading to some hard questions from technologists and humorous headlines from The Register, a UK site. Two recent examples:

• “Boffins challenge Internet of Things’ lust for power”
• “Stop the IoT revolution! We need to figure out packet sizes first”

In all seriousness, technologists do seem to be having a “holy cow” moment over the IoT. It’s pretty clear that there are more questions than answers about the IoT — particularly when it comes to handling the data.

TechTarget’s SearchDataCenter published a good first read on the IoT implementation challenges. The title, “Plan an Internet of Things architecture in the data center,” is actually too narrow for the article, which covers the end-to-end architecture issues, including WAN bottlenecks and the potential impact on applications.

Despite the TechTarget title, you’ll actually find a better analysis on how this impacts the data center in this ReadWrite.com article. It draws on a recent, subscriber-only Machina Research study to argue that NoSQL databases will be critical for managing the heterogeneous data streaming in from sensors, devices and gateways.

The big takeaway in both pieces: The IoT is going to force you to restructure, rebuild and re-architect IT. How much of a restructure are we talking about here? Let’s just say Gartner says it could trigger “a sea of change in IT.”

Let’s walk through the integration and data-relevant concerns:

Rethink applications. Again. All that work you did centralizing applications as a way to cut costs? Bad news: The data volumes may make that “unsustainable,” the article notes. Instead, IT will need to support a more distributed computing approach.

Data lakes? Try data puddles. Much has been said about the potential for data lakes, particularly in light of sensor data. That may be so, but we’re still dealing with relatively small amounts of incoming data. A more mature IoT will put more demands on storage. Gartner contends that that will force organizations to collect data in “multiple distributed mini-data centers,” the article notes.

Data gets edgy. The experts quoted do agree on one thing: Given the potential data load from the IoT, more data will need to be sorted and analyzed, at least initially, at the edge of the network. That edge may be in the sensor, somewhere outside your centralized systems, or in the cloud — but the data will need to be pre-sorted in some way. That will make latency and bandwidth major factors for IoT data. You’ll need to consider how your data integration or ETL processes may help or hinder that.

Got NoSQL? Probably not. No one is saying abandon your relational database management systems. On the contrary, every piece I read is quick to reassure CIOs that there will always be a role for this highly structured approach to data, including Matt Asay’s ReadWrite article. Always start with the data model before you decide on storage, the experts remind us.

But seriously, folks: Let’s get real here. NoSQL was designed for this kind of data, and if you’re going to dabble in the IoT, Asay’s piece suggests that it’s time to suit up with some NoSQL superpowers:

… it’s not merely a matter of heterogenous, semi-structured data that creates the need for NoSQL. According to DataStax CTO Jonathan Ellis, ‘Relational databases like Oracle are great for dealing with data from a single company or department, but cannot provide the scale or availability that a database designed for the cloud like [NoSQL] Cassandra can.’

Okay, fine, you’re convinced, right? Good luck finding a NoSQL programmer. As Asay points out, we will need about 813,000 by next year, since research shows there are about 300,000 right now.

These aren’t the only problems facing the IoT, of course. Other data issues raise different, but still difficult, concerns about its usefulness to organizations, as well. It seems my IT Business Edge colleague Carl Weinschenk was right when he wrote, “Reality threatens the Internet of Things Dream.”

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.