Much of the attention surrounding the build-out of IoT infrastructure has gone to networking. As the enterprise edge pushes past the cloud all the way to the consumer device, the need to collect data from perhaps millions of points and route them to appropriate processing stations is indeed a major challenge.
But so is the need to store all that data. This facet of IoT infrastructure requires not just a major expansion of available storage but also increasingly sophisticated data management tools to determine where each bit is to be housed, and for how long.
According to 451 Research, storage has edged networking as the top concern among enterprises pondering their IoT infrastructure strategies. Nearly a third of respondents to a recent survey of top IT executives say they plan to increase spending on IoT storage infrastructure in the coming year, compared to about 30 percent each for network and compute resources. Interestingly, only about 27 percent say they plan to increase their off-premises cloud spend for IoT purposes, indicating that even massive scale is not enough for enterprises to trust others with their data.
Of course, simply throwing money at storage is not the right way to plan for the IoT future. As SanDisk’s Charlene Wan noted earlier this year, the key questions going forward will be what kind of storage is needed, and where. One of the chief requirements of the IoT is real-time analytics, which means data must be kept close to analytics engines in order to cut down the inevitable lag that accompanies its movement. This is probably the chief argument against the cloud as a significant contributor to IoT storage. While it does provide the scale for high-volume IoT workloads, it takes too long just to get data to remote sites. At best, the cloud can support broader analytics goals for non-time-sensitive applications, leaving the more critical operations to micro data centers on the edge.
Indeed, says Bill Evans, principal architect at tech consultancy ePlus, the preferred IoT storage solution is shaping up to be a multi-tiered architecture incorporating various storage mediums, governance policies, performance metrics and use cases. In this way, data can be stored temporarily while it is undergoing pre-analysis classification and verification, and then handed off to an analytics engine, stored for future use or simply discarded. One thing is certain, however, the sheer volume of IoT data excludes the possibility of saving everything.
It also means that traditional block and file storage will not be able to handle even a fraction of the overall IoT load, says Key Information Systems’ Clayton Weise. The hierarchical classification system of file storage becomes too Byzantine for high-volume, high-speed environments, while neither block nor file offer the scalability required of IoT infrastructure. Only object storage can provide the kind of flattened architecture that allows data to be accessed quickly and at relatively low cost. As well, object storage works well with both HTTP and RESTful APIs, just like most IoT devices, providing a more streamlined data exchange environment across distributed infrastructure.
If IoT infrastructure has anything in common with traditional IT, it’s that within the broad outlines of basic compute, storage and networking there will be countless customizations and optimizations aimed at giving one organization an edge over its rivals. Ultimately, this will lead to wide-scale architectures that are as unique as the enterprises that create them.
And perhaps the greatest irony of all is that technology decisions going forward won’t be driven by the latest and greatest system to hit the channel, but by the business outcome the enterprise is hoping to achieve.
Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.