5 Steps to Wrangle Uncontrolled Data Flow
The ability to inexpensively collect massive amounts of Big Data using Hadoop is one thing. Being able to manage all that data is another matter altogether. With that issue in mind, Hortonworks today announced the upgrade of a Hortonworks DataFlow (HDF) streaming analytics platform with an eye toward addressing a broad range of enterprise management issues.
HDF 2.0 now sports a revamped user interface that tightens the integration between the streaming analytics platform that is based on the Apache NiFi data routing engine and Apache Kafka messaging software and Apache Ambari tools for managing Hadoop deployments. Apache NiFi itself is based on streaming analytics software created by Onyara, which Hortonworks acquired last year.
In addition, Hortonworks has announced that it has integrated HDF 2.0 with Apache Ranger software for securing access to data in Hadoop.
Finally, Hortonworks revealed that it has developed Apache MiNiFi, an implementation of Apache NiFi that has been optimized to process streaming analytics on an Internet of Things (IoT) gateway, while at the same time formally certifying over 150 third-party HDF 2.0 connectors.
Jamie Engesser, vice president and general manager for emerging products at Hortonworks, says a truly comprehensive approach to Big Data requires an ability to process and manage data both in motion and at rest. HDF now extends the ability of Hortonworks to process data all the way out to the edge. That approach, adds Engesser, also serves to limit the amount of network bandwidth that needs to be allocated to an IoT project.
“Now organizations can actually prioritize what data needs to be sent back across the network,” says Engesser.
HDF 2.0 is an example of how streaming analytics is now being applied to Big Data both inside and out of Hadoop itself. The next challenge IT organizations face is going to be determining to what degree any of this open source streaming analytics software might either complement or obviate previous investments in analytics applications throughout the rest of the enterprise.
