Beware the Data Collectors for They Are Us: Privacy and Big Data
Moving to further integrate Hadoop and Vertica, Hewlett-Packard this week released an update to its SQL columnar database that adds support for data streaming and advanced log file text search to better support high-speed analytics typically associated with Internet of Things applications.
HP made the announcement at the HP Big Data conference, and revealed that it is making additional Big Data open source contributions in the form of tools that help optimize Hadoop performance, provide tighter integration with the Apache Kafka distributed messaging system, and advances to the Distributed Rpredictive analytic project that HP launched earlier this year.
Specifically, HP is making available HP Vertica for SQL on Hadoop native file support, which provides an access layer to enable SQL queries to run as much as five times faster against Hadoop formats such ORC and Parquet. HP is also making available HP Vertica Flex Zone Table Library, which can be used to create schemas on demand against unstructured data.
HP is committing to providing in the future native integration between HP Vertical and Apache Spark in-memory cluster software. Finally, HP launched an HP Haven Startup Accelerator program, which provides early-stage companies more affordable access to both HP Big Data platforms and application delivery management software and services. Haven is an HP platform that combines Vertica and Hadoop with the Intelligent Data Operating Layer (IDOL) search technology that HP gained when it acquired Autonomy.
Jeff Healey, director of product marketing for Big Data at HP, says this integration effort further tightly couples a columnar database that supports SQL queries with a Hadoop platform that is rapidly becoming the foundation of modern data warehouses. While IT organizations are embracing Hadoop, however, they remain steadfast in their investments in SQL. As such, IT organizations are looking to build composite applications that span columnar databases that are as tightly integrated with Hadoop as possible, says Healey.
Naturally, there are multiple ways to integrate SQL with Hadoop. But as far as HP is concerned, Healey says none of those alternatives is likely to match the performance of an HP Vertica columnar database co-located on the same cluster as Hadoop.
