One of the major challenges with anything to do with Big Data is that it usually requires somebody with a lot of deep data expertise to set up anything resembling a useful application. While the cost of acquiring and storing massive amounts of data has dropped substantially, the process of creating a Big Data application is anything but simple.
EMC via its Greenplum business unit, which specializes in Big Data technologies, this week took on that specific challenge by making a new Chorus framework for collaboratively creating Big Data applications generally available. It also decided to make that framework available as an open source project. That move immediately attracted the support of providers for analytics applications and other technologies related to Big Data.
In addition, EMC this week also acquired Pivotal Labs, a firm that specializes in building applications based on agile development methodologies. While Pivotal Labs will continue to operate as an independent unit of EMC serving a variety of customers, it will also work closely with EMC's Greenplum division to develop a new generation of Big Data applications, says Scott Yara, senior vice president of products for Greenplum.
The degree to which those applications will put EMC on a collision course to compete with enterprise application providers such as Oracle, SAP and IBM remains to be seen. But the advent of Big Data clearly presents EMC with an opportunity to use its data management expertise to break into new market segments.
Chorus, says Yara, is a core component in that effort. EMC is trying to foster a community of data scientists and developers that will create reusable data models and frameworks to speed the development of packaged applications based on Big Data platforms such as the Greenplum massively parallel processing database and Greenplum's distribution of the open source Hadoop data management framework.
The end goal, says Yara, is to make developing these applications anywhere from 100 to 1,000 times less expensive than it is today. How long that will take remains to be seen. But one thing that is for sure is that new classes of packaged applications specifically designed to use Big Data technologies are now on their way.