Gartner estimates that within two years, only 10-15 percent of businesses will take full advantage of Big Data. But that 10-15 percent will outperform their laggard competitors by 20 percent in financial metrics.
What can you do to make sure you’re not one of the laggards? A good first step would be to download and read the “Big Data Consumer Guide” recently published by the Open Data Center Alliance (ODCA) Data Services (DS) Working Group. It’s an excellent resource for understanding when and how to use Big Data, including an outline of which use cases by industry.
“Today we stand at the threshold of yet another transformation, where those who ‘get it’ will continue to thrive and grow, and those who remain lodged in outdated technologies will fall by the wayside,” the paper states. “What used to be considered a storage problem is now a strategic asset.”
One issue with Big Data is that the definitions of it do vary somewhat — although, from what I’ve seen, everybody pretty much goes back to the “Velocity, Volume and Variety” definition we hear so often. Still, this paper is an attempt to iron out some of the language differences by way of the ODCA, which is an independent IT consortium. It’s led by a 12-member steering committee that includes IT representatives from BMW, Capgemini, China Unicom, Deutsche Bank, JPMorgan Chase, Lockheed Martin, Marriott International, Inc., National Australia Bank, T-Systems, Terremark, Disney Technology Solutions and Services, and UBS. Intel serves as technical advisor to the Alliance.
I’m sure you’ll notice that there are plenty of technology firms missing from that list. Still, the ODCA’s paper is extremely useful, and includes discussion of solutions offered by traditional vendors such as IBM and Oracle, as well as open source solutions, such as Hadoop.
There are two things I really like about this report. First, it identifies the “gaps” in Big Data technologies. One of these gaps speaks to the much-discussed skill gap, but it’s very thorough from a data management perspective as well. For instance, it notes that Big Data technology tends to be batch-oriented, rather than micro-batch or real-time, that there’s inadequate support for:
- Data governance
- Data lineage
- Data integration
“Established architectural patterns do not exist, and systems for typing things together are still in the design phase,” the report states. “Compare this situation to the integration that exists between enterprise application integration, enterprise service bus, and ETL technology, and it is obvious there is significant room for improvement in this area of Big Data technology.”
The second thing I thought was excellent is the report walks you through planning a Big Data strategy, and when I mean “walks you through,” I mean it takes your hand and walks you through every consideration, including:
- Finding executive stakeholders
- When to use Big Data
- A discussion of open source versus third-party distributions versus proprietary solutions
- How database admin, developers and database developers need to adjust for Big Data
- How to fill that skill gap
- Infrastructure considerations
Given that it’s only 24 pages, that’s a lot of ground to cover, and it does it better than any free document I’ve seen thus far. It’s available as a free PDF download, so you can read it on your smartphone or tablet this weekend.