Editor’s Note: This is part of a series on the factors changing data analytics and integration. The first post covered cloud infrastructure, the second discussed new data types, and the third focused on data services.
Data keeps expanding, but only recently have organizations been able to store the data in useful ways. Now, organizations can theoretically keep data at the ready, whether it’s in the cloud, a data lake or in-memory appliance.
Hopefully, it will soon be archaic to hear my doctor say, “Oh, we sent that x-ray to tape. We could get it — but it’s a huge hassle.”
The ability to store mass data is one of the five data evolutions that David Linthicum cited in his thesis on “The Death of Traditional Data Integration.” The ability to pool Big Data sets would not be disruptive, though, if it weren’t coupled with the ability to access it easily and as needed for analytics. As Informatica CEO Sohaib Abbasi points out, this “richness of big data is disrupting the analytics infrastructure.”
But why? When I started looking at this, I said that I thought there were more than four trends changing how we deal with data and data integration. Certainly, mass data storage and the richness of the data are major drivers, but I think they point to a more significant macro trend within organizations: Business users, from marketing and sales down to supply chain managers, want access to that data.
Why It Matters
We’ve talked about the data-driven business for years, but now, the technology exists to actually support it. What’s more, business users know it and want access to that data. In the past, business users and leaders were content with BI reports or dashboards, but now they want to leverage the data for pro-active decisions. That requires predictive analytics, which leverages each of the technology trends identified by Linthicum and Abbasi.
“Assume that the strategic use of technology and data will begin to provide even more value to most enterprises, and hopefully create a sense of urgency that things need to quickly change,” Linthicum writes in his report.
The question is: Will IT be ready to act?
You already know that the IT/business paradigm has shifted in the past five years, largely because of cloud. Because of that change, IT has spent five years playing catch-up on data integration and data governance issues. Perhaps it’s time to take a page from the business and shift from reactive to predictive mode.
Fortunately, the business is already helping make the business case for data and analytics spending, but there are still challenges. Here are a few questions to consider:
How will you fund an enterprise-wide data infrastructure? Right now, business has been leading — Gartner predicts that the chief marketing officer's IT budget could outstrip the CIO’s budget by 2018. Marketing has lead analytics investments, but now other business functions want in — so who pays? If you use a chargeback approach, how can that be leveraged for broader spending? If you use a central IT budget, which business allies can help you make the case for a bigger data and analytics budget?
How do you prioritize analytics and data work? Analytics isn’t just a stand-alone project anymore. Enterprise software vendors are now partnering with analytics companies to offer embedded analytics tools — but there’s a cost. This is no time to shirk a business case. Mark Shilling, a principal with Deloitte Consulting, told me each analytics project should offer a “quantifiable value on what is being generated.”
What integration tools do you need? Integration becomes much more of an issue when you move to streaming data or real-time analytics, since it can become a bottleneck that holds up the entire process. It’s all too easy to invest in point solutions and sometimes that’s actually a smart plan. What you want to avoid, though, is multiple solutions adding complexity and barriers to an enterprise-wide approach. If you don’t have an integration center of excellence, start one. If you do, start a discussion about what needs to change as data goes enterprise-wide.
How does data governance and data quality need to change going forward? Your requirements for data quality will vary greatly depending on the use case. So what types of reports or decisions require the top standards for data quality? What about governance — which departments own not just the data, but responsibility for the data?
Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.