The Challenges of Gaining Useful Insight into Data
Business users aren’t just technology savvy these days. They’re also increasingly data savvy, and that’s lead to a major shift in what business users expect when it comes to accessing and using data, according to data integration veteran Sachin Chawla.
“These guys don’t even exactly know the questions,” Chawla said during an interview with IT Business Edge. “They want to start playing with the data and then the questions will emerge as they do that, and the value will emerge as they do that. So it’s more about exploration than ‘Oh, tell me how much product we’ve sold in this region every month.’”
This represents a significant shift from the traditional approach, in which business users request reports that may take IT months to produce, Chawla said.
Chawla should know. He’s worked in the data and integration space for the better part of three decades, according to his LinkedIn profile, starting with data analytics company Acta Technology, which was acquired by Business Objects in 2002. He went on to serve as VP of data integration there before moving on to become a VP of product development at GoldenGate Software, and then working with Oracle and Informatica’s data integration platforms.
Chawla is now the vice president of engineering at data transformation company Trifacta, which focuses on giving business users tools for visualizing and analyzing data.
“Our contention is that the line of business user now wants to start accessing all this rich information that’s sitting in this Hadoop cluster, in this data lake,” he said. “They want to start blending that data, wrangling that data, sort of conforming it to how they want to view it and even understand that data.”
The challenge for IT will be to enable that exploration without inhibiting it, he said.
“The problem is that by analyzing and looking and understanding what’s there, more questions pop up and I need to operate at that speed. I can’t do that with IT and I’m capable enough of a business user now to do that myself,” he explained. “So IT’s job is to do maybe some level of pulling in the data, ingesting the data, organizing it and then making it available and that compute power available to me, so I can get the answers I want.”
I asked Chawla whether that meant his customers were primarily business users, perhaps trying to run around IT in the same way the LOBs bypassed IT by adopting cloud applications. Actually, no, he said: The company almost universally works with both. Having built Hadoop clusters and data lakes, IT wants to make that data available to business users.
Tools such as Trifacta are designed to fill that gap, he said, allowing business users to explore large data sets in a controlled fashion, without compromising security or privacy.
Chawla and others I’ve spoken with recently say this shift doesn’t mean IT will be cut out of data. On the contrary, IT will still be responsible for maintaining the data and ensuring that its security, privacy, quality and heritage are maintained. That will leave exploration and discovery to the business user, although perhaps with some help from a specialized data science team.
Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.
