The discussions on Big Data are shifting away from the tactical toward a deeper discussion about Big Data's strategic value and how organizations can really put it to work.
https://o1.qnsr.com/log/p.gif?;n=203;c=204663295;s=11915;x=7936;f=201904081034270;u=j;z=TIMESTAMP;a=20410779;e=iAlthough I'm sure many people still have questions about the tactical issues, it's a welcome and timely shift. That's because all the Hadoop clusters and NoSQL databases in the world will be money poorly spent unless we first think through the challenges of turning all this Big Data noise into something meaningful.
Last week, I shared some of the expert opinions I'd found on the first key question: Are you really ready for Big Data?
This week, let's look at what those in the industry say it will take to translate Big Data stores into usable, relevant information.
The people. Maybe it's my past as a PR person, but before processes, before business cases, I think it's important to choose the right people for any initiative, and Big Data is no exception. In some ways, Big Data will be easy to sell - it's hip, it's hyped, it's happening now. But the realities of Big Data - the data discipline it will require, the shift in how we think about data - these things won't come so easy.
That's why the CIO is one of the key people who needs to be on board with Big Data. And to do that, CIOs really need to rethink their role. I know CIOs have been told this before, but insights from Big Data tend to come differently than other data.
In the past, you formulated a question, built a query and checked the data. Big Data requires a bit more of an open-minded process than that. You're not going to look for specific answers, so much as you're going to hunt for insights.
"Where capturing every data point was the legacy mission before, Big Data analytics will necessitate categorizing and discarding selected information," Rawat states. "The CIOs who intelligently trim costs while increasing functionality and agility will beat rival companies. They will ascertain which data is important, decide how much volume is statistically significant, and determine what can be removed."
The "I" in CIO will grow to mean not just information, but "intelligent" information, he writes.
A CIO who understands is a first step on the right path with Big Data.
There are a few other roles that need to be filled before you move ahead on Big Data, according to a recent TechTarget article, "Five first steps to creating an effective 'big data' analytics program."
Data owners. Data owners matter in a big way, with Big Data. That's because Big Data queries run against such large data sets, it's going to be important to make sure you're accessing the right data in the first place. You also need to work out all the business rules around that data, according to TechTarget. In the past, you might have gotten away with fudging this part, but now, it's no longer optional.
"In order to get the right analytical outputs, it's essential to include business-focused data owners in the process to make sure that all of the necessary business rules are identified in advance," TechTarget notes. "Once the rules are documented, technical staffers can assess how much complexity they create and the work required to turn the data inputs into relevant and valuable findings."
You're also going to need data owners who can help maintain and govern all that data, but more on that later.
Project leaders. Much has been written about the shortage of workers with the analytical skills required for Big Data. I think it's true for now, but I'm already hearing from BI vendors working to change that through templates and other resources, so I'm not sure it will stay true indefinitely.
But what is going to remain true is that you're going to need a project leader who can mediate between IT and the business on Big Data projects, according to TechTarget:
The better and more accurate that queries are in the first place, the less redevelopment will be required. Many projects require continual reiterations due to a lack of communication between the project team and business departments. Ongoing communication and collaboration leads to a much smoother analytics development process.
A Big Data Competency Center. If anything ever needed a competency center, it's Big Data. So take a page out of integration, and start a competency center. But as you recruit members of the IT and business staff to join a Big Data Competency Center, keep in mind the goal isn't to start out as know-it-alls; rather, the goal is to become competent.
That's because, at this point, the rules are still being written on the best way to deal with Big Data. For the first time, possibly ever, we have a cutting-edge technology approach that's available and could be possible for more than the usual set of early adopters. That's because it's cheap - most solutions are open source and can be run on any old server or in the cloud.
But that's no reason to be lazy about it. Put together a team that can meet monthly or quarterly (depending on your needs) and discuss what's been done and what you've learned from it. Start to share knowledge and best practices now to achieve benefits from Big Data sooner rather than later.