The hype and information overload around Big Data is in full swing, causing confusion for the many executives chartered with the task of leveraging Big Data for their organizations. The approach toward Big Data implementation may seem daunting, but early adoption of this new disruptive technology can provide serious competitive advantages. Big Data helps enterprises enhance their ability to scale seamlessly and handle the avalanche of data coming at them from a variety of sources, such as internal teams, public entities, social media, web applications, data centers, etc., and analyze these data sets quickly with greater efficiencies and lower support costs.
There are many issues to be addressed before Big Data is ready for wide-scale adoption, like platform maturity, training, deployment ease and flexibility, management, compliance, change management and many others, but while the technology ecosystem is evolving, enterprises can get started by developing a roadmap for Big Data adoption.
A systematic approach to adopting and scaling big data usage within the organization will help achieve business goals and can be divided into four phases – learn, initiate, scale and manage.
To get started, companies need to get answers to some of the most important questions that will help decide the components of Big Data infrastructure. This is the phase that lays the foundation for why there is a need for a new data infrastructure and also helps identify a starting point.
Identify a Big Data champion - A person or group who owns the new technology charter e.g., CTO office, innovation group, R&D labs etc.
Understand the technology and vendor landscape - Organizations should gain a comprehensive knowledge of the platform vendor options: Hadoop distributions such as IBM BigInsights, Cloudera, Hortonworks and other platform players like MapR, and Asterdata. Depending on the business need, a platform investment can be made.
Plan to adopt a Big Data Stack while leveraging existing data investments - Big Data will need to be complementary to the existing data infrastructure. Understanding how the new will integrate and co-exist with the old is crucial.
Understand the applicability of Big Data in your company – It is essential to know how to leverage the Big Data infrastructure (tapping structured, semi-structured and unstructured data sources). Examples of structured data are traditional DB with data type, format, first name-last name fields, etc. Semi-structured data is email and unstructured data is data from social media. When implementing the pilot project, take into account organizational business priorities rather than just proving technology capabilities. The more relevance to business priorities, the better the follow-through adoption of Big Data will be because people inside the organization will understand the impact better.
To be able to begin Big Data implementation, the organization needs to know the right use case to evaluate Big Data and should also define the goals and scope of a pilot project. The success of a pilot project should be flexible enough to drive broad adoption across the organization. This can be done by identifying 5-10 potential use cases. The projects should bring value to a specific group within the organization that is clearly measurable to help ensure successful adoption.
A robust project execution model is key to Big Data implementation and to manage execution risks associated with changing business needs and the evolving technology landscape. The right way to think about Big Data infrastructure is with the use of a shared infrastructure and cross-functional teams.
Once the big data infrastructure is set up, it becomes critical to measure the success with the help of predetermined SLAs and business metrics for ongoing improvement. Some of the key factors to consider are:
Taking on Big Data in a phased, systematic way as outlined above will help ensure successful adoption and implementation within an enterprise – and help deliver on the promise of Big Data.
As Senior Vice President, BI/Analytics Practice for Persistent Systems, a global software product development company, Vish Vishwanathan is responsible for driving technology, strategy, and business development; key focus areas include Big Data Analytics/Hadoop, Text Analytics/Sentiment Analysis.