dcsimg

How Hadoop Is Being Used for Business Operations Today

  • How Hadoop Is Being Used for Business Operations Today-

    Types of Workloads Do Matter

    There is an interesting correlation between the types of workloads and the size of Hadoop clusters. Respondents who cited "streaming / real-time" as one of their workloads tended to have more clusters in production (46 percent had four or more clusters). Among respondents who did not have streaming or real-time workloads, only 20 percent had four or more clusters. The move to real time is adding cost and complexity to Hadoop deployments, through the use of cluster isolation as a best practice to guarantee performance. In order to successfully run Hadoop in production, organizations need to start moving away from cluster isolation and toward Quality of Service for Hadoop so they can run real-time/streaming applications (e.g., Spark) alongside batch workloads (e.g., MapReduce) on a single cluster.

1 | 2 | 3 | 4 | 5 | 6 | 7

How Hadoop Is Being Used for Business Operations Today

  • 1 | 2 | 3 | 4 | 5 | 6 | 7
  • How Hadoop Is Being Used for Business Operations Today-3

    Types of Workloads Do Matter

    There is an interesting correlation between the types of workloads and the size of Hadoop clusters. Respondents who cited "streaming / real-time" as one of their workloads tended to have more clusters in production (46 percent had four or more clusters). Among respondents who did not have streaming or real-time workloads, only 20 percent had four or more clusters. The move to real time is adding cost and complexity to Hadoop deployments, through the use of cluster isolation as a best practice to guarantee performance. In order to successfully run Hadoop in production, organizations need to start moving away from cluster isolation and toward Quality of Service for Hadoop so they can run real-time/streaming applications (e.g., Spark) alongside batch workloads (e.g., MapReduce) on a single cluster.

There is little doubt that Hadoop adoption is growing, and not just among enterprise-sized organizations, but by small- and medium-sized businesses as well. In an effort to understand this maturing market more deeply, Pepperdata conducted a survey about how and why Hadoop is used for business operations.

The 134 survey respondents came from a range of experience, but all work at companies currently running Hadoop in production. The majority of respondents were from software engineering/development, data scientist, or data architect job titles (25 percent, 17 percent, and 12 percent, respectively). Almost half (40 percent) were from the information technology industry, with education and financial services (11 percent and 10 percent) coming in second and third. Over 45 percent have been in production for two years or more, with 15 percent of those being "advanced users" (four years or more in production).

In this slideshow, Pepperdata shares findings from the survey, such as key use cases, the size of Hadoop environments, and biggest challenges to production deployment.