The emergence of Hadoop as the de facto Big Data operating system has brought on a flurry of beliefs and expectations that are sometimes simply untrue.
Tips to avoid this pitfall:
Hadoop provides virtually unlimited horizontal scalability. However, hardware and development costs can quickly hinder sustainable growth. Therefore, it’s important to maximize developer productivity and per-node efficiency to contain costs.
- Choose cost-effective software and support, including both the Hadoop distribution and the data integration tool.
- Ensure tools include features to reduce development and maintenance efforts of MapReduce jobs.
- Look for optimizations that enhance Hadoop’s vertical scalability to reduce hardware requirements.