As Big Data has come to the forefront of the next wave of technological innovation, in-memory data grids have become a powerful tool.
Using in-memory data grids for ETL on streaming data.
A key challenge for any data warehouse is to supply data in a format for easy ingestion and analysis. This is the role of the well-known process called "extract-transform-load." In today's Hadoop world, this often means extracting data from external sources and transforming it into a form that can be stored in HDFS. Traditional ETL processes don't work for streaming data that continuously flows into the data warehouse.
An in-memory data grid with an integrated MapReduce engine can capture the data stream in real time, perform ETL, and offload the data warehouse.