Continuing yesterday's theme on old school data integration tools, today I learned something interesting about ETL tools: More companies are deploying ETL (extract, transform, load) tools outside the typical data warehousing project.
During a TechTarget podcast, recently made available on Forbes, Bloor Research Director Philip Howard said ETL is still primarily used for data warehousing, but there are now four other "markets" or use cases for ETL:
It's interesting to note that while companies are deploying ETL tools in wider scenarios, Howard said his research found there's actually very little functionality differentiation in this crowded market, which includes over 40 tools. And your task is made even more complicated by the fact the tools are too complex for a proof-of-concept test. Said Howard:
"It would be nice to think you could do proofs of concepts, but the typical tool takes probably at least a couple weeks to learn how to use, in some cases more, particularly the big complex tools."
So, if all tools are very similar, why not just go for an open source solution and save money, right? Wrong again, said Howard. Recent Bloor research found that, in fact, open source solutions did not offer a better total cost of ownership, in part because most companies are apt to reuse these tools over the course of years.
Howard did offer a few points on how you might go about picking an ETL tool:
To hear this podcast, you'll need to register with Bitpipe, which is free, but takes about as long as applying for a mortgage. Also, plan to listen to it in a very quiet environment -- even with my headset and volume maxed out, it was difficult to discern parts of Howard's answer.
Bloor Research isn't the only organization that's documented the spread of ETL beyond the data warehouse. TDWI Research also identified the trend. TDWI Research Senior Manager Philip Russom will offer a free webinar on "Operational Data Integration: The Growing Practice of ETL Outside of Data Warehousing," August 13, starting at 12 p.m. ET/9 a.m. PT.
TDWI's study showed ETL's use is actually growing fastest outside of data warehousing. Russom will discuss why this is happening and what role SOA and SaaS play in this growth. Since TDWI's focus is on data warehousing, he'll also discuss how you can accommodate the need for non-warehousing ETL projects without depriving your data warehousing team of its tools.