Continuing yesterday's theme on old school data integration tools, today I learned something interesting about ETL tools: More companies are deploying ETL (extract, transform, load) tools outside the typical data warehousing project.

During a TechTarget podcast, recently made available on Forbes, Bloor Research Director Philip Howard said ETL is still primarily used for data warehousing, but there are now four other "markets" or use cases for ETL:

1. Data migrations and conversions, which, when coupled with data warehousing, compose two-thirds of all ETL projects
2. Application synchronizations
3. Business-to-business for converting SWIFT, HIPAA and other messages
4. Providing data services for SOAs

It's interesting to note that while companies are deploying ETL tools in wider scenarios, Howard said his research found there's actually very little functionality differentiation in this crowded market, which includes over 40 tools. And your task is made even more complicated by the fact the tools are too complex for a proof-of-concept test. Said Howard:

"It would be nice to think you could do proofs of concepts, but the typical tool takes probably at least a couple weeks to learn how to use, in some cases more, particularly the big complex tools."

So, if all tools are very similar, why not just go for an open source solution and save money, right? Wrong again, said Howard. Recent Bloor research found that, in fact, open source solutions did not offer a better total cost of ownership, in part because most companies are apt to reuse these tools over the course of years.

Howard did offer a few points on how you might go about picking an ETL tool:

• If you have a complex environment with lots of targets and transformations and you will be using it for large projects running weeks to months, then start with the usual big vendors -- Informatica, IBM and so on.
• Talk to an industry analyst, not a consultant who will be implementing the tool and may have a vested interest in you picking something that works well for the consultant.
• Look at the cost per project. Often, these tools get reused, which gives you productivity gains over the long haul. The average Global 2000 company, for instance, does four and a half migrations a year, Howard said.
• Consider a trial run. While the tools are too complex for just a proof-of-concept, some companies do offer short-term licenses for single projects. This will lower your up-front costs and allow you to decide if you really like using the tool before you commit to a long-term license.

To hear this podcast, you'll need to register with Bitpipe, which is free, but takes about as long as applying for a mortgage. Also, plan to listen to it in a very quiet environment -- even with my headset and volume maxed out, it was difficult to discern parts of Howard's answer.

Bloor Research isn't the only organization that's documented the spread of ETL beyond the data warehouse. TDWI Research also identified the trend. TDWI Research Senior Manager Philip Russom will offer a free webinar on "Operational Data Integration: The Growing Practice of ETL Outside of Data Warehousing," August 13, starting at 12 p.m. ET/9 a.m. PT.

TDWI's study showed ETL's use is actually growing fastest outside of data warehousing. Russom will discuss why this is happening and what role SOA and SaaS play in this growth. Since TDWI's focus is on data warehousing, he'll also discuss how you can accommodate the need for non-warehousing ETL projects without depriving your data warehousing team of its tools.