ETL Still the Workhorse of Enterprise Data Integration

Loraine Lawson

There aren't a lot of vendors boasting of their ETL tools these days. Extract, transform and load seems so well, basic, in this age of Big Data, cloud and virtualization.


But whoa there, Silver. If you look a little closer, you'll see that ETL is actually the workhorse driving most of today's "hot" integration solutions, according to this year's Forrester Wave: Enterprise ETL, Q1, by Principal Analyst Noel Yuhanna. He writes:

Slide Show

Data Integration Remains a Major IT Headache

Study shows that data integration is still costly and requires a lot of manual coding.

Product marketers working for ETL vendors have worked feverishly to lose the ETL' from their product names, marketing collateral, and websites, instead positioning their solutions as comprehensive data integration platforms. And while the data integration platform' moniker is accurate for most of the vendors that have since added complementary capabilities like data profiling, data quality, metadata management, CDC, and data federation to complement their ETL core, the fact remains that many of their customers are only looking for an ETL solution.

In fact, Yuhanna predicts organizations will become more dependent on ETL than ever before, largely because ETL supports the "heavy lifting of data from transactional systems to operational data stores, data warehouses, BI platforms, MDM hubs, the cloud, and Hadoop platforms."


Yuhanna identifies all the "new" technologies where, if you wash off the paint, you'll find that old workhorse, ETL, driving the integration, including:


  • Data virtualization, which often uses ETL to move data to the virtualized data services layer and as an integration tool for heterogeneous sources such as mainframes and different database types.
  • Cloud computing, where ETL is used to support the loading and unloading of data while ensuring transactional integrity and security, he writes.
  • MDM. In recent years, Informatica, Talend and other ETL-based companies have expanded into master data management. Did you ever wonder why? "ETL technology is the foundation of many MDM implementations supporting the high-volume batch loads required to migrate data from new sources to the MDM hub and publish the cleansed master data in multiple target environments," explains Yuhanna.
  • Real-time data warehousing. ETL is best known as a batch-processing solution, which you may typically associate with a big job run once a day or week or month. But integrate or embed ETL with change data capture, and you get a solution that can run in near-real time, according to Yuhanna.
  • Retiring legacy apps. ETL is the go-to solution when companies want to retire a legacy system, Yuhanna writes. And over the past few years, enterprises have been retiring more systems by archiving the data or moving it to a data warehouse using - you guessed it - ETL. Some ETL vendors even target this use case by offering prepackaged data migration solutions at special prices, he writes.
  • Big Data solutions. Hadoop may provide the powerful storage capacity, and MapReduce the processing, but when you want to actually move the data into or out of Hadoop it's ETL to the rescue. That's why so many ETL vendors are touting new connectors with Hadoop.

Social data is a big component of Big Data, too, so watch for ETL vendors to start promoting solutions that target the integration of social media. Informatica got the ball rolling on this use case at its recent analyst summit by demonstrating how its data management platform could be applied to social media tools such as LinkedIn and Facebook.


Ventana Research CEO and Chief Research Officer Mark Smith said Informatica had taken "the most progressive position of any vendor in data integration and information management toward social media" and social media is expected to play a big role in Informatica's push to become a $3 billion revenue company.


But with other ETL vendors offering connectors to Hadoop and Big Data solutions, I suspect Informatica will soon have competition in that space.


The Forrester Wave report is available for free download and definitely worth a quick read, particularly if you - like many organizations - are thinking about investing in an ETL tool. What's nice about the Wave - as compared to Gartner's Magic Quadrant on data integration tools - is that they look at a few more solutions and give you some perspective on when you might use less well-known solutions, including Pervasive, Pentaho, Talend and Syncsort. It also offers an assessment of Microsoft's SQL Server Integration Services, which will be tempting for many since it comes with SQL Server, but is typically used in addition to an enterprise-wide ETL tool, he says.


Forrester's Wave on ETL is available for $2,495 on the research firm's site or Pervasive Software is offering it as a free download, if you're willing to provide the usual registration information.

More from Our Network
Add Comment      Leave a comment on this blog post
Mar 7, 2012 7:36 AM Michael Waclawiczek Michael Waclawiczek  says:

Your assessment is right on.  ETL is her to stay.

Aug 10, 2015 11:35 AM wilbertkilnton wilbertkilnton  says:
It’s really a great pleasure to provide an opinion about ETL tools. These are very important and useful to all the people from all over the world. ETL tools are useful to everyone which help to transform any data into any database fast and easy and comfortably. ETL Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.