Articles for Effective ETL -- And Eliminating ETL

Loraine Lawson

Intelligent Enterprise is revisiting what it calls the 34 subsystems of ETL -- which stands for "extract, transform and load," just in case you've been living in a IT vacuum the past 10 years.


ETL is a key data integration strategy. It's widely used, still very effective, and hence is unlikely to be usurped any time soon by SOA or any other Johnny-come-lately. So, the article on building and managing an ETL architecture is a great resource. It's written by Bob Becker, a member of Kimball Group -- hence the "Kimball University" tag before the title. If you have questions, Becker even gives his e-mail at the end of the article.


But what I found more interesting is actually in another article, a few pegs down on the home page, about how Pfizer Global Research and Development uses data integration middleware to eliminate ETL projects. To be frank, ETL was dragging everybody down -- IT, researchers, the finance people at Pfizer who are trying to make money back on all that research.


Pfizer Research does have data problems that go beyond your run-of-the-mill data challenges. The company must shift through hundreds of sources for anything relevant on a drug, including its effectiveness, possibly toxicity, patent information and so on. Portfolio managers need that information to make critical business decisions. Until now, they could only get it through IT, which used ETL to integrate the data from various sources.


To eliminate the need for ETL jobs and new warehouses, Pfizer created a portal environment using integration middleware from Composite Software. Now, developers can log into a portal and quickly access the data they need.


What once took three to four months for IT to accomplish can now be done in less than a week with the portal. And that's a tangible result for the business, which no longer has to wait months for IT to process and package critical business data.

More from Our Network
Add Comment      Leave a comment on this blog post
Oct 24, 2007 4:13 AM Robere Eve Robere Eve  says:
I run Marketing for Composite, so let me get that disclosure on the table from the start.We have seen a big shift in data integration buying interest in the past year. We used to have to evangelize and educate on "EII", real-time data integration, data federation, data virtualization or any of a number ways people describe our data integration without replication approach.But lately, our approach is more accepted, more mainstream. and the interest has shifted from "what is it? " to "how will I benefit?"The Pfizer case study captures the big ideas. You will benefit by not having to build a mart, warehouse, or ODS, and thus you'll integrate data far faster, for far less money. Pretty simple really.While not appropriate for the super simple projects or the massive data warehouses, there is a large middle ground where a "virtualized" approach to data integration is "just right."A remember its the data silos and complexity that got us into this mess to begin with. So why perpetuate your problem with "solutions" (ETL) that generate more silos (marts, warehouses, operational data stores). - Bob Eve, VP Marketing, Composite Reply
Oct 30, 2007 9:26 AM Vincent McBurney Vincent McBurney  says:
The rapid deployment of EII has got the attention of customers and the data integration vendors with Informatica and Cognos reselling the Composite tool and IBM putting the Federation Server onto the Information Server. So far ETL remains more common than EII as companies struggle with the mapping, metadata, performance and security issues. The data integration vendors need to make ETL tools faster and easier to implement as consolidate is still the preferred way to build BI solutions. Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.