Maintaining Control of All That Data

Michael Vizard
Slide Show

Data Integration Remains a Major IT Headache

Study shows that data integration is still costly and requires a lot of manual coding.

Due largely to performance issues and the sheer volume of data involved, the whole extract, transform and load (ETL) process has been gaining an increasingly bad reputation within IT circles. The core issue is that current approaches to automating ETL simply don't scale, while the amount of time that IT organizations can allocate to support this kind of batch processing is shrinking.


As a result, the folks at Syncsort are starting to make a case of ETL 2.0, which in their view will soon lead ETL engines running in memory as a way to solve the current performance and scalability limitations of ETL.


In a Big Data era, the whole data integration process is lagging behind recent advancements in memory technologies. Jorge Lopez, Syncsort senior manager for data integration, says the sheer complexity of all the data sources involved in a modern application integration project are starting to overwhelm existing approaches to ETL.


Lopez says it will take a little longer before Syncsort can move its ETL engine into memory, but does say that IT organizations should start planning today for what he says will happen sooner than later. That means ETL technologies are something that IT organizations need to work around, and the next generation of ETL platforms will be technologies that IT organizations will find hard to keep up with.

 

At a time when data integration across multiple platforms residing both inside and outside the enterprise is increasingly becoming critical, advances in the way the whole ETL process is performed and managed can come none to soon.



Add Comment      Leave a comment on this blog post

Post a comment

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


 
Resource centers

Business Intelligence

Business performance information for strategic and operational decision-making

SOA

SOA uses interoperable services grouped around business processes to ease data integration

Data Warehousing

Data warehousing helps companies make sense of their operational data


Thanks for your registration, follow us on our social networks to keep up-to-date