Accelerate Data Virtualization with Data Services

Ciaran Dynes

Instead of physically moving and transforming data, data virtualization aggregates data from disparate sources to create a logical, single view of information for use by front-end solutions. Data services accelerate and streamline data virtualization, providing organizations with a complete view of the information, when they need it.

Leveraging Data

More than ever, organizations are focused on leveraging their data to get more value from it. They want fast, real-time and reliable information to better understand their business and make better decisions, while IT wants to lower costs, minimize complexity and improve operational efficiency.

However, the typical enterprise today runs multiple types of information management systems, on-premises or hosted, in databases or applications, which weren't necessarily designed to play well together. Add to this the fact that enterprises are seeing an unprecedented rise in the amount of data they store and you've got one heck of a data integration challenge on your hands.

Replication Impedes Data Freshness

Historically, the traditional way to solve the data consolidation challenge has been simple — take all the data from all the sources and replicate it into an operational data store. While this method works on some level, it’s incredibly inefficient, because you end up using just a fraction of the data that is replicated. In addition, the information you get quickly becomes obsolete. Indeed, when replicating data, it must be stable until the next replication period — however, in some industries, the "shelf life" of data can be measured in minutes or even seconds. Common examples could include flight availability or the price of commodities; the minute you put in the information to replicate it, it has lost its freshness.

Data Virtualization: A Single Abstract Layer for Accessing Data

This growing demand to access current data for agile decision making requires that organizations change the way data is accessed and used by business users. The solution is to give users a well-defined interface to the data they need, regardless of where the data is stored or how it is organized. Data virtualization is a set of techniques and technologies that provide the key underpinnings to enable organizations to do this.

Wikipedia has a good definition of data virtualization:

To integrate data from multiple, disparate sources — anywhere across the extended enterprise — in a unified, logically virtualized manner for consumption by nearly any front-end business solution, including portals, reports, applications, search and more.

Data virtualization is used to integrate data from multiple disparate sources that may exist within the enterprise, outside the enterprise, or in the cloud, to provide a unified (but virtual) view of the data for use by any number of data consumers. This provides those who leverage data virtualization with the ability to fully leverage the database resources where they exist, and leverage them as if they were one physical database, without having to drive changes to the physical data, or to move it.

Data Services for Data Virtualization

One of the approaches to data virtualization is to abstract heterogeneous data sources via a single data services layer that can be invoked in real time, near-real time or polled, to support multiple applications and processes that require access to underlying data sources.

For example, consider a media organization that delivers content to its customers on demand, 24×7, and is handling hundreds of simultaneous requests during peak load. This organization uses a SaaS CRM (Salesforce.com), a home-grown customer license management database running on Oracle, and SAP for invoicing and accounts receivables. Yet, a complete view of the customer is required by applications such as the customer Web portal (where customers can retrieve licenses they have purchased, but only if their account is in good standing), the CRM itself (where sales reps need to see invoicing and payment history), the business intelligence system, and more.


In short, getting the complete view of a customer requires access, at the same time, to all three systems listed above. How can data services help?

  • Data integration jobs and tools provide access to the sources through native protocols and APIs, often without the need to code. Data transformation, reconciliation and enrichment can be defined graphically and results are delivered regardless of where the underlying data is physically located.
  • These data integration jobs are then exposed as Web services, using SOAP or REST, and made available as a common services layer throughout the organization via an ESB.
  • Instantly, any application that needs to access a complete view of the customer can invoke the data services and retrieve, upon request and in real time, the consolidated information required, as simply as if it were residing in a single data source.


The advantage of leveraging data services is that they may easily be leveraged by any software system that can call Web services. Of course, to make this process smooth, complete integration between data integration and the ESB is required. This approach enables automatic deployment, metadata compatibility and ensures proper version control of the services.

Data services — simplifying your integration challenges.



Add Comment      Leave a comment on this blog post
Aug 13, 2012 7:29 AM Database testing tools Database testing tools  says:
Interesting article - given the amount of fake yet realistic data required for system testing, virtualized database testing tools surely must be the way to go. Reply

Post a comment

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


 
Resource centers

Business Intelligence

Business performance information for strategic and operational decision-making

SOA

SOA uses interoperable services grouped around business processes to ease data integration

Data Warehousing

Data warehousing helps companies make sense of their operational data