The Challenges and Opportunities of External Data

Loraine Lawson

One of the more exciting business developments these days is the potential for buying and sharing data via the cloud. I'm not talking about your product catalog, I'm talking about obtaining data that could strategically help reshape business and government.


The most exciting, of course, is Linked Data, a W3C project for adding information to the Web that's growing every day. (Here's a fun interactive image of some of the available data sets.)


But there's a more immediate and available path organizations can take to supplement their business intelligence and other initiatives with rich data from external sources sold as commercial Web data services.


Commercial Web data services aren't particularly new. The financial services sector has known about and used them for some time, but they're quickly moving into other sectors as more organizations become adept at consuming Web services and data in general through better BI tools. Users see the information online, and they want to be able to use it with their internal reporting tools, as Stefan Andreasen, co-founder and CTO at Kapow Technologies, told Dana Gardner, principal analyst with Interarbor Solutions, during a podcast on Web data services last year.


Web data services are a way of doing that. But while they're more readily available and accessible than ever before, there are complications that come with this data, as a recent TechTarget article points out. Integration, data quality, latency and even your internal applications' ability to handle the volume of data are all issues IT has to consider before subscribing to external Web data services.


It's a seldom-discussed topic that I think will quickly become more of an issue, and, frankly, this is an unusually thorough discussion on such a new topic. Basically, it's like a checklist for all the issues, including what you should consider before building an integration layer, how an ESB can help, and why it's critical to test scalability when integrating commercial Web data services with your enterprise applications.


The only thing I didn't see discussed is how companies can manage changing APIs, which can affect integration. But I believe that's because if you follow the suggestions of Accenture integration architect Fawaad Khan, you'll mitigate through your integration architecture, which he says should provide enough isolation to handle availability problems.


There's also a discussion of which technologies and niche vendors can help you handle the data sets, which can be huge and complex.


If the idea of commercial Web data services is new to you and you'd like to read more discussion on its potential, check out Gardner's four-part podcast series, which includes transcripts. It's sponsored by Kapow Technologies, but it's a broader discussion of the topic with a variety of experts. Part one talks about the proliferation of available information, while part two looks at actually getting the information into your organization and using it for BI. Part three looks more at text-based content access and management issues for real-time BI, while the final installment moves from theory to how one company -the Deutsche Brse Group in Frankfurt, Germany-uses Web data services.

Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.