The Need for BI Speed

Michael Vizard

There are a lot of data warehouses out there that can no longer keep up with the pace of the business.

Most data warehouses were built on the assumption that performance would hardly be an issue. The business side wasn't expected to stress the platform by making too many queries, and just about all the queries were of the canned variety.

But today, people use the data warehouse more often as they discover the power of a wide variety of business intelligence applications and they want answers quickly to all kinds of complex queries.

This need for BI speed comes as more and more decisions are being made in relatively real time. But rather than just making gut decisions, companies are rightfully insisting that managers have some actual data to back up those decisions. That means people increasingly are turning to the IT department for analytics.

This fundamental shift in the way data is used creates the need for faster data warehouses. This is why you see traditional data warehouse providers such as Hewlett-Packard upgrading the performance capabilities of its Neoview data warehouse, while relative newcomers such as Aster Data and ParAccel strive to carve out a piece of the market based on their ability to quickly process large amounts of data.

All this interest and activity makes what was once a sleepy back-office corner of the IT department suddenly one of the best places to be in terms of adding value to the business. Most businesses are never going to operate in real time, but the gap between production systems and the data warehouse is narrowing considerably. The only real issue is whether the existing data warehouse can really keep up with it.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post
Nov 9, 2009 1:32 PM Dave Schrader Dave Schrader  says:

Teradata saw the evolution you cite in the numbers of users and the types of queries about 5 years ago. We coined the term "Active Enterprise Intelligence" to cover extend the traditional BI-centric uses of data warehousing to the newer connect-your-front-end-systems-and-people-to-the-database opportunity, which we call "operational intelligence".  This user base is not BI-centric; in fact, most of the time the insights from the traditional back office, e.g., Next Best Offer, are embedded in web portlets for the web site or contact centers, thereby augmenting the front-end experience.   As a consequence of this shift, we invested in TASM, Teradata Active Systems Management, a workload priority scheduler that allows DBAs to provide the highest priority to these front-end mission-critical workloads.  And we've been doing this for years.

For example, Travelocity uses Teradata to not only run 300,000-400,000 traditional BI queries per day, but also run 1.5M website SOA callouts to Teradata from their websites to show customized air, cruise, rental car, and hotel content, based on previous webclick patterns. There is a strong SLA 20-40ms response time requirement on these queries, so they get the highest priority.  But the new workload consumes well under 1% of the system, hardly a performance issue.  China Post does the same with their system, adding up to 70,000 "where is my package" queries/hr,  plus a new trickle-fed workload of 6M records/day on package status, again all running othe same system that handles their 240,000 daily BI queries.  Insurance carrier Unum added frontline queries from their contact center IVR and agent screens, plus their 3 web portals,  to an existing BI workload - again all on the same system.  They reported that the new combined utilization is 45% ETL, 35% ad-hoc BI queries, and 25% to support the contact center and web workload - a total of 10M queries/month, surging sometimes to 18M queries/month.

So while some data warehouses may suffer from the problems you cite, Teradata is not one of them.  Bring the workloads on!

For more information about these and other case studies, browse

1.     Teradata Magazine: Pervasive BI completes the picture, Shirley Savage, June 2008 (Travelocity sidebar - http://www.teradata.com/tdmo/v08n02/pdf/AR5613.pdf),

2.     Teradata Magazine: Speedy delivery - China Courier Service Corporation gets a boost with active data warehousing, Dec 2007 (http://www.teradata.com/tdmo/v07n04/Features/CaseStudies/SpeedyDelivery.aspx); and

3.     Teradata Magazine, "Integrated insight: how Unum Group grows its data warehouse into an active platform," Sept 2007(http://www.teradata.com/tdmo/v07n03/Features/CaseStudies/IntegratedInsight.aspx )


Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.