Warehousing in the Cloud: Useful, But Try Not to Overdo It

Executive Insights for Building Trust in the Cloud

If two technologies have ever seen a timelier rise to prominence, it would have to be cloud computing and data warehousing. The symbiosis couldn’t be clearer: Big Data and analytics coupled with the ability to quickly and easily scale infrastructure to petabyte levels.

Acceptance of cloud technology is clearly entering the mainstream, and this seems to be one of the key drivers for the advanced analytics capabilities that warehousing provides. According to TechNavio, the global data warehousing market is set to grow by more than 11 percent per year between now and 2018, which is impressive considering that even with the cloud as a foundation, warehousing is still a complicated process that reaches into just about every legacy system and architecture in the enterprise.

Of course, the big boost came last month when Amazon introduced new tiers on its Redshift warehousing service that enables clients to tap into highly dense computing systems and advanced business intelligence software. The platform now provides compute blocks of 32 processing cores, 2.56TB of solid state storage and 244GB of RAM, all for about 25 cents per hour. These can scale into petabyte-class infrastructure at costs as low as $1,000 per TB per year, which Amazon says is about a tenth of what an equivalent on-premise warehousing solution would run.

The cloud is clearly an asset to warehousing, but before you jump in blindly, it would help to assess your true needs first, said IBM’s Adam Ronthal. A realistic deployment for most enterprises is about 100TB, and while it may be tempting to support cloud warehousing with full 10GbE connectivity, a 1GbE link will still transfer about 450GB per hour at a fraction of the cost. You’ll also need to determine whether full EDW is necessary in the cloud or just select applications, and if it’s the latter, what sort of backward compatibility will you need to support legacy applications at home?

Perhaps ironically, it seems that increased use of the cloud for general-purpose applications is driving the need for cloud-scale warehousing footprints, says Teradata’s Greg Taranto. As most CIOs are aware by now, application silos in the cloud are becoming just as nettlesome as data silos in the enterprise, so in order to maintain a single view of business operations, warehousing needs to migrate to more agile and scalable infrastructure. And it doesn’t take a genius to understand that broad, standards-based platforms will be needed to maintain functionality across increasingly distributed data environments.

Warehousing and the cloud, then, are two of the proverbial “great tastes that taste great together,” but their union is by no means a slam dunk for every occasion. Remember that cloud-based data and analytics may still be subject to compliance and discovery, which can get tricky in the cloud, and that even the most robust warehousing platform is only as good as the value of the intelligence it produces.

The cloud is great for providing scale, but functionality still rests with the enterprise.