The cloud is widely recognized as the answer to infrastructure and resource limitations in the age of Big Data and the advent of mobile computing. But this offers only a limited view of the technology's true potential.
Far from simply providing a means to keep your head above water, the cloud represents the best chance in a generation to truly remake data operations from the ground up. In so doing, it gives the enterprise not just more of the same, but more of what you never thought you needed in the first place.
A key example of this is the increasing availability of high-performance computing (HPC) services on the cloud. For decades, smaller organizations have looked at the supercomputing capabilities of large government agencies and top research facilities and dreamed of the possibilities. With infrastructure constraints and capital budgets being what they are, however, the dreams went unfulfilled.
In the cloud, however, resources are ripe for the taking, and costs are manageable because they can be portioned out according to need. Amazon's EC2 cloud, for instance, can be spun up to 50,000 cores, backed by a highly parallel networking architecture built around a 10 GbE fabric, which the company says is adequate for 99 percent of HPC jobs today. Typical deployments consist of a pair of eight-core Xeon E5-2670 processors, 60.5 GB of memory and 3.3 TB of storage, although the company can also deliver paired quad-core Nehalem instances as well.
One sign that a new market is taking off is whether it is drawing interest on the Internet. In the case of HPC on the cloud, the movement has already started with websites like, well, hpcinthecloud.com. The site features a broad range of coverage, spanning topics like XaaS (Anything as a Service) to cooperative, multi-provider services for higher education users and other clients. The site features top HPC and cloud experts like ISC Cloud Conference Chairman Wolfgang Gentzsch, Dr. Jose Luis Vazquez-Poletti of the University of Madrid, and veteran HPC and cloud analyst Steve Campbell.
Still, HPC may be necessary for extreme projects like gene sequencing and climate modeling, but will the typical enterprise need such computational power for its day-to-day operations? Perhaps, although they might not know it yet. As Intel noted in a recent Techworld editorial, functions like real-time business process monitoring, product development simulation and customer/market behavioral mapping and analysis could significantly enhance bottom lines, although most enterprises probably don't realize it because they haven't had access to the infrastructure needed to support such applications, until now.
Indeed, opening new minds and new business opportunities is one of the reasons many top HPC firms have been so aggressive in developing cloud-based platforms of late. Bright Computing, for example, has introduced a cloud bursting component to its Bright Cluster Manager platform as a means to supplement private infrastructure running HPC computations. The system works with Amazon instances to provide a single control platform across internal and cloud resources, alleviating the need to build and maintain large local infrastructure in order to support occasional HPC projects.
We've seen this all before. Once one organization demonstrates the profit potential or cost-benefit advantages of HPC in the cloud, the mad rush begins to get in on The Next Big Thing.