The cloud has drawn substantial interest among enterprise users. Now, it appears high-performance computing (HPC) circles are starting to take notice.
On one level, it would seem HPC users like universities, research facilities and government agencies would jump at the opportunity for unlimited scale and dynamic load balancing. But when you consider that most of these institutions already operate computer clusters the size of which would make your head spin, perhaps they never saw a tremendous need for the cloud.
But that may be changing now that more of the cloud's side benefits are becoming well-known. One of those is management. As CTO Edge's Mike Vizard argued recently, most clusters are built around particular applications, which results in poorer performance if any one system or node runs into problems. As management technologies improve to the point that multiple apps start sharing the same cluster, they will start to look more and more like private clouds. It's almost as if cloud computing will come to the HPC market by default.
Cluster management firms are already honing in on accommodating increased scale requirements as HPC moves toward the clouds. A new shining star in the industry is Bright Computing, which has abandoned the traditional toolkit design in favor of a more integrated approach that is heavy on flexibility and scalability. As CEO Matthijs can Leeuwen told HPCWire this week, many users are starting to tap into cloud resources to burst capacity from existing clusters or to host entirely new clusters. Management systems, then, need to adapt to exascale environments that may involve tens of thousands of heterogeneous nodes.
The trend toward cloud-based HPC is already under way at some of the largest facilities. The University of Texas, for example, recently expanded its existing cluster architecture to create an 8000-processor private cloud to conduct cancer research. Part of the project involves a new 12,000-square-foot data center built around AMD technology and SOA-style Web portal access to handle upwards of three petabytes of data - enough to accommodate a range of compute-hungry processes like genome sequencing to trial simulations.
Some of the leading HPC vendors are tapping into the synergy between cluster and cloud technology, and are finding their way to entirely new customers. Amax, for one, already has a comfortable living providing HPC clusters to CalTech, DoD and others. Its most recent release is the CloudMax appliance, which provides private cloud functionality to small businesses for less than $65,000. The half-rack configuration (a full rack is on the way) provides seven, two-socket Xeon 5600-powered servers, rather than a highly dense blade configuration, which tends to be overkill for the SMB set. The device uses the ISF 2.1 cloud controller preloaded with XenServer 5.6, although it can also support ESX, Red Hat KVM and Hyper-V. Additional controller options are on the drawing board as well.
Like chocolate and peanut butter, HPC and the cloud will likely find they are better off together than apart. When it comes to massive scale and coordinated use of extreme levels of computing power, no one has more experience than HPC.