Storage clusters are getting more powerful and more expansive by the day, which is making it more of a challenge not only to manage data once it is stored but to find and retrieve it when the need arises.
The latest clusters are using things like advanced IP networking to take advantage of the fact that block-level virtualization is eliminating the need for traditional storage and RAID controllers. Start-up Pivot3, for example, has come out with the RAIGET (RAID Across Independent Gigabit Ethernet), which uses IP connectivity between clusters to boost performance five-fold, compared to traditional RAID, at about half the cost.
Microway, meanwhile, is trying a similar approach, except its network fabric of choice is InfiniBand. The company's ServaStor system features a load distribution technology that reduces contention between nodes, boosts overall bandwidth availability, and allows one cluster to serve hundreds or even thousands of nodes.
This is all well and good for power and performance, but there's a real dearth of storage resource management solutions (SRM) that can adequately deal with clusters, according to this article in InfoStor. The problem is the tendency of most systems to treat each host as a separate storage entity, rather than one aspect of a shared array.
Still, a number of new tools are coming out specifically for storage clusters. Exanet recently introduced the ExaSearch search engine aimed at clustered NAS environments. Part of the ExaStor storage management system, ExaSearch hunts through multiple sources, including file servers and e-mail systems, for structured and unstructured data and queries. Real-time indexing also provides access to the most recent and/or relevant hits.
Too often, the focus on storage expansion centers around capacity, and clusters certainly are a great way to scale up available storage as needs increase. But without a robust system of managing, maintaining and, most of all, finding and retrieving data, even the most state-of-the-art cluster is nothing more than a cyber trashcan.