Big Data Breaks the Back of Existing Storage Architectures

There’s a school of thought out there that suggests that with the advent of Big Data, existing storage systems are getting the math wrong. This doesn’t refer just to the cost of that storage, but also the actual algorithms used to store all that data.

The basic theory says that standard RAID lacks the scalability to affordably manage petabytes of data. The core issue comes down to a broad family of algorithms known as erasure codes. Proponents of this theory argue that existing erasure codes used in RAID systems were never designed to cost-effectively manage Big Data at scale without compromising performance.

As such, startups such as Amplidata are making the case for a new object-based storage system, called AmpliStor Optimized Object Storage. It makes use of a more modern approach to erasure code known as an Amplidata BitSpread algorithm that is embedded in Amplidata controllers and spreads bits of data across multiple storage nodes. Those nodes are in turn connected to applications via HTTP using a relatively simple REST interface that from an application perspective reduces the complexity of deploying an object-based storage system.

To help make that argument, Amplidata today not only announced the appointment of a new CEO, it also revealed that is has gained an additional $6 million in funding. A storage industry veteran that worked at Intel, newly appointed Amplidata CEO Mike Wall knows he has his work cut out for him. After all, storage administrators are notoriously conservative when it comes to new and emerging technologies.

But Wall contends that the rise of Big Data is already making it apparent that most existing RAID systems are fundamentally flawed. Wall argues that most storage administrators are going to be a lot more open to innovation because Big Data is going to force the issue, which he adds Amplidata now has the appropriate level of resources to begin to address.