Typically, compromise can be a great thing – certainly a mandate in most relationships, but not in the world of IT, and especially not in the storage space. In IT, compromise always comes at a cost. For years, IT has been inventing and implementing workarounds to cope with the poor performance of storage systems bound by mechanical hard disks. As processor, memory and network speeds have repeatedly doubled over the years, storage media have failed to keep pace, creating unbalanced ecosystems.
Often, these compromises are so ingrained that they have become second nature. No thought is given to the overhead because no alternatives existed. That is, until now, with the proliferation of flash-based storage systems and the trajectory of the all-silicon data center.
Fahima Zahir, product marketing, Violin Memory, has identified the top 10 coping mechanisms found in today’s storage environments that should be avoided.
Click through for the top 10 coping mechanisms found in today’s storage environments that should be avoided, as identified by Fahima Zahir, product marketing, Violin Memory.
The reality is that many legacy storage systems remain largely unused due to performance degradation if used at full capacity. At the heart of the problem lies the fact that there exists an imbalance between disk capacity and the number of I/O operations it can service per second. Over-provisioning on outdated, mechanical disk translates into storage waste as well as data center sprawl and its associated power, cooling and maintenance costs.
Legacy disk technology has physical constraints to read/write to a sector on the disk; the disk head first physically travels to the correct track and then the disk must rotate to the correct sector. With short stroking, only the outer edge of the disk platter is used, reducing the transit time of the head and increasing the throughput. In short, there is less space to cover, and hence faster reads and writes. The compromise is that between 75 percent and 90 percent of the disk capacity is wasted just to remove one or two milliseconds of latency.
Input/output (I/O) scheduling is fairly self-explanatory, referring to the concept of the OS using an I/O scheduler to manage the order in which I/O operations are submitted to the storage layer. These are complex algorithms that re-order I/O operations so that the movement of disk heads (known as a “seek”) is reduced. When you direct CPU cycles to “seek time” reduction, the result is a direct hit on performance.
Many legacy storage vendors mask the slow performance of their storage systems with the use of caches of DRAM. This results in hit or miss performance, allowing some “lucky” I/O operations to complete with faster response times, a “cache hit.” However, “unlucky” operations whose data is not found within the cache experience the extra penalty of a “cache miss:” looking unsuccessfully for the data and then having to take the additional hit of searching again on the underlying disk storage. Performance becomes unpredictable, a problem exacerbated by operations such as backups and batch jobs, which clear the cache of useful data.
Automatic tiering software attempts to migrate data depending on its level of use, with “hot” data on the faster tiers. The issue with this tactic is that past behavior is not indicative of the future. Data that may not be used for weeks at a time (like month-end reporting tables) ends up sitting on slow-tiered storage when it is needed. Once used, it moves to faster-tiered storage, replacing the data that may be used on a daily basis. The result is that any irregular usage pattern can have a vast and detrimental ongoing effect on performance.
An example from the database world is creating data files on faster storage in order to contain active, quarterly partitions of an accounting or sales table, with older quarters being archived to slower storage. Other examples of manually distributing data include the placement of transaction logs on alternating groups of spindles and the use of SSD or PCIe flash cards for temporary tablespaces. Manual distribution carries not only the challenges of complexity and error, but also the overhead of constant monitoring to ensure the configuration remains optimal.
In the database world, indexes are often the biggest challenge because by nature, index reads are random. A common tactic is to over-index tables in order to reduce the number of blocks that must be read from storage. This behavior can span the extremes. At one extreme is the method of creating an index containing all of the required columns needed by a query, so that the index itself supplies the data and no table access is necessary. Clever, but the overhead required to maintain numerous indexes is significant and has an impact on any insert, update or delete operation.
Many database systems use summary tables to calculate point-in-time snapshots of aggregated information. An example is the use of a “materialized view” to show the total and average invoice values on a billing system at the end of a business day. Due to the volume of data in the invoice table, a legacy storage system would not be able to continuously calculate these values as each user requires them, so a “snapshot” of the values is calculated only at certain times of the day.
Many organizations use reporting or decision-support databases to offload reporting overhead. Often these databases have data “bulk loaded” from the live database so that multiple aggregation and calculation routines can be run without affecting live operations. In many cases, the additional license cost of running these databases is substantial, but the real compromise is often the stale data. By the time the data is loaded, it may already be out of date. To address the stale data challenge, you can synchronize the data in real time through other solutions. But now, you have increased licensing costs and have added more complexity to the environment.
How many different combinations of RAID options do you need to consider when designing a storage solution? RAID level 0,1,4,5,6 or 10? What about the drive type? Do you need SAS, SATA, Fibre-Channel or SSD? What would be the best stripe size for this system? In a database environment, would you use the same design for all file-types, or would you need to design separate solutions for the data files, the transaction logs, the temporary files and the backups? How much time needs to be spent looking for the best solution? Let’s stop there. That should sufficiently illustrate the point: RAID is complex.