For the last two years everybody has been obsessed with the cost of IT and getting more utilization out of their servers. As a result, there hasn't been much of a focus on performance.
But even though the financial services sector came pretty close to ruin, the battle over who can build the fastest systems continues. And in that battle, Linux has quietly become the preferred platform for extremely high-performance applications.
Deutsche Borse, which runs the International Securities Exchange in New York and the Eurex and Xetra Exchanges in Europe, is laying claim to having the fastest trading system in the world. Capable of executing more than a million trades per second, the system will be built on top of IBM's Websphere MQ Low Latency Messaging middleware and Linux servers running on Intel processors when it is deployed late next year.
The Deutsche Borse effort is significant on a couple of levels beyond the bragging rights it gives IBM. First, it shows that Linux is rapidly becoming the de facto platform of choice for high-performance applications. It also shows the continuing importance of intellectual capital and custom application development in the financial services sector. And because of the way the Deutsche Borse application was built, it shows the rise of cache memory as a critical new tool in the drive to maximize application performance.
For the Linux community, the Deutsche Borse system is nothing short of vindication. It shows that a community-based software platform is capable of keeping pace with anything a commercial vendor can throw at them.
The Deutsche Borse system should also serve to caution politicians such as Sen. Chuck Shumer (D-N.Y) who are calling for regulation that could limit the speed at which trades can be made. It's true that companies with deep pockets can develop an advantage using new technology to execute trades faster. That was also true when the telegraph was invented. What is also true is that new technology becomes widely distributed quickly, so it tends to only provide a momentary advantage. We should make sure that there is a level playing field when it comes to information, but we need to make sure that such regulation is not overly broad. Exchanges in the U.S. can't sit idly by while overseas rivals bypass them in the technology arms race because of some well-meaning, but short-sighted, piece of legislation. In fact, there is no guarantee that the Deutsche Borse system will still actually be the fastest by the time it deploys next year.
Finally, the Deutsche Borse system also highlights the diminished role of the database. To achieve these application speeds, Deutsche Borse executes all the transaction in cache memory. After the fact, those transactions are recorded in a database. But the database plays no part in achieving the overall performance of the system because the I/O speeds of storage systems are too slow to keep pace with the needs of the system.
To one degree or another, every vertical industry is involved in an application performance arms race with their rivals. They may never need to execute a million transactions per second. But they still need access to the fastest technology available at a reasonable price. With the rise of Linux and cache memory, costs are dropping sharply. So soon the only real limiting factor when it comes to performance is going to be the innate talent of your development teams.