The use of commodity network devices could both improve data center performance and lower costs, according to a group of computer science professors at UC San Diego.
In a paper submitted to the Special Interest Group on Data Communications (SIGCOMM), professor Amin Vahdat and his colleagues argue that the traditional "tree and branch" style of networking that uses specialized router and switching technology supports only a tiny fraction of the combined bandwidth available to end hosts. Not only is this expensive, but it limits cluster sizes and results in non-uniform bandwidth availability at the node.
A much better approach, they say, is to foster scalable interconnection bandwidth so that arbitrary hosts in the data center can communicate with all other hosts at the full bandwidth of the local network interface. You'll have to alter your end-host network interface, as well as the OS and applications, but the approach would be fully compatible with Ethernet, IT and TCP networks, and you can swap all those expensive switches for off-the-shelf commodity devices.
Think of it along the same lines as the PC. Does it make more sense give each worker one commodity desktop, or have the entire workforce using a single, giant supercomputer?