It seems that every server vendor has their own way of measuring power/performance ratios, usually to the benefit of their own systems.
But now, an industry group has established a set of benchmarks that it says offers a more meaningful look at the true capabilities of server technology. The SPECpower_ssj2008 standard aims to take into considering things like application workloads, form factors, processor speed and memory capabilities to ensure that comparisons are being made on a level playing field. The old apples-to-apples saying comes to mind.
The SPEC (Standard Performance Evaluation Corp.) standard has a number of tricks up its sleeve. It measures power consumption at various performance levels to gauge results over a normal time period, rather than merely providing a snapshot view. The number of transactions at a given level are compared against energy consumption to provide a watts-per-transaction measurement, similar to a car's MPG.
Workloads are standardized around server-side Java applications so as to represent the most common business environments in terms of scalability, portability and multi-threaded capabilities.
Like all industry standards, this one won't amount to much unless it sees widespread adoption. But with big names like AMD, Dell, HP, Intel, IBM and Sun involved here, there's a good chance that SPEC is a keeper.