Archive for April, 2012
The Blade Myth and 10Gb Ethernet
Over the last several years there has been a big push to adopt blade servers. The idea being that you can cram more CPU cores into less space allowing you to build a more efficient data center. Lets take a few minutes and look at IBM BladeCenter vs 1U white label pizza boxes. The results may surprise you.
I picked IBM BladeCenter because it has the largest market share in the blade space and frankly is a great product! It is price competitive with other blades and I think gives a fair representation of the market.
The BladeCenter E chassis supports 14 bays in a high density chassis. At only 9U high you can support 4 chassis in a rack and still have 6U free to support two top of rack switches and any other support hardware. This provides a maximum density of 56 dual processor blades in a standard 42U rack.
Before we jump into the actual blades, lets next look at 1U chassis. In a standard 42U rack you can support 40 dual processor 1U servers, leaving room for two 1U top of rack switches. This is one of the biggest selling points for the blade camp, we are able to support many more raw CPUs with blades.
Our test systems are Dual Intel Xeon E5-2680 8 core 2.7 GHz CPUs, with dual 10Gb Ethernet, and 128 GB of RAM. The BladeCenter HS23 rings in at $11,713 (including 1/14th the BladeCenter H chassis) and the white label rings in at $5,882 based on Supermicro X9DRH-7TF motherboard.
Both of our systems come with 10 Gb Ethernet, however they are very different from factors. On the IBM we are handed older SFP+ connectors and on the white label we have newer copper 10GBase-T ports that support standard Cat-6 cable.
For aggregation, I chose Arista Networks 7050 series switches based on the Broadcom Tident+ ASICs. This switch supports 40 GbE ports and 4 QSFP+ 40 GbE ports on a 1.28 Terabits/sec fabric. The white label solutions is using the DCS-7050T-64-F ($20,995) with 40 10GBase-T ports and the BladeCenter requires the bit more expensive DCS-7050S-64-F ($24,995) with SFP+ ports.
Our white label box has 10GBase-T ports on the motherboard, however the BladeCenter requiress two Ethernet Pass-Through Module at $4,999 each and $75 twinax cables. Bringing our cost per port of 10 Gb Ethernet including switch at $328 for white label and $1,175 per port for IBM BladeCenter.
In order to compare our numbers in an easy way I decided to look at a per CPU core metric of cost per CPU core. With dual 8 core CPUs and dual 10 Gb Ethernet ports. The BladeCenter comes in at $834 and white label at $408.
But what about cost for space?
It turns out that cost for space does not effect the per core number that much. With BladeCenter at 896 cores and a cost per rack at $750, over two years the cost per core is only $22.77 compared to the 640 cores and $31.88 per core over 2 years cost on the white label. So even if we take 2x amount of space for our 1U white label servers we still come way way ahead using 1U pizza box servers saving almost $7K per server!
Distributed storage in today’s clouds
How does this fit with today’s cloud deployments? Well the interesting thing is that today many cloud deployments are using servers for more then just compute resources. Today 10Gb Ethernet controllers such as the Intel X540 are able to directly access CPU cache greatly reducing the processor and memory requirements of 10 Gb flows. This allows servers to also act as cloud storage using new distributed file systems such as Gluster and Hadoop.
Unlike blades, our 1U servers have plenty of room for disks. In fact, the Supermicro X9DRH-7TF motherboard already has a LSI SAS2208 hardware RAID controller supporting 6Gb/s SAS/SATA drives. With 8 2.5″ 1TB SATA drives one could at a low cost add 280TB of raw RAID5 storage per rack to the compute cloud.