Bullshit Vendor Whitepapers...

Mike Laverick over at RTFM (link in my linkage section) has made a post regarding a Dell whitepaper that has just been released, that purports to compare "2 socket vs 4 socket hardware", specifically the Dell 2950 vs the HP DL585 G2. As a longtime reader of Mike's blog and someone with enormous respect for his opinions and contribution to the VMware community, Mike - I'm severely disappointed in you for simply reporting this as if you were on Dell's payroll. These kind of whitepapers prove absolutely fuck all, and are nothing short of pure marketing bullshit.

I guess anyone reading the title of the paper, the 'Executive Summary', the 'Introduction' and then glossing over the actual hardware specs used in the testing and going straight to the results numbers would reach the same conclusion... 3 x 2 socket 2950's show more grunt for less cash than 2 x 4 socket DL585 G2's.

First, comparing retail price is useless - anyone who has worked in a large enterprise knows how close Dell, HP and IBM really are in terms of price. Second, they are comparing Intel in the 2950 vs AMD in the 585 G2. But third and worst of all, the 2 socket Dell boxes are loaded up with quad core CPU's, where the 4 socket HP's only have dual core CPU's!!! Gee, what an 'Apples vs Apples' comparison!

I'd probably be less annoyed if the authors didn't go at such lengths to hide what is actually being compared. Count how many times they refer to sockets, and how many times they refer to cores. Look at where and how they actually say "we had 4 cores per socket in our stuff and only 2 cores per socket in their stuff". How about this direct quote

... this paper compares three two-socket Dell™ PowerEdge™ 2950s with two four-socket Hewlett-Packard™ ProLiant DL585 G2s. Or more simply – 6 sockets vs 8 sockets.

What, saying 24 cores vs 16 cores instead of 6 sockets vs 8 sockets is more complex? Oh, wait... no, that would screw their results.

And that's the funny part - going on cores, the performance numbers actually work against Dell. I quote:

...these tests show that three PowerEdge 2950 two-socket servers can provide up to 44% more performance...than two HP ProLiant DL585
G2 four-socket servers.

So with 50% more cores than the HP hardware, Dell gets 44% more perfomance... ummm, either my math sucks, or your boxes suck Dell.

*UPDATE* Good pickup by Kharbin over on the VMTN forums... the Dell boxes also have half the RAM of the HP boxes, if you bring them up to the same level of RAM, the prices are pretty much the same too!

OK, maybe that's being a little harsh, and I know that Dell are not the only offenders when it comes to psuedo-research papers, but this kind of thing really pisses me off. There are so many permutations that would have made this whitepaper meaningful. Why didn't they use Dell 6950's instead of HP 585's? Why didn't they at least use a HP DL580 G4 (or even better a Dell 6850) thus making it an Intel vs Intel comparison? Why didn't they use an equal amount of cores for comaprison (ie 2 dual socket quad core Dell's instead of 3)? How would 4 single socket quad cores stack up against 2 dual socket quad cores? You get the idea.

The whole scale up vs scale out question with regards to VMware is something we'd all really like to see some solid numbers on, but to date I don't think any hardware vendor has done the work. Maybe the fact that a 4 socket box is often more than twice as expensive as 2 x 2 socket boxes has something to do with that. The hardware manufacturers are already staring down the barrel of severely diminishing sales thanks to virtualisation, better for them to get the higher margins on 4 socket boxes I imagine. So rather than Dell shoot themselves in the foot by saying 3 x 2 socket Dells are better than 2 x 4 socket Dells (or god forbid, 2 x 2 socket quad core Dells are roughly equivalent in performance to 2 x 4 socket dual core Dells, but the 2 x 2 socket boxes are 200% cheaper), I guess the only choice these pseudo-researchers had was to compare them to another company's 4 socket product. And I guess Dell is really hurting since HP knocked them off the #1 PC seller spot.

I can only hope that HP retort when the quad core Opteron's are released. No doubt Dell would cry foul if that happened though, so I'm hoping for at least a Dell 2950 with Intel quad cores vs a HP DL385 with AMD quad cores (the 385 is HP's 2 socket AMD box for those who aren't familiar with HP). I'm sure you can guess what my money will be on.

And just to be clear, I'm not arguing that 'few big boxes' is better than 'many small boxes' (what design you use depends on the risk aversion of the stakeholders / customer more than anything) - all I'm saying is that this paper needs to called out for what it is... another BVW &trade