Parallels Desktop benchmarked

Just recently I wrote about virtualization benchmarking. My point was that a good benchmark has to

• Be based on real-world use cases
  
• Be vendor and technology neutral
  

VMmark was certainly neither – based on an artificial workload mix and designed to work best with ESX and not work at all with Virtuozzo.

CNET recently did a performance comparison between Parallels Desktop and VMware Fusion. Surprisingly enough, their benchmark also suffered from unrealistic scenarios and vendor bias.

Pretty much everything about this test is upside down:

• It does not make sense to use an exclusive monster 8-core desktop for benchmarking Windows on Mac. Most people use laptops with 2 CPU cores and 2GB of RAM at most.
  
• Since Mac OS is the primary OS, it does not make sense to give both cores to a Windows VM that runs Word, Excel and Outlook. Office applications don't benefit from multiple CPUs. Perhaps that's why default configuration of Fusion is a single-CPU VM. Come on – most of the VMmark (server benchmark) workloads run in a single-core VM. Yet, desktop benchmark is run with dual cores – does not make much sense to me.
  
• It does not make sense to use Vista inside a VM. Most people run XP because Vista license only allows the most expensive Vista SKUs to run inside VM, not to mention application compatibility issues.
  
• It does not make sense to run QuickTime and Photoshop inside Windows VM. Why use Mac OS in the first place if not for running Mac OS-native multimedia apps?
  

Actually, some people noted that the benchmark looked strange. Some even took time to run some tests for themselves.

The results show that Parallels actually performs better than Fusion on most tests and on the overall benchmark. And, as you already know from CNET test, Fusion could not even run the 3D gaming test. Yet, I'm sure my colleagues at Parallels are already working hard and the next Desktop update will improve responsiveness and performance so that even artificial tests will pass wellJ

Finally, I wanted to remind you that Parallels has always been focused on the real-world productivity. It is the time wasted on extra clicks, not raw speed that affects productivity the most. And it is the real-world scenarios – running real software applications on real hardware configurations – that need to be reflected in the benchmarks.