Cloud PCs can’t be benchmarked, so don’t even try, Microsoft says

A senior Microsoft executive has laid out why benchmarking cloud PCs is an incredibly tricky process that, in many cases, just isn’t possible.

In a recent blog post, Microsoft Windows 365 Cloud PC and Azure Virtual Desktop Senior Product Manager, Ron Martinsen explained why it’s nigh on impossible to benchmark cloud computing owing to the huge amount of variables.

Instead, we should just trust that continuous upgrades provide us with optimal performance.

Comparing cloud computers is difficult

Martinsen drew on almost three decades’ experience and “countless performance studies” as he explained how, “it’s nearly impossible to get repeatable data in an environment that reflects the reality of what users will be using.”

He even goes on to explain that repeat tests may deliver different results and that making valid comparisons can be challenging at best. With that advice, it’s best to opt for virtual hardware that meets or exceeds requirements rather than trying to compare figures that fail to represent reality.

In his blog post, Martinsen explains that the operating system, network latency, disk controller caching, and memory optimization are some of the most important factors to consider when trying to benchmark.

Put that into the context of cloud computing “where the host compute is running on a cluster and serving a variable number of active Cloud PCs” and the number of variables becomes “overwhelming” – and even more so for virtualization workloads like Windows Subsystem for Linux.

Then, there is the fact that the virtual desktop you accessed one year ago will likely have been updated and upgraded in the following months, helping it to achieve improved performance and/or efficiency.

Moving forward, even outside of the context of cloud computing, Martinsen criticizes benchmarking tests for being carried out in sterile environments whereby the best possible figures are achieved – not relative figures that apply to specific use cases.

To conclude, while it’s likely that higher benchmark figures will result in improved performance for the end user, the difference will almost never be so stark. As for cloud computing, Microsoft’s relative performance indications serve as a basis for comparison.

• Check out our pick of the best productivity tools