This report looks at roughly 300 million individual tests of nine cloud platforms, conducted over a seven-day period. Cedexis, a company that measures Internet health through browser instrumentation in order to optimize its customers’ use of clouds and CDNs, shared the data with us.
This is dense stuff; we’re distributing it under a Creative Commons license, and we ask that you attribute us when you re-use it. In addition to data about cloud performance from almost every country and service provider, the report also shows which clouds perform best from which regions.
Some quick takeaways:
- Regional effects matter tremendously, and the difference between countries can be significant.
- Directing a regional client to the correct regionalized zone for a cloud provider does improve performance.
- The average time to complete an HTTP request and receive a response, worldwide, was 426.4 milliseconds. The average availability worldwide was 97.69%.
- While most cloud providers had roughly similar average availability, a closer analysis of percentiles shows that the worst-served visitors fared very differently. This underscores the importance of doing proper analysis on performance data.
- End user information is an important complement to synthetic testing. While the results aren’t as consistent, they provide insight into the conditions of far-flung end users across a broad spectrum of networks and countries.
- Given variance in cloud performance and availability by region and by day, it makes sense for serious cloud users to hedge their bets, and find ways to arbitrate cost and service quality across providers.
As with other studies we’ve published, this one was sponsored by Cedexis, who also gave us access to their raw data. They didn’t have any editorial input into the content; as with other firms with whom we’ve published research—Citrix, Coradiant, and Neustar Webmetrics—Cedexis is a proponent of sharing data that advances the understanding of emerging technologies. We’re big fans of this approach to marketing, and thank them for their support.
SIDENOTE: When we first published this study on March 3, 2011, we let Google know that we’d calculated a significantly lower percentage availability for Google App Engine than other cloud providers. We let the folks at Google know, and the App Engine team looked into the situation. They helped us investigate the problem, which initially seemed like a billing problem. The team worked closely with Bitcurrent and Cedexis, and we’ve updated our study with a detailed analysis of that discussion. The short version: there’s a big difference between what App Engine sees as availability, and what end users see; and that experience is significantly worse for certain countries.
