Editors note: Recently we wrote about caching plugins, diving into the what and why. At the time we also noted our intention to do some performance benchmarking of popular caching plugins as a follow up, to identify the best caching plugin based on real performance data. Philip Blomsterberg, manager of Intripid Sökmotoroptimering, read our post and reached out offering to undertake this performance benchmarking as it was something he too had been considering. His plan of attack sounded really good so we happily accepted. What follows are the results of Philips benchmarking project.
- About the tests
- Benchmarking tools
- Google Page Speed, GTMetrix & YSlow Results
- ApacheBench Results
- Pingdom Results
- Webwait Results
- Server Performance Factors
- Overall Conclusion and the Winning Plugins
While developing my own site, we naturally wanted to achieve the lowest loading time possible, and pretty much made it a sport gaining just a few tenths of seconds in loading time. We knew how important caching was, but discovered that there were many ways to deceive yourself just by looking at the speed metrics. During the tests, we, of course, looked at all the different caching plugins out there, and after reading Charles' post I decided to make a more thorough test of their performance.
Quick comments about Caching
Speed is becoming increasingly important in Search Engine Optimization, conversion and user experience. Today, site speed is one of the most important ranking factors on Google. Google recently declared that all sites that are mobile friendly (and speed is a way of being “friendly”) will have a significant advantage in the SERPs, starting from the 21st of April. You might have already noticed the “mobile friendly” tag in the search results. Also, when using Google Page Insights, the pane which you are presented with first is the mobile one – not the one for desktops. Google’s intentions are clear and speaks volumes for any SEO or webmaster. I point this out to emphazise that we should be concerned with the performance of both the desktop and mobile versions of our site and we’ve tried to reflect this in our benchmarking tests.
There are different ways of improving your site’s loading performance, but for most site owners, using a caching plugin is not only the easiest, but also the single thing that will make the most difference. WordPress (and most content managed sites) dynamically generates pages on the fly which requires multiple calls to your database to retrieve the various content elements. By caching these dynamically generated pages visitors see plain HTML pages instead. This significantly lowers the loading times and the strain on your server.
On to the tests…
My initial idea was to test both a plain vanilla theme such as 2014, and then a more complex one which would more adequately resemble a “real” site. Upon starting the tests however I noticed that whatever impact the plugins had on the speed of 2014 was miniscule, and not much to write home about. There is just so much you can do to improve the performance of 2014, and in my opinion it's much more a question of your server setup than caching.
For this reason I decided to use the theme “Novelty” from Tesla Themes. We used one of the demo pages which is very magazine like, and implemented a slider option and jam packed the test site with images and text. We also installed some plugins with Twitter/Instagram feeds and a news ticker. Now, we had a page that was much heavier to load and more like a “real” site (more on the WordPress hosting set up below).
WordPress Cache Plugins Tested
So which caching plugins did we test? I scoured the WordPress plugin directory and found the following:
- AIO Cache
- Alpha Cache
- Bodi0's Easy Cache
- Gator Cache
- Hyper Cache
- Hyper Cache Extended
- Lite Cache
- Next Level Cache
- Really Static
- Super Static Cache
- W3 Total Cache
- Wordfence with Falcon
- WP Fast Cache
- WP Fastest Cache
- WP Rocket
- WP Super Cache
- WP-Cache.Com (site seems to be down)
- Zen Cache (formerly Quick Cache)
These were left out:
- Brutal Cache – Didn't work.
- Batcache – Seems to depend on Memcache, which wasn't used during this test.
- WP Superperformance – a new premium caching/performance plugin that had not been launched at the time of testing. We'll be testing this soon.
Autoptimize and Widget Cache were also left out since they are meant to be supplementary to the other plugins. That said, if used in the right way they look very interesting, and so I hope to get back to you with a separate test done on them.
Hosting and Benchmarking Tools ?
During the tests, we used a shared hosting account comparable with a majority of the alternatives out there. This way, we could get a better idea of actual speeds attainable for those users who are on a budget. The testing site had no traffic, and had no bot visits during the benchmarking. Our server did use Nginx as a proxy, not pure Apache, which meant that it had an advantage in performance from the get go, but this was the case for all tests so no plugin was advantaged.
In order to get a better idea of performance, we choose to use a whole variety of tools, which may be divided into two groups – site grading and timing tools.
Site Grading Tools
In this group we find the tools offered by Google, GTMetrix and Yahoo. These tools do not really judge a site on speed, but rather, on a list of quality factors such as:
- Optimization of images
- Server latency
- Use of browser caching
- Location of scripts
- Use of CDNs, parallelization/domain sharding
- Use of Gzip compression
- Number of HTTP requests
Google PageSpeed Insights
Google's PageSpeed Insight tool looks at a site from the perspective of both a desktop and a mobile device, and then gives a score of 1-100 for each test done respectively. Page Speed Insights is easy to understand, but very crude and does not give a whole lot of information on what can be improved. Even though the tool does give us quite a few insights into what Google might find important, the information given by GTMetrix and Yahoo is much better.
Some of the “advice”, if interpreted literally as requirements, are also a bit over the top…There is just so much you can do if your shared server has a high latency of 22ms, and if you complain about an image being 803b (bytes!) too large, you'd better take a look at your priorities.
Google, as far as we have seen, doesn't account for CDNs either, and upon making other tests we actually got lower scores using them. Using a CDN might very well make your site marginally slower on a “local” level, but able to handle “global” traffic much faster – which is more important?
GTMetrix and YSlow
GTMetrix and YSlow, which are tools based on Yahoo's performance guidelines, also score sites from 1-100. They are, however, much more sophisticated when it comes to measuring. Page Insights merrily gives a few pointers on what can be improved, while GTMetrix and YSlow base their grading on no fewer than 50 different metrics. GTMetrix also offers a “waterfall” chart to dissect the loading process as well as a very nifty loading time “history”. If you really want to find clues on how to improve your site speed, this is the place to go.
The following tools were used to measure page loading times and also to benchmark how the server would perform under load.
ApacheBench is a tool you use for checking how well your server handles load. We choose to check how many requests per second your site can handle using the different plugins. We performed this test by sending 1000 requests with 10 concurrent threads. We ran the test 10 times and then saved the best result for each plugin.
Here it should be noted that using Nginx does decrease the difference between a plugin/no plugin setup. Nginx is considerably quicker than Apache, which is why you should probably expect at least twice the difference with Nginx, than you would with Apache.
Pingdom is a well known server monitoring and testing service. We ran 20 tests with each plugin and saved the best result. Since our server is located just outside Stockholm, Sweden we choose to use their Amsterdam/NL server for the tests. Just like the others Pingdom has a page grading tool which we however, left out – GTMetrix and Google were enough.
Webwait is a simple but very useful testing tool. The whole idea with Webwait is to time how long it takes to load a page in your own browser. Hence, it's not a server side service, but runs locally. It loads a page over and over and then presents you with the average or median results. Thus everything in your own setup can affect the time. I choose to run this over ethernet, with no other services running, and to use the Opera browser. Each page was reloaded 101 times to attain the average and median load time.
Page Grade Test Results ?
We ran the test site through the different page grading services above and these are the results for Google, GTMetrix and Yslow.
As you can see here, some plugins seemingly do nothing in this area – their scores are either the same as or close to what you would get when no cache was used at all. In Google we see Supercache giving the best score both for mobile and desktop users. In GTmetrix and Yslow we see how Fastest Cache and Rocket outperform the others that remain pretty much the same. I attach more importance to the latter values since the tests (seem to) take into account more factors than Google Page Insight.
The best plugins according to this test are WP Fastest Cache, WP Super Cache and WP Rocket Cache. The best score, by far, for mobile devices was achieved by WP Super Cache. And, for those wondering, we did not forget turning on mobile caching wherever applicable 🙂
Timing Test Results ?
As we have discussed above, the grading scores are more a measure of the code quality of a site. They give clues on how your site might – MIGHT – get faster, though having a higher score than another site does not mean it really is. So, this is a misconception – the grading tools give you an idea of what you can do to get lower loading times, but do not really take timing into enough of an account. You can see this here on this screenshot from Pingdom…
As you can see here, this example page (not this one) has a performance grade of 96/100 which probably is better than 99% of the sites out there, THOUGH, it loads completely in a whooping 35 seconds. Is that an accurate way of measuring speed – I think not.
Here, we use Apache’s benchmarking module to see how quickly the server is able to handle requests. The more requests it is able to handle per second, the more visitors can be efficiently served.
Above, you see the server handles a measly 18 requests per second with no cache, and mind you, that is on an Nginx system. Each request takes about five-hundredth of a second.
Many plugins improve the amount of requests, though some perform much better. Checking the results we see that Hyper Cache Ext, WP Fastest Cache, WP-Cache.com, and WP Rocket perform almost 300% (depending on how you look upon it) quicker than the no cache solution. WP Rocket is the quickest. Somewhat surprisingly, WP-Cache.com takes the second place.
Here, the benefit of using a cache plugin is quite evident – you are able to handle three times as many requests with the best ones. Use Nginx, a good caching plugin and maybe something like Varnish, and you'll get a server that flies.
The test site with no caching plugin loads in about 2.8 seconds, which leaves many plugins twice as quick. WP Rocket once again tops the list being almost three times as quick as the reference site. This goes hand in hand with the figures seen for Apache.
I have chosen to present both the average and median load times for the Webwait test. These loading times are lower than the Pingdom ones since I am located closer to the server here in Sweden (remember Webwait is a browser tool).
Average Webwait Loading Times
Median Webwait loading times
As you see, WP rocket excels once again and looking at the median figures we also see Super Static Cache doing a terrific job, almost splitting the loading time in half. After that, we once again find the almost unknown WP-Cache.com doing quite well.
Server Performance Factors ?
Everything isn’t about caching – there are many factors that affect your server performance. These are the ones I feel are most important and almost always undervalued:
- Choice of server platform. Are you using Apache, Nginx or something else? Is it correctly configured? Do you use something like Memcached or Varnish cache?
- Type of server. Are you on a shared account, VPS or dedicated server? In most cases when I see a really slow site it is usually being hosted on a shared server. Here, the upside of changing to a VPS or dedicated server is tremendous.
- Number of images and image optimization. Today it's quite common for people to upload images directly from their phones or cameras. These images may, first of all, have a 10x too high resolution, and secondly may not be optimized for the web. If you see a big red warning when using Google Pagespeed Insights this is often always the reason.
These are my own thoughts and you can get an idea yourself just by analyzing the figures which are also available in their entirety on my site, along with a simple infographic.
All plugins have various features – which could be just a few, far too many, which may or may not be what you would ideally expect. Users of Super Cache, W3, etc. are often hardcore users who also use CDNs and all other bells and whistles. Others prefer easily configured stuff like Lite Cache or WP-Cache.com, and looking at the figures for the latter, they are not all in the wrong. WP-Cache.com does perform excellently in the Webwait tests.
Personally, I use a blend of WP-Rocket/W3 Cache on some sites, but I am also testing WP-Cache.com. Installing and configuring the former, which literally means clicking one button, is a set and forget process, which is just great.
The Best WordPress caching plugins ?
Choosing the winner wasn't hard, but the rest of the positions were. Number one is a no brainer really. WP-Rocket outperforms the others in almost every single test, and more importantly, even in those concerning the load time. When you take the ease of installation and support into consideration, it is even better. You do, however, have to pay 39 USD per installation, though you can get an unlimited license for 199 USD.
Coming in at position number two we have WP Super Cache. It comes as no big surprise that you see it here, and considering I already dubbed Rocket the winner, your best bet would have been either Super Cache or W3 Cache, right? To me they are a bit like red and green apples – almost the same. I am sure there are many out there who would spend hours at their keyboards talking up one or the other, but here I'm guided by our results.
I contemplated giving WP-Cache.com position three just cause of its incredibly easy installation. I however, once again, figured that the test was really about speed and not usability, and therefore put W3 Total Cache in spot three.
Almost taking a spot at three we have WP-cache.com, which is why an honorable mention or runner up position is well deserved. Getting these sorts of results with so little effort means the developers knew what they are doing. Had the tests been about ease of use WP-Cache would have easily been the winner.
I hope you liked the tests and that you will read my followups on similar subjects.
Thanks to Charles, who gave me the chance to write this article and also a big thank you to the Swedish host Inleed, who assisted me in setting up the testing environment.
If you have suggestions for future articles, or would like to read more about internet marketing, SEO and so on, visit my recently started blog at seolo.gy
Philip Blomsterberg is a Swedish internet marketer, who has been involved with several online businesses since the mid-nineties. He has worked on a myriad of web projects ranging from eCommerce, domaining and affiliate marketing. Philip worked as a consultant for companies ranked in the Alexa Top 150 and now specializes in SEO, which he blogs about at Sökmotoroptimering.se