We're hiring!

How you should test your website speed for performance, and why you are doing it wrong


Article by Erlend Eide. Published on July 3, 2018

Have you ever had the feeling that your website is not as fast as the tests tell you it is? And do you wonder why? We talk about and work on website speed and performance with developers and website owner every day, and in this article we are lining up a few testing methods that may help you on the path to make websites with super snappy response times.

Let’s just say it out loud before we start this; you think your website is fast because you are testing your cache.

My hosting company says my website is fast, but I still feel it is not performing all that well

When it comes to web performance, it is important to pay attention to how it feels. After all, web performance is all about perception. Especially when something feels slow, it always is because it actually is slow.

Hosting companies (maybe also some agencies) tend to show you tests that deliver results that are in line with the recommendations from Google Insights or point you to looking at a PageSpeed score on Pingdom Tools. But how can it be that the feeling you get when browsing your website differs from the results you are looking at in the online tests?

Tests produce false results because you re-run them

The most common reason is that the tests you run do not provide results that are equivalent to a normal, random visit to your website. What people tend to do, is to run a test in Pingdom Tools, and then re-run it a couple of times until it produces the loading speed numbers that are acceptable. But doing this is tricking only the test, and does translate into the experience your website visitors get.

Visitors to your website do not reload the page when they land on a page on your website, they move on, read, and hit the next link they find. To make a website truly performant, you will have to deliver fast results on this very first visit.

Before we dive into how you can make a website fast on the first visit, we’ll provide you some techniques to test the real underlying performance of your site.

How to test the performance your normal website visitors experience

There are a variety of testing tools available that you can use to check the performance of websites. We usually just use the waterfall analysis in Chrome’s developer tools from our own laptop, but if you find it easier to use online tools like WebPagetest or Pingdom Tools that is perfectly fine too. We avoid GT Metrix, because it  produces unreliable results, especially when you want to test the underlying performance of your website.

Visitors enter your site through random pages

The first thing to be aware of is that users do not just enter your website through your front page. If you check out the landing pages view in Google Analytics you will probably find that you have visitors that land on close to every page on your website. The front page usually has a larger share than many other pages, but even if 30% land on your front page – there are still 70% of visitors landing on other pages.

There is also a large share of visitors that have never visited your website before, you can get the details in your New vs Returning Visitors report in Google Analytics. New visitors do not have any page elements stored in their local browser cache, and the full page and all its assets have to be delivered before you can view the page.

Fix the test by adding a cache buster parameter to the URLs you test

Simulating a fresh new visitors experience can be done by adding a cache buster to the pages you test. A cache buster is a parameter that you add to the URL of your website, to make the request bypass any full page cache you may be using. The parameter needs to be unique for each load of the page since the request, with those specific parameters, will be in cache after the load. We usually use https://domain.tld/?cache=busted and then add a number after busted for each page load.

So you might ask – why would I want to test my webpage without hitting the cache? The explanation for this is dreadfully simple – your cache does usually not work when visitors land on your website.

Testing the real performance of a cache based site

WP Engine primarily bases their hosting speed and performance on caching. Their own website is a perfect example of how caching does not work very well for websites.

We tested their site with Pingdom Tools. The first test of their front page loads super fast, in 0.13 seconds. That is a result which is well within the recommended thresholds outlined by Google PageSpeed Insights and other similar recommendations.

But let’s test the how fast their underlying performance really is. We can simply add a cache buster to their URL, so we make the next request to the following URL:

> https://wpengine.com/?cache=busted

WP Engine’s front page with the cache buster added loads in well over 1 second, which is way over the recommended limits for good web performance.

To re-run tests with cache-busters, make sure to change the parameter ?cache=busted to for example ?cache=busted2 – else you might be getting cached results again.

So why do we do this?

Why full page caching does not work for web performance

There are two absolute truths for caching that you need to know about. First, caches need to be filled before they can be used. That means that caching as a performance enhancement, will never work for the first request. The second important thing is that caches always expire. This means that even though you cache a page now, the cached page will at some point be erased from the cache – and the cached element needs to be rebuilt.

Caching as a performance enhancement is widely recommended by close to all hosting companies. The reason for this is that full page caching always will have a positive impact on resource reduction on the server level. However, this does not automatically mean that full page caching will give your website overall better performance.

In our article about How Caching Works in WordPress we spoke about the origins of caching on the web, and under what circumstances caching can be effective. Full page caching can be effective for scaling, or if you have largely static web pages. The problem is, that is not how the average normal WordPress, WooCommerce or Magento websites are built.

How caching works live in action, is that full page caches only provide performance improvements for recently accessed pages (good for scalability, but limited effect on performance).

Testing the performance of WP Engine’s full page cache

To use WP Engine as an example, we ran a crawler against their website to check out the number of pages that responded quickly, and which did not. The following graph shows 189 largely random requests that the crawler made to the WP Engine website, organised by response time in seconds on the vertical axis.

Of the 189 requests only 12.4% of requests responded within 400ms. Hence, 87.6% of their pages responded in more than 400ms. What’s worse – about 50% of pages had a more than 1 second delay before being delivered.

Is it acceptable to deliver a loading time distribution like this, and if you would be happy with that for your website? Google Insights has outlined the limit 200ms as the number for acceptable response time from websites on desktop networks, and for mobile websites this number is 400ms.

Why caching doesn’t improve the speed of your website

The average response time graph above is very similar, often even worse, for most other websites that use caching for performance. There are several reasons why caching is not effectively improving the speed of your website:

  1. The full page cache does not work for the first request to your website
  2. The full page cache contains only recently accessed pages, older items are expired
  3. Visitors enter the site through a large subset of random pages, which means the chance of hitting a cached page is much smaller than the chance of hitting a non-cached page
  4. Site admins clear caches regularly to get new content published

If you at this point still hope to make your website fast by using full page caching as the main performance enhancer, your site has to have the following characteristics:

  1. The web site content must be largely static
  2. The users of your website should not initiate sessions or log in
  3. You should seldom clear the cache
  4. Your site needs to have a lot of traffic, to ensure that visitors usually hit cached pages

The problem with these requirements is that they are never true for e-commerce sites. They are also very seldom true for other normal websites. The kind of sites where all of the requirements are met can for example be online newspapers and magazines or some times blogs, where dynamic parts of the site are offloaded to external services.

The potential speed gains from caching are limited

Caching as a performance enhancement does also not apply to all parts of your website. The effects of full page caching are limited to work for anonymous users on your website. It is possible to cache more dynamic pages, but then we move in to a territory that is complex and advanced, requires a lot of setup, maintenance and introduces a lot of new complexity both for developers and website owners.

What we’re saying at Servebolt is that full page caching is like bad make-up. At a distance it may make your site appear good looking, but if you look closer nothing has really changed much.

How to get serious about working with web performance

  1. Stop hoping and tricking yourself into believing that you can solve your web performance problems with full page caching, you won’t succeed!
  2. Start tracking real web performance data (instead of just testing the performance of your cached pages)
  3. Start testing your sites performance with a crawler
  4. Start focusing your attention on possible performance enhancements that will apply to all parts of your website

Tracking real performance data from site visitors using Google Webmaster tools and Google Analytics

Google crawls your website continuously every day, and provides some statistics that you can find in Google Webmaster Tools, Crawl > Crawl Stats there is a «Time spent downloading a page» graph. This metric often provides a good indicator of the base performance of your website. What is the average value for your site?

If the numbers are above 500ms on average, there is a lot of room for improvement. It is also very common that sites with full page caching have varying data from day to day, where the crawler is “lucky” one day and hits some cached pages, while the next they crawl the uncached parts – and get high response times. This reflects what regular visitors to the site experience.

Real User Montioring (RUM) with Google Analytics

The cheapest and easiest way to get real performance data from your websites visitors is to use the Speed Reports in Google Analytics. They are by default configured to sample only 1% of your website traffic, which for most websites means that the data is useless by default. You will therefore have to increase the SiteSpeedSampleRate to for example 50% or 100% to get useful metrics (adjust it according to max 10 000 page views per day).

This graph is from a site that switched from dedicated VPS hosting with caching to non-cached managed hosting. The server response time dropped to a fraction, and the average full page load time was sliced in half.

Test the performance of all pages on your website using a Crawler

We use crawlers to get a real view of how websites perform. By crawling all pages of your website, you will easily be able to identify page types or specific pages that perform worse than others. We have made configurations that exclude all front-end related performance metrics for the results in this article. The obsessive focus on back-end performance is what in the end will allow you to build a fast front-end.

  • Sitebulb (Full free trial available, this is a tool worth subscribing to – it is evolving at great speed)
  • SEOFrog (What we have used for the average response time graphs below, payed application with focus on SEO)

The web performance impact of the first request and the time to first byte (TTFB)

The average response time of the first request on any random page load on your site is the single most important thing to pay attention to. If the response time of the first request is slow, your site will never feel fast no matter how much you optimise the rest of your webpage.

The reasoning for a quick first response time is very simple. Your site has to respond within the blink of an eye, else the delay will be noticeable for the visitor. In practice that means you have about 400ms to make the page and get it delivered to the end user (including network latency).

For websites basing their performance on caching, the load time distribution of a website often looks similar to this:

A small subset of the pages respond in the 0-1 second rage (typically cached), while the majority of pages respond in 2-3 seconds, and the average over all pages is approximately 2 seconds. When your response time distribution looks like this, your website is having performance issues – and you need to get your hands dirty to improve the performance on all pages. Caching won’t save your day.

What people think they get with caching, but don’t – are the following numbers. These are from a highly dynamic high traffic E-commerce website that runs completely without full page caching, and where absolutely all requests to the website respond in under 0.5 seconds.

To relate this to the WP Engine website’s loading times, this graph is delivering 100% of page views in ~0.4 seconds, whereas WP Engine does this only for about 15% of pages on their own website.

How do you want your site to perform?

What kind of improvements will affect all parts of your website?

When you have turned off caching and you can touch and feel the real performance of your website, and started tracking some real user performance metrics – you are ready to start implementing improvements that affect the performance of your whole website.

At the end of the day, investing your time on solving general performance issues will make you work a lot less – and give you much better results in terms of web performance, than by trying to solve performance issues for only parts of your site.

Every website is unique, but here are a few tips on how to improve the performance for your visitors:

  • Switch to better hosting – Servebolt delivers out-of-the-box performance that outpaces any VPS or Cloud instance.
  • Use hosting that is close to your audience
  • Use modern PHP 7.2 (or at least 7.0+)
  • Use databases right (you should especially make sure that queries use indexes properly)
  • Use faster databases
  • Get rid of dead weight in your application (remove plugins and clean up)
  • Fix bugs – in your back & front-end (you wouldn’t believe the impact bugs has on performance)

Takeaways from this article

  • Don’ t take it for granted that caching is making your site faster, track and check real statistics
  • In the end, the non-cached performance of your website is usually what determines the performance for the end user