Trying to measure and analyse your site’s mobile performance is like taking aim at a constantly moving target. As soon as you think you have some idea of what exactly it is you’re trying to measure, it changes.
Actually, using this analogy, trying to measure and analyse mobile performance is more like taking aim at dozens of moving targets – all at the same time. When you consider the huge range of results you can get from measuring different permutations of devices, browsers, and networks, it’s no surprise that many site owners do the equivalent of closing their eyes and shooting blind.
This past summer, my company, Strangeloop Networks, conducted an industry first: we measured the performance of top ecommerce sites – both m.sites and full sites – as these sites load on newer and older Android and iOS phones and tablets (Galaxy S and S3 phones, Galaxy tablet, iPhone 4 and 5, and iPad 2), and over both 3G and LTE networks.
Our goal was to take a snapshot of mobile performance and see what trends emerged. We released the results in our 2012 State of Mobile Ecommerce Performance, but here’s a high-level overview of a few of our key findings, which should be a wake-up call to site owners:
1. Desktop vs smartphone performance
The median page (full site, not m.site) took 11+ seconds to load for both the Galaxy S and iPhone 4 over 3G.
Keynote recently released a study that found 65 per cent of mobile users expect sites to load in 4 seconds or less. Eleven seconds is unacceptable.
This finding should be a wake-up call for site owners. In a study we performed last year, in which we tracked mobile user behaviour through a leading ecommerce site (one of our clients), we learned that 35 per cent of visitors chose to view the full site over the m.site. Knowing that, and knowing that many people rely on 3G at least some of the time – particularly when they’re consulting their smartphones in-store – we need to take full-site performance over 3G seriously.
2. LTE was 27% faster than 3G
The average load time for pages over LTE was 8.5 seconds, compared with 11.7 seconds for 3G. We expected pages to load faster over LTE, but frankly, we expected the improvement to be much more compelling. A lot of industry experts have touted LTE as offering 10 times the performance of 3G, but we simply weren’t seeing it in our results.
3. One in three don’t have a mobile-specific site
We defined 'mobile-specific' as any site that has a unique URL, to which mobile users are automatically directed. Digging into this finding reveals a lot of the issues with the current 'm.site vs full-site' debate. A year or two ago, it was pretty clear that when someone talked about a 'mobile site' they were referring to an m.site that was a stripped-down version of the homepage. These days, 'mobile site' refers to any page that has been designed to serve a different user experience – via different content and/or layout – to mobile devices.
In 2010 or 2011, knowing that a site did or didn’t have a mobile-specific site told you whether or not that site did or didn’t have a mobile strategy. These days, it’s harder to call, as more and more companies jump on the responsive web design (RWD) bandwagon. RWD opens up a whole new can of worms, however: poorly implemented RWD can make pages even slower.
4. 32% of owners don’t let visitors go to the full site
As I mentioned earlier, this is a huge no-no – and it’s not just because up to one-third of your traffic may prefer to see the full site. It’s because a mobile user who visits the full site spends up to 5.5 times more than a mobile visitor who stays on the mobile-specific site.
5. Not all tablets are alike ... for site owners
One third of sites served the mobile-specific site to the Galaxy tablet, while serving the full site to the iPad 2.
If you’ve ever used a tablet and found yourself on a mobile page, you know how ridiculous it looks. This finding is a reminder to site owners that they need to optimise for a multi-tablet world. Apple’s tablet dominance is waning as cheaper Android tablets, including the new Kindle Fire, are gaining a foothold with consumers.
So what does the mobile future hold?
There are three things you can count on in the months and years ahead of us:
- Mobile networks are going to get faster
- Mobile devices are going to get faster
- Mobile browsers are going to get faster
But there are two other things you can also count on:
- Web pages are going to get much bigger and more complex. We have images, rich media, and third-party scripts to thank for this. Today, a typical page is more than 1MB. Based on the current rate of growth, this number will hit 2MB by 2015.
- Mobile users are going to expect faster sites. In 2011, most mobile users reported that they expected pages to load in five seconds or less. In 2012, that number has dropped to four seconds. Expectations never stagnate.
I don’t believe this is a stalemate. I do believe that site owners need to arm themselves with as much information as they can, not just about their users and their sites, but about the many permutations of experience that can happen between their users and sites across different devices, browsers and networks. It’s the difference between taking careful aim and shooting in the dark.
I invite you to download the State of Mobile Ecommerce Performance and check out the rest of our findings.