How A/B testing can fuel your creativity

Phil Williams

Phil Williams

It's methodical, calculated and many believe, unimaginative. A/B testing is a scientific process which draws insights from vast amounts of quantitative and qualitative data. We draw hypotheses from this data, make improvements and set it against the control as a test. But to what extent does testing kill creativity in our industry?

The truth, I believe, is that A/B testing your site actually encourages a far greater level of creativity, since it makes us ask that all important question: 'What if..?' What if we removed that image, flipped the layout, or changed the process completely?

These are fundamental questions often only asked after a session of rigorous user testing. After we've asked the all-important 'if' question, we come to the 'why' and the 'how'. For example, 'why should we alter the site map, and how should we do it?'

Asking these questions enables creativity to enter into every stage of the optimisation process, as creative teams are less afraid to voice any questions or ideas over fear of upsetting the status quo.

Think back to childhood. You could ask those 'why', 'if' and 'how' questions on a regular basis without fear of judgement. In the working world as an adult, to ask a question like this might seem far too daunting or risky.

This is the reason why questioning your website or challenging a particular process can lead to whole new world of discovery, change – and creativity.

Avoiding those pink elephant ideas

The creative process on a basic level is lateral thinking in its simplest form. We are presented with a problem and we wrack our brains to try and think of something else. Let me demonstrate:

Please don't think about a pink elephant… What are you thinking about now? Is it is a pink elephant?

Although I asked you not to, I'm betting that right after I simply mentioned 'pink elephant' that's all you could think about. When we are presented with an obstacle, our brains try to find a way around it which often sends us straight back to square one.

Illustration by Mike Chipperfield of Magictorch:

Illustration by Mike Chipperfield of Magictorch:

With A/B testing, our control works like this pink elephant. As a designer, my job is to think laterally rather than just changing the colour – it can be completely the opposite, like removing it completely.

Great creativity isn't about making the flashiest website with the most aesthetically pleasing graphics. My ethos is against designing for design's sake. I look at a problem, and figure out a way to deliver maximum results with making as few changes as possible – much more creative if you ask me.

Proof that it works

An A/B testing exercise was undertaken for EA Game's Sim City 5. The objective was to increase the number of pre-orders for the game. The control (on the site already) offered the reader $20 off their next purchase via an attention-grabbing banner.

The original creative process is clear to see here – draw attention to a great offer and more people will click through to buy the game. It seems obvious, but the results tell a different story.

Surprisingly, the simpler version without the promotional banner actually drove 43.4 per cent more purchases. The banner was having the opposite effect than intended.


What the data drawn from this A/B testing tells us is that users just wanted a simple way to buy the game, and instead of enticing people to buy, the banner was actually acting as a barrier to purchase.
It might be argued that the web page without the banner is less creative, but in this case, the decision was taken to remove it completely.

After pinpointing where the problem areas on your site are, it is creating new hypotheses to test that can truly allow creative thought to flourish. Crucially, it's about keeping your mind open to new possibilities for how things could be. A/B testing simply provides an audience to validate one creative choice over another. A/B testing doesn't kill your creativity, it adds fuel to it.

Words: Phil Williams

Phil William is a UX designer at conversion optimisation agency PRWD.

Like this? Read these!

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

*Read 5 free articles per month without a subscription

Join now for unlimited access

Try first month for just £1 / $1 / €1

The Creative Bloq team is made up of a group of design fans, and has changed and evolved since Creative Bloq began back in 2012. The current website team consists of eight full-time members of staff: Editor Georgia Coggan, Deputy Editor Rosie Hilder, Deals Editor Beren Neale, Senior News Editor Daniel Piper, Digital Arts and Design Editor Ian Dean, Tech Reviews Editor Erlingur Einarsson and Ecommerce Writer Beth Nicholls and Staff Writer Natalie Fear, as well as a roster of freelancers from around the world. The 3D World and ImagineFX magazine teams also pitch in, ensuring that content from 3D World and ImagineFX is represented on Creative Bloq.