The importance of data in design

When Doug Bowman left Google in 2009 he lamented being a designer in an organisation where "design ... lives or dies strictly by the sword of data".

Having been on both sides of this most barbed of design fences, I counter Doug's unease with a lament of my own; that we as an industry have settled on the idea of the designer as mentalist, soothsayer, bringer of wisdom – they who know the answer to the most obscure of problems. I know and have evidence that we designers are not just occasionally, but routinely, wrong.

A revelation

I joined Booking.com three years ago after a stint of agency work in New Zealand, following five years of freelancing in the north of England. In both those past lives I traded on this idea that I – the mighty designer – knew the answers. That I had an innate ability to understand exactly what customers wanted and needed without any proper user research, without any real testing or data.

Success in these situations was in the satisfaction of the client. Sign-off was my ultimate goal, and hopefully a bit of additional work would come as a result of the confidence I inspired.

Fast-forward three years, and as the principal designer at one of the world's most successful ecommerce companies, I look back on my past roles with a mixture of distaste and disappointment. I am lucky to now have access to as much data and as much qualitative user research as I want. This data forms the cornerstone of our product development process – which is distilled into three steps:

• Use all available data-points to conceive of ways to improve the customer experience
• Implement and measure the impact of your change
• Rinse and repeat

What this process highlights is one constant and reliable truth: I am normally wrong. And I am not unusual in that regard. We're all wrong most of the time, regardless of experience, skill level or favourite football team. Some of us may be wrong less frequently than others, but we're all mostly wrong, most of the time.

In an environment like ours, designers expose themselves to constant, aching rejection at the hands of millions of people. The idea that it is possible to know for sure that you did something right is a novel one for a designer, once they disabuse themselves of the notion that knowing is what makes them a designer. I feel incredibly lucky that I can find out whether my ideas are right.

Testing, testing, testing

An oft-uttered complaint about data-driven product development is that it leads to a schizophrenic user experience in which design is by consensus and not 'led' in the Don Draper sense. We're acutely aware of this risk, and, having the job title of principal designer, I consider it my responsibility to make sure that in the midst of our thousands of A/B tests, a common language and experience is presented to our users.

But the character of that experience is absolutely defined through empirical, reproducible, successful experiments. I wouldn’t want it any other way.

Many proponents of A/B testing limit the science to the edges of their product – the position of a button, the headline in an email. We go much further than that: our product itself is defined by customer need, and if you're going to design by committee, you'd be well served by electing a few million customers as committee members and giving them the very best experience they'll find on the web.

In our industry, the gulf in knowledge between designer and client is huge. And whilst data can help to bridge this divide, it also erodes the mystique of the designer. We must ask ourselves whether we want to make the things we do better, or protect the mystery that surrounds 'design'.

Words: Stuart Frisby

Stuart Frisby is the principle designer at Booking.com, in Amsterdam. This article originally appeared in net magazine issue 260.