Erika Hall on evidence-based design and looking beyond data

Erika Hall isn't a surgeon, but she's not afraid of dropping a visceral medical metaphor to explain the intricacies of web design: "If you redesign the experience you're offering to the world, and don't redesign the workflow inside the organisation, that's like doing a face transplant without hooking up the nerves and blood vessels underneath it," she announces. "The new face will wither and die and slide off the skull."

Hall is talking about Mule Design's redesign of the National Audubon Society's site, which resulted in a 300 per cent increase in returning visitors. More specifically, she's talking about why it's essential to engage the entire organisation during a redesign, and how one of the hardest parts of that process is persuading people to change their habits.

"Users have habits we have to design for – clients and designers get that. Organisations have habits too. And these habits are extremely difficult to change, " she continues. "You need to recognise that any change is work, even if what you're doing is going to eliminate a lot of difficult workarounds and make everyone's life easier on paper. The workarounds [still] feel easier than any new process because they're habits running on cognitive autopilot."

To change habits, says Hall, you need to understand why people in an organisation do what they do. "You need to connect the organisation's goals to a material benefit for anyone who needs to work differently to achieve that goal. This requires understanding how people view their jobs, and do their jobs and communicate with each other. You have to listen to people before you ask them to change."

Mule work

Audubon is the perfect example of Mule doing what it does best: web design and content strategy, in close collaboration with its clients.  The interactive design studio provides strategic consulting, design and technology services to organisations around the world, alongside quirkier personal projects like Unsuck It, an online translation device for corporate jargon labelled "delightfully hostile" by The New Yorker.

Hall co-founded Mule with designer Mike Monteiro in 2001, after the pair met at experience design firm Hot Studio (acquired by Facebook in 2013). "We started Mule because we loved the work and wanted to keep doing it, but on our terms. We were both terrible employees. Really, I feel bad for all of our former managers," she laughs. "Also, in 2001 the much-hyped new internet economy cratered. So that helped. We had to make our own jobs."

The studio's first client was a wildlife preserve called Fossil Rim. Yahoo followed soon after, then the UN – and they haven't looked back. Now a tight-knit team of 12, the studio has started doing more training and organisational consultation in recent years, alongside its project work. 

Despite reports claiming otherwise, Hall says there are plenty of opportunities around: "There was a lot of noise a couple of years ago about the end of client services. This noise seemed to emanate from designers who were tired of running their shops and wanted to go in-house, which is fine. However, there is still a tremendous amount of work out there to be done."

Everything Mule does starts with research. It's a topic particularly close to Hall's heart: she's the author of Just Enough Research – an engaging cookbook of research methods – and has delivered a number of talks on the subject, reminding people of its importance in improving your odds of success. 

The problem, she argues, is that we also use the term 'research' for activities intended to gather new knowledge for its own sake. "That makes it sound like an optional luxury," she points out. "We need to stop thinking of research as a bonus add-on. Gathering the information you need to solve a problem is a part of solving the problem."

Hall suggests reframing it as evidence-based design. "The point is that all of our design decisions should be based on evidence, otherwise it's just speculation. Imagine hiring an architect and telling them to forgo surveying the site to save money, or not to think about how the building will be used. That seems ridiculous."

The way to begin the research process, she advises, is to start by thinking about the scope of your decisions, and then the inputs to those decisions. Are you working from information or guesswork? How confident are you in your information? Where are the risks that you're wrong? The answers to those questions will tell you what information you need to gather.

Too much data

One mistake Hall sees a lot of designers making is to forget that data alone won't guide us. We're generating an unfathomable amount of data: we carry devices embedded with sensors, and more and more human activity is routed through the internet. 

"There's no limit to what we can track and measure," she explains, adding that this is exciting to designers and technologists who want to make better-informed decisions. But more data doesn't necessarily create more meaning, and might even make it harder to see what matters. 

"It's very easy to react to the data, and treat the things that are easiest to measure as though they are the most important," she warns. "We have lazy brains that are overactive pattern-matchers, so we want to go to where the patterns seem to be, rather than taking time to be disciplined, sceptical thinkers. Also, it's very tempting to give into confirmation bias. When there is so much data available, it's quite easy to find something to back up an opinion you already have, or to support your desired design."

Hall argues that what we can't count still counts. But how important – really – is unquantifiable data like human experience and choice, and how can designers build these aspects into their strategy? 

"When you say 'unquantifiable data' , it can sound like some sort of magical fairy dust, but qualitative data simply means descriptions. It's often much richer information than measurements," she explains. "Descriptions are much easier for humans to work with because we are storytellers, not calculators. Measurements will never tell you why something is happening, and the 'why' matters a tremendous amount."

The way to harness this information is simple: observation and discussion. By watching a handful of people using a site and talking to them, Hall claims you could find out why conversions have dropped off in just a day. "Sure, you could A/B test different fixes, but that's just trial and error, and depends on you seeing statistically significant changes. It's very wasteful."

The point is this: if you start by counting and measuring, you will likely count and measure what's easiest – and that might be the wrong thing. "Figure out what your goal is and stop worrying about looking smart. That's my best piece of advice ," Hall asserts. "Most bad decisions are somehow grounded in insecurity about looking smart. That makes people dumb."

Fear goo

Hall is big on tackling fear. She calls it 'fear goo' and says it's endemic in humans. In design, it often manifests in people trying to solve something by themselves, rather than being collaborative or asking for help. It's also found in organisations where leaders are afraid of being questioned, and staff are afraid of questioning the leadership.

But asking questions is the cornerstone to good research – and it's also key to knowing how much research is enough. "Start with the problem you're solving and ask what information you lack," she advises. "Take a hard look at your embedded assumptions. Your questions will determine the approach and methods you should take to answer them. If you ever stop asking questions, you're doing it wrong."

She's also big on avoiding complacency. Striving to be comfortable isn't productive: comfort means you're not growing. Fortunately, Mule keeps Hall happily on her toes.

"Running one's own business is a fantastic way to remain uncomfortable all the time," she smiles. "Really, I'm not even being facile. So, I have that on lock. New challenges all the time."

She continues: "The gift of experience is that I'm now comfortable in situations that would have made me throw up earlier in my career. I want to take on new kinds of work that make the best use of all of this accumulated expertise. And we have such fantastic people working at Mule, I want to do right by them and give them the support and opportunities they need to shine."

Mule also has a renewed focus on its home city of San Francisco, in terms of client work and beyond. The studio's new space has given the team room to run an art gallery and offer a variety of events that are open to the community. 

"That's been a lot of fun, and really important to us. So many new companies seem to be taking from San Francisco and not giving back. Cultural shifts that seem innocuous, like catering lunch in-house, deprive local restaurants, for example. We're here because we love it. If you're just going to stay sealed up in your posh office all day, you could be anywhere."

It's all about focusing your energies: "Technology changes, but human nature doesn't really. We need to stop getting distracted by the next shiny thing, whether it's wearables or bots or VR, and focus on what we're trying to accomplish," Hall reasons. "Design is bringing change. What change do you want to see in the world?"

Photography: Tory Putnam

This article originally appeared in net magazine issue 282; buy it here!