Why technology should never get in the way of a story

By his own admission, Jake Barton is "obsessed with collaborative storytelling". It's a field with seemingly unlimited potential for innovation. Digital technology, Barton believes, can help us tell more effective stories, blending shared experiences from large groups of people to create something much greater than the sum of its parts.

Barton's agency, Local Projects, has found a lucrative niche developing immersive experiences for the world's most forward-thinking museums and public spaces, making use of everything from facial recognition technology to interactive light sculptures to engage and delight the general public. We caught up with the charismatic designer to get to the heart of his creative process.

Local Projects is a "media design firm for museums and public spaces". What's so exciting about this field?

Almost every bit of media out there, from websites to software to mobile, is created for a single person to look at and relates to a screen. We create experiences that create social dynamics by overlapping meaning with innovation. We look for how to leverage new technologies in ways that are both engaging and meaningful, creating lasting impressions and long-term solutions.

How does a project like Big Heart, for example, fit into this ideology?

Big Heart is an interactive light sculpture that we made with [architect] Bjarke Ingels in Times Square. It invites visitors to place their hands on an interface, and as more people gather, the heart sculpture beats faster and faster. This simple social cue, asking more people to connect together, unites strangers in America's most crowded public space.

What does innovation mean to you?

I often say that it's not hard these days to be first at doing something. Whether it's applying 3D scanning to architecture or facial detection to museum displays, being the first to apply a new technology to an old context is not difficult. But it's really hard to create innovation with a point.

That's our highest endeavour; looking to leverage technology
to make human, emotional and engaging experiences. We don't find comparisons easy, and would see ourselves in the same category as other firms that invent with purpose, whether that's the architects Richard Rogers or Frank Gehry, or design firms like Art+Com or Ideo.

Do you consider Local Project to be 'rivals' with any other studios?

Ha! We’re working on a project with Stamen right now actually, and the founder is an old friend of mine. We generally don't spend a lot of time thinking about rivals, as we have found this very unique niche of pursuing innovation and meaning, and there frankly aren't very many - if any - firms that are looking to that model at the moment. We feel like knowing how to make things is critical to a leading strategy and concept design.

You've said that the ideal relationship between design and technology is like that between a painter and paint. Can you elaborate?

Painters get direct feedback as they create, seeing the painting emerge before them and creating in direct response to what they see. We make experiences that induce behaviour within our audience. So to get that same direct feedback and response, we engage with prototyping even at the beginning of the concept stage and use it as a creative tool to understand what's working.

We build fast and dirty mockups out of foam core and duct tape, see new ideas emerge quickly and put them in front of a range of audiences to understand what's working from the beginning.

How do you develop and test your ideas?

We follow the visitors' responses to our prototypes. We take nothing for granted, and present our prototypes to fresh eyes to see how our visitors will respond.

On two occasions, we've had clients take out their phones when we have revealed a prototype. On many more occasions we've seen ideas we thought were unimportant become central. And on even more occasions we saw ideas we thought were guaranteed winners crash and burn, leading us to rethink and approach a challenge anew. When we release projects, they work because we've already launched them and seen how people use them.

The necessity of allowing for failure during the innovation process has been discussed a lot in recent years. How do you factor this in and manage the process?

I find the failure meme both instructive and, at this point, clichéd. Yes, failure is an outcome that you need to integrate and listen to, but we believe in putting forward smart and engaging ideas and then watching as they work - or don't work - with audiences.

Our 'prototype first' process is meant to engage visitors even during the concept design stage of the process, and we use it to watch and deduce what visitors see and find interesting.

Integrating that into a client services business is very, very hard. It's hard financially, because it's unpredictable. And it's hard emotionally because you look to lean into, admit and analyse failure, versus avoiding discussing it - or worse yet avoiding making innovative work from the beginning. Because we look to create the overlap between innovative and meaningful work, we can't rely on replicating past models.

We are looking to make new work that can engage - and that takes an iterative approach.

What are the potential pitfalls of innovating for the sake of it - can you give any examples of when this has happened?

Any project that relies on being first to market almost invariably misses its true potential to be innovative with a point. We look to leverage our staff interests, passions and insights into the projects, which is what makes them emotionally engaging.

That's the key to innovative work that endures; it tells a good story or induces a lasting emotion. examples of when that has happened include almost every startup that isn't engaging deeply, solving something key, or connecting people in an elegant way.

You've done a lot of work in the past with gestural interfaces, notably with the Cleveland Museum of Art. Where do you see this field developing next?

Gestural interfaces are just beginning to become viable, but I think what is far more interesting is how they become thematically meaningful. The fact that visitors can draw a shape with their fingers and be rewarded with a work of art that matches that shape is a meaningful thrill; similarly, a body pose to match a figurative statue, or a facial gesture to match a portrait.

Allowing visitors the chance to participate in something larger and more meaningful than themselves is the real key to the entire endeavour.

Are there any technologies that you’ve had your eye on for some time and would love to work with but haven’t yet had the chance?

I'd love to really do a good job on Google Glass. The technology and the interfaces are so poorly conceived that everyone is missing that somewhere deep inside there are some very, very exciting things that can be done.

It's a technical marvel, but it's so alienating and cumbersome that the key ability to offer an intuitive, networked experience that offers critical, contextual information gets lost. It's really a failure of imagination - and a sign that Silicon Valley isn't enough in touch with what being a human is like for most of us - that they thought Glass would fit right in.

This article originally appeared in Computer Arts issue 228.