Designing for displays that don't exist yet

What if you could decorate the walls of your home with electronic wallpaper, and change the content at will? This might sound like science fiction, but with a trend towards larger, more flexible, lower-power displays, we believe some form of 'smart wallpaper' could plausibly be with us in the not-too-distant future.

Smart wallpaper is just one example of how our screens might evolve. Not long ago, almost all our web browsing took place on a desktop monitor. Now, many of us own numerous electronic rectangles of all shapes and sizes, and we're increasingly adopting novel technology like wearable tech, flexible displays, virtual reality headsets and more.

As developers and designers, one of the challenges we face is how to create compelling content for such a variety of unconventional devices. How can we anticipate how they will be used? And how can we create beautiful, intuitive content on such unfamiliar platforms? Additionally, at the BBC we need to understand how we can use these devices to enhance media experiences for our audience.

The fact that these devices are so experimental (some of them don't even exist yet) can make creating new experiences for them seem like a very daunting task, but it is the only way we can test ideas and find out what does and doesn't work. We believe that by anticipating emerging technologies and creating prototypes for user testing, we can refine our ideas so when they do eventually get manufactured, we will be ready for it.

Recently, we built a browser-based smart wallpaper prototype. I'd like to share some insights we gathered during the project, in the hope that it might help guide other developers and designers facing similarly unconventional tasks.

Let UX drive design

The idea is to let the user experience, rather than the technology itself, drive the design. This concept is sometimes known as 'design fiction': taking inspiration from science fiction to create real-world prototypes in order to envision future technology and test out ideas. Think big about the experience you want to create for the user, and then let the technology catch up with you.

This is pretty much the opposite of the approach traditionally taken: invest a lot of time and money inventing cool new technology, leaving the intended user experience as an afterthought. Inevitably, the result is underwhelming, the use cases feel contrived and the product struggles to make an impact.

Instead, if you can create a sense of excitement and anticipation for a technology through how it could be used, the market appetite could drive manufacturers into developing the technology sooner. And if you can somehow prototype your ideas and obtain user feedback, you could provide insights that ensure the technology will genuinely suit its audience.

In practice, this means having an 'ideas amnesty': an open discussion to tease out all of your thinking (no matter how ridiculous!) about how the technology could be used. Try not to censor yourself – no idea is too silly – you're trying to get people enthused. The next stage is to choose the most plausible and engaging ideas to take through to prototype.

Constraints boost creativity

The problem with futuristic technology is that, often, you won't know exactly how it will evolve. This requires us to make some assumptions. With smart wallpaper, we thought it unlikely to be touchscreen, because it would be irritating having to get up to interact with it. Instead, we posited that interaction would be gesture-based via a mobile device. We also assumed the display would be low power to save electricity, and non-backlit so as to be unobtrusive.

We came to the conclusion that it might be less like an LED monitor, and more akin to e-ink. This obviously presents challenges, as e-ink is not optimised for video: it has low-refresh rates, low power and no backlighting. But in considering these limitations, we actually found they helped us think more creatively, and we ended up with more novel use cases.

Rather than simply imagining a giant wall of video, we began to think about the wall as a companion device to the video-optimised TV, adding stats and data to complement live sports and music events, or using panoramic images to create a more immersive experience. To test interaction methods, we even built an immersive 'Hide and seek' game for children!

Of course, it is also important to be flexible. Our assumptions may turn out to be incorrect, and if so we don't want all our work to go to waste. That's why our next insight is so important.

Stay hardware-agnostic

By nature, there are lots of unknowns when dealing with experimental technology. Because of this, it's risky to rely on a specific architecture in case the technology happens to evolve differently. We found the best way to avoid this was to use standard web technologies.

Since we can be pretty sure any new devices will be internet-connected, our work should be pretty future-proof. We wrote our 'Smart Wallpaper' application using HTML5 Canvas and JavaScript, with a RESTful API and Node.js backend. In our prototype, we simply used giant projector screens displaying fullscreen browsers to simulate a smart room.

Another advantage to using standard web technologies is that anything with a browser can represent a wall, so we don't have to wait for full-blown smart wallpaper to exist for our work to apply. As people get larger screens, and more of them, our system can start treating these as walls and delivering new experiences.

Push responsive design

We expect our web content to look beautiful and feel intuitive, no matter what the device. This links to the idea of responsive design, which aims to adapt content for different screen sizes. But as the variety of devices, displays and screens increase, we need to start expanding our understanding of responsive design. We can no longer assume our screens are going to be rectangular, flat or have other properties we've come to expect.

With smart wallpaper for example, we have to contend with furniture obscuring the display. Additionally, we have to find the right balance between automating the design and allowing the user to have some control over layout. We also have to determine which type of content suits which device.

All this presents a new challenge for responsive design. We need to go beyond arranging rectangular elements in a rectangular space, and start looking for solutions elsewhere. Perhaps we can use computer science algorithms like bin packing, or data visualisation principles such as clustering? Maybe we can take inspiration from the natural world? Can we use JavaScript to automate these new responsive layout systems? To what extent should they be automated?

These are some of the questions we would like to answer, and we imagine anyone hoping to create novel experience for emerging technology will face similar ones. By thinking about these problems now, we might just bring the future that little bit closer.

Words: Rosie Campbell

Rosie is a research technologist at BBC R&D, and has been working to prototype new content experiences for emerging broadcast platforms. This article originally appeared in issue 268 of net magazine.

Liked this? Read these!

• Why UX is not about design
• Free tattoo fonts for designers
• The ultimate guide to logo design