Convergence isn't just a buzzword. Creative areas that used to be separate are growing together, and it's no longer enough to have a skillset that's limited to one set of tools for doing one kind of work. So far audio has been illustration and design's poor relation, so aside from dink-donk noises on Flash sites, seasoned with the odd whoosh and perhaps a chilled background loop, audio and music haven't had the exposure needed to make them prominent in their own right.
A few design houses have always bucked the trend. Rick Smith and Karl Hyde's Tomato set the scene in the late 90s with a combination of design and a spin-off band project called Underworld. More recently Fred Deakin of Airside has combined a design career with a commercially successful music project as half of Lemon Jelly. Blending music and graphics makes for interesting promos, but it also makes it possible to offer clients an all-in-one service across multiple media. The in-house advantage means more chances of work, and wider creative options.
But there's still a perception that music is difficult, specialised and expensive. Where design only needs a laptop, music needs special rooms, big speakers, extra hardware and years of training - doesn't it? As well as being a columnist for Computer Arts, Jason Arber is a director and producer at Wyld Stallyons, an animation and moving image company based in London with a client list that includes ITV, BMW Sauber F1, MTV and Bacardi. He explains, "We do most audio in-house now, although occasionally we'll outsource work. Projects will be a mix of elements - for example we're working on a fairly corporate piece of animation at the moment, and the music bed is a piece of library music augmented with our own voiceover and sound effects recordings. We sometimes record what's called Foley - footsteps, other live sound effects - here."
Being able to record sound effects also transfers to being able to record vocals. Voiceover studios aren't cheap, and for the price of an affordable collection of hardware and some basic sound-proofing, it's possible to build a basic working studio almost anywhere. For many projects, the audio features included in motion graphics software can record too, making everyone's life simpler. Arber says, "For this project we recorded the original voiceover into Final Cut Pro, then used Adobe Soundbooth to tidy up the vocals. The soundtrack and voiceover were mixed in GarageBand, and the final mix was done in Soundtrack Pro, because it offers so much more control. We'll often use GarageBand for both music and sound effects. It's not a high-end tool, but for us it's about the speed and convenience, and the fact that the latest versions can save uncompressed broadcast-quality audio."
Anyone expecting to find a mixing desk the size of a battleship will be disappointed. "In terms of hardware we have some Samson Studio Condenser mikes and an Alesis USB mixer. It's quite budget as equipment goes, but we can still get top results. We did a lot of research before buying the hardware, and we have essential accessories like pop shields and de-essers, which make sure close-up vocal recordings don't blow out the bass, and the esses in recorded speech aren't too obvious. Part of the secret is that I know what I'm doing - I've been in bands for 30 years, so I know what to do with compression and other audio processing."
Building a soundscape
Creatively, the challenge isn't just to get top audio quality, but also to mix and match sounds to create a result that supports the project without being intrusive. This is easy enough to do at the entry level, with entry-level results. But expert sound designers have trained ears and a knowledge of the tools, meaning they can offer more creative solutions.
You can hear this on any movie score. The sound effects are mostly dubbed on afterwards and the squealing tyres in a car chase will have been tuned and timed to create a deliberate soundscape. Newcomers to sound design can learn a lot by watching movies: ignore the plot, the dialogue and the special effects, and listen to where the sounds are placed and how they're put together. Movies usually aim for an exaggerated hyper-realism, and the motion graphics aesthetic will be leaner and more abstract. But it can be helpful to start thinking about sound in a way that contributes something specific and tells a story, rather than just as a filler sample.
On the music side, the options are in-house composition, library music or commissioned work from a composer. Loop libraries make it easy to put together beds and beats for motion graphics, but a trained ear can create the difference between something that does the job and a score that stands out. Loops also tend to be limited in their tunefulness. It's easy to find a beat, but it's less easy to find enough phrases that can be chained together to make a song.
Library music is a time-honoured alternative, but by its nature it's often bland and doesn't draw attention to itself. There are hundreds of library music companies and most will happily send a set of CDs or download links for free on request. Library music is pre-composed - it can be slotted into a project for little or no cost. Composers and licensing companies are paid from performance royalties - each time the spot is played, a fee is collected. There's nothing else to learn about it: dub it into your track and you're done.
But what about custom music? Lester Barnes of Dreamscape Music is a composer who has worked on projects for Reebok, Burger King and Swatch, so he's very familiar with the commissioning process. "Initially there's an intense week of demos," he says. "The agency will approach a number of composers and then pick one after a pitching session. At that point the ad won't be finished, but there will be a temp track, which shows the direction that's needed, and I'll be briefed to follow the pace, the flavour or some other elements. Music has structure - it may have to resolve at some point, and often the ad will be cut and recut before I can do a final arrangement. Sometimes it takes a number of repeats before it's finally finished."
"Financially, there's a production fee for the recording, which can be big or can be token," he adds. "Then there's a licensing fee for permission to use the music in various territories and media. And finally there are performance fees for each play, which aren't huge but can soon add up for a global campaign. The initial fees come out of the production budget. The ad agency - which ultimately means the client - pays for each play."
Given today's creative tools, this kind of approach is starting to seem old-fashioned. TV animation and film projects still need a fixed score and will approach it as a creative element with hard edges - even when the composer will be the one pushing the edges on demand. Similarly sound design for the web will usually default towards a small collection of canned sounds triggered by specific events. Sometimes there may be a score, but it won't usually be variable.
Where sight meets sound
Games design offers an even richer experience. Sounds and music are triggered by specific events and are mixed and remixed live. This is the future of sound - dynamic sound and music that responds to user input and perhaps to other stimuli. But it's also possible to blend motion graphics and sound in the other direction, creating video action from sound triggers. The Advanced Beauty project commissioned a number of artists to use the Processing environment to create unique animations. Some of them were audio-reactive and responded to music that had been created by Simon Pyke of Freefarm.
Pyke has produced idents for S4C and Sky, and used Logic to compose the music, with his own custom-recorded sound libraries rather than canned music loops. He says: "For this project we did quite a few sessions with musicians, including an opera singer, a guy playing clarinet and a drummer. It was kind of an organic process. I'd write music in the studio and then get people to play over it. There was some improvisation too, which we cut-and-pasted together afterwards as a kind of collage."
If Processing inspires you, you'll also want to look at Cycling '74's Max/MSP and Jitter package, which combines a different set of programmable audio and video tools. Like Processing, you can use it to build complex interactive audio and video machines, although with more of a twist towards video driving audio, rather than vice versa. Simpler audio-reactive tools are also built into After Effects and Autodesk's Combustion, but these tools are often underused in motion graphics - remember, Flash isn't the only option, and anyone who wants to push beyond its default, slightly flat look can gain real creative benefits.
If there's a bottom line to working with audio, it's that creativity makes as much of a difference as it does in visual design. As interactivity increases, the old stand-by of a few loops and bleeps will become as popular as the idea of presenting a client with some cheap clipart. Tomorrow's designers won't just be sound-literate, they'll be comfortable building interactive sound and video systems, which will include both visual and audio design elements. For designers with no experience of audio there's going to be a lot to learn. But it can lead to completely new kinds of creativity. 'Studio' will stop meaning 'recording studio' or 'design studio' and will start to mean whatever you want it to mean.
