Interview with Richard Jacques

Gaming Sound—Taking Gaming to a New Level

Computer games are constantly evolving to include more realistic graphics, better storylines and game play, and improved audio. Developers are continuously searching for an enveloping experience that will draw players back to the game time and time again.

Dolby® has been involved with games since 1992, bringing its experience of surround sound in the cinema and the home to the game console, helping developers add that extra something to the game and really wrap the player in a 3-D gaming experience.

As game audio has developed, so have game audio developers. Dolby spoke to leading games composer Richard Jacques to see how he is using surround sound in his work.

Richard Jacques

Music and computer gaming had been a passion of Jacques' since he was seven years old, when he first started composing. He played Sega® Megadrive and PC games throughout his university years while he was studying for a classical music degree.

From there, it was a logical link to take a job at Sega as a composer. This involved music, sound design, and dialogue production, and the first game he worked on was Shinobi X (a Japanese ninja game) on the Sega Saturn.

These audio tracks and sound effects were still a long way from surround sound. Jacques had always been a big fan of surround sound in the cinema and had a large collection of laser discs featuring Dolby® Surround. He knew that one day surround sound would make it into computer games.

In 1997, when Dolby, in cooperation with Digidesign, released tools to mix in surround sound, Jacques used them to test and run computer game demonstrations. He believed that DVD would take off and surround sound content would become readily available—and that it would only be a matter of time before consoles picked it up.

"Gamers get totally immersed in games," Jacques said. "In games you control the action, so in my opinion game audio is even more important than film audio. You are the character, the director, and the center of the action, so audio mixes need to revolve around you.

"Now surround sound in the home is more affordable, it's really important to the action that surround is used. We're wasting effort if we don't add surround sound to games—and if people at home don't plug it into a 5.1 system and listen to it!"

Although creating soundtracks for games became his core business, surround sound still haunted him. Jacques tried some tests on the Dreamcast, but the CPU just wasn't available and he couldn't create surround tracks without a significant hit on the main game engine.

Jacques' first game in terms of surround creation was Jet Set Radio Future, which was produced mainly in Japan for the Microsoft® Xbox®. It had a cartoony, urban feel and was the first game to use 3-D cell-shading graphics. Jacques created various tracks for the soundtrack, which utilized Dolby Digital 5.1 surround, and felt that he had found his niche.

"It was in 2001, when the Xbox came out with 5.1 capability, that gaming sound really took a leap forward, but Pro Logic® II on PlayStation® 2 and GameCube™ provides an effective alternative while we wait for advances on the next generation of consoles."

This 5.1 capability on games consoles enabled Jacques to work on his favorite games to date: the Headhunter® series. The Swedish team at Amuze wanted to create a truly cinematic game, making realism a key production value. There were scriptwriters, sound designers, a score composer (Jacques himself), and, of course, a director. The sound designers, Dominic Gibbs and his assistant Gregg Wilson, were accustomed to working in Pinewood or Shepperton studios on sound design for movies. They hadn't worked on a game before, so they were not used to working within the constraints of hardware. This was what made the working relationship so important, as Jacques took care of the technology side, allowing the sound designers total creative freedom. Headhunter's music has a cinematic theme with a live orchestra recorded at Abbey Road studios.

"I was trying to emulate the scope and depth of a cinematic score," said Jacques. "I worked with the other sound designers to ensure that the surround mixing of effects and score was balanced. We decided that effects were better used in the surround and center speakers while the score created a more cinematic feel from the main front left and right speakers."

"Headhunter: Redemption has over two hours of cut scenes with full surround mixes and they are used to the best effect," Jacques said. "Even film mixers are sometimes too scared to use the surrounds, and not many people are doing much with them. In Headhunter, we used them to create a dark space and a dramatic effect, drawing the player in and keeping their attention.

"I couldn't have delivered this type of effect without the advancements in surround sound technology—primarily the Xbox having 5.1 capability. It's so easy to do and to deliver the files, and causes no problem to the main game engine because of the built-in 5.1 Dolby Digital real-time encoder."

Jacques also finds that Dolby Pro Logic II on the PlayStation 2 and GameCube makes it easy to create surround tracks. Dolby's European games manager, Valerio Faggioni, has been instrumental in ironing out problems. Faggioni took the 5.1 stems from Headhunter's surround mix and encoded them in Dolby Pro Logic II using a Dolby DP563 encoder.

Jacques also found Pro Tools®, Logic® audio, and Dynaudio monitoring extremely useful in creating his surround mixes.

Problems and solutions.

Acording to Jacques, people in the games industry are still concerned about volume standards, which is opening up useful dialogue among audio creators. Although reference tables for dubbing for film and TV exist, volume levels still remain an issue in games. Video standards have improved enormously; audio is just catching up slowly. Jacques says that audio budgets and schedules still need to be understood better, as development studios often put an unrealistic maximum limit on the sound budget, then say they want surround sound mixes and live musicians which they previously hadn't thought about.

"If they don't push the resources forward, their games are just going to sound worse than the rest of the titles on the shelf, and that equals bad reviews, which can be the kiss of death to the sales of a game, "Jacques explained. "They need to know what good audio really needs in terms of budget. It cost £250,000 for Headhunter's sound—and this included compelling features like live newscast recordings, multiple voice actors, Foley artists, and the orchestra."

When quizzed about the future of audio, Jacques describes large-scale scores, other musical styles, licensed tracks, and sound design as being integral to the game play (for example, creatures creeping up behind you that you can hear before you can see, as well as interactive mixing).

"Films have huge audio budgets (around a minimum of 5 percent of the total film budget), and we have to make the players' experience even more immersive than that of a film," Jacques said. "There is no denying that quality audio costs, but why should we have to try and better the production quality of a film in a tenth of the budget?"

Jacques believes that with the next generation of hardware there will be no going back to stereo. He's already upgrading his studio with a 7.1 system, so however many surround speakers gamers have, they can hear properly mixed surround tracks.

He also sees a healthy future for licensed music in games, and a growth of music-driven games such as Singstar™ and EyeToy™: Play 2. He's recently been working on an air guitar game.

"There's a lot in the pipeline for me because audio is being taken more seriously and clients know that I am obsessed with quality, constantly pushing the boundaries," he said. "Currently I'm working on the Starship Troopers™ score for Empire Interactive, EyeToy: Play 2 for Sony London Studio, and have also just finished remixes for Sega's OutRun 2, as well as a PSP game for Sony. But I can't tell you about that or I'd have to kill you!"

What Jacques is hoping to create in future games is a fully interactive orchestral soundtrack, where eight or more streams of music are cued and mixed interactively, so the soundtrack is much more tightly integrated with the game. "I have the design in my head for a fully interactive and dynamic multi-layered, multichannel surround score that does not limit the composer's freedom and creativity. I can't wait to get my hands on next generation consoles!"