Dolby Podcast Episode 29, December 6, 2007
Marty O’Donnell, the sound director and composer for Halo 3, talks to Jack and Craig about how the Bungie team creates the surround sound effects for Halo 3 as well as all kinds of interesting bits about sound in video games.
Hear them all: Listen to Dolbycast on iTunes® or subscribe using your favorite RSS reader.
Mentioned in this Episode
Jack Buser: Hello! And welcome to Dolbycast, the insider's guide to entertainment technology from the experts at Dolby Laboratories. I'm Jack Buser.
Craig Eggers: And I'm Craig Eggers.
Jack: And we're here to give you the straight talk on everything you need to please your ears. Hey, Craig, how are you doing today?
Craig: Jack, I'm excited about today's podcast.
Jack: I know. I am, too. This is going to be great. Today we have, here with us--actually, here with us virtually, over an ISDN line...
Craig: Virtually. Virtually with us.
Jack: We have Marty O'Donnell, who is the audio director, composer--actually, audio director/composer, but he tells us that the...
Marty O'Donnell: The slash is silent.
Jack: The slash is silent.
Craig: [laughs]
Jack: Over at Bungie. And we're super-excited to have him here.
Craig: And for our listeners, who are not familiar, Bungie are the people who bring you the great soundtrack and great experiences on that video game called Halo.
Jack: [laughs] That's right. So we're really stoked. Now, Craig, you've played Halo.
Craig: It's cool.
[laughter]
Craig: It's very cool. And it's even cooler in surround sound.
Jack: [laughs] Absolutely. Well, listen, we're going to talk about surround sound. We're going to talk about audio. We're going to talk about what's going on with Bungie, because, as we're told, Bungie is now its own independent company.
Craig: That's right.
Jack: And we're going to just basically talk about all things video games, Marty, when we come right back.
[Musical interlude]
Craig: And we are back at Dolbycast. I'm very proud to introduce our next guest, Mr. Marty O'Donnell, the person responsible for those incredible soundtracks and that incredible experience that we call Halo.
Jack: What's going on, Marty?
Marty: Well, I'm just having a good time here in Seattle .
Craig: Welcome to Dolbycast.
Jack: Excellent.
Craig: Is it raining there, Marty?
Marty: No, it's not raining at all.
Jack: Very cool.
Marty: It's a little overcast, but it's nice. It's a nice day.
Craig: It's going to rain. [laughs]
Jack: [laughs] Well, we're so happy to have you here on the show. I know you guys must be extremely busy. You just managed to get out the door Halo 3, and I've got to say: what a game. How was the experience of building Halo 3? Was it really, really intense, or did you guys have all the time in the world? Or what's the deal?
Marty: It's always really, really intense doing a Halo game. Actually, doing any kind of video game is extremely intense. And I just want to also just mention that I'm here representing my boys, Jay Weinland and C. Paul Johnson, who are right-hand men for me. Can I have two right-hand men?
Jack: Yeah, sure.
Craig: Yes. On Dolbycast, you can have that.
Marty: I have a right and a left.
Jack: [laughs]
Marty: [laughs] OK, good. But they're incredible sound designers and audio engineers, and they're both musicians, and I really depend on them for everything. And they would be here too, except that I'm the only expendable guy in the office...
Craig: [laughs]
Jack: [laughs]
Marty: And they're actually working on something very important, so that's why I'm here.
Jack: Very cool.
Marty: So yeah, it's intense.
Jack: Well, so there are three people on the team. How do you guys divvy up the work?
Craig: Yeah.
Marty: I'm the director, so I basically just tell them all sorts of things, and then they go do what they want.
Craig: Interesting.
Jack: [laughs] So, how do you get a game made?
[laughter]
Craig: [laughing] Exactly.
Marty: Officially, Jay is the audio lead on the project, and C. Paul is a really good sound designer, and I'm in charge of, basically, the audio vision of the entire project. I mean, nothing is going to come out of the speakers that I haven't approved of or thought through or something, and that includes the music and the dialog.
So I direct the actors, for the most part. I work with the story guys on story. A lot of times, I'm editing together some of the major, dramatic cinematic moments. And C. Paul and Jay will work on the Foley and all the sound design for weapons, vehicles, characters, animations. I write most of the music. I also work with my partner, Mike Salvatore, who is back in Chicago . And he and I sort of do what I call tag teaming on the music and collaborate together on a lot of it.
Craig: So, what I find interesting, first of all, Marty, is the concept of people sitting down and thinking about the weapons, the vehicles, the aliens--all the different things that are part of this whole concept called a video game.
Marty: Mm-hmm.
Craig: Can you talk about how you interface with that and how that idea process..?
Jack: Yeah, how do you create the sounds?
Craig: Yeah.
Marty: So I come from a background of doing commercials and television and film, and one of the things that always frustrated me in my many years in that career was that the audio tasks were always split up amongst lots of different people--all very talented people and very specialized.
And when it came to the end of the whole process, usually the director or, like in television's case, the producer would be sitting there making the decisions about the final mix. There was a re-recording mixer, but a lot of times the re-recording mixer was getting all the tracks and hearing them for the first time at the mix.
Jack: Right.
Marty: And there was always this... Well, I don't care if I make any enemies, but there was always this excuse...
[laughter]
Marty: There was an excuse like, "Hey, the re-recording mixer has fresh ears." And of course, they were all really talented, so it's not like they ever did bad. But they hadn't been involved in the project. They didn't necessarily have a vision for the project, whatever it might be, whether it's a commercial or a TV show or a film.
Jack: Right.
Marty: And for the most part, that was in the hands of the director, who, frankly, many times, depending on who the director is...
Jack: Don't make those enemies.
Marty: Yeah, I won't make any enemies.
Jack: [laughs]
Marty: But many times, directors might not be the best audio judges possible.
Jack: So you guys work differently, is that right?
Marty: Yeah.
Jack: You work sort of as the director. Explain the equivalent of a re-recording mixer in the world of video games, then.
Marty: Well, wow. Now that's actually a good question, because there's really no equivalent to that, because what happens in real-time gameplay is that the game itself is being mixed depending upon what the user is doing. So we essentially have to build what I call an artificial intelligent mixer and have it as part of the audio engine, because the game itself actually has to say, "Oh, wait a minute. I'm supposed to be playing this piece of dialog, but somebody just threw a grenade which is about to explode, so what do I do?"
Jack: Right.
Marty: So we make all those decision ahead of time, as good as we can, saying, "Duck this event three dB if this other thing is happening." And there are a lot of those if-then-else kinds of programming statements inside of our audio engine, hopefully to give the user the best audio experience possible.
Jack: Well, there must be a lot of back and forth, where you play the game and you say, "OK. That didn't quite work." And then, do you go back to the drawing board? How do you communicate? Because there must be somebody sitting there writing the artificial intelligence code, I guess.
Marty: Yeah. We have an engineer, or a couple of engineers, in Halo 3's case, which work on what we call the audio engine and all the code for the audio. So that's happening right from the get-go. And Halo 3, of course, is an iterative engine. It's been built for almost 10-11 years, really. Some of the original concepts and some of that original code go back quite a ways...
Jack: Wow.
Marty: And we just keep trying to make it better and better. So it's really nice to have an in-house team, like us, and someone who is the prima donna audio director visionary, like me, [laughs] to try to make sure I get my way with both programmers, game designers, writers, cinematic director, all that stuff. So, right from the beginning of pre-production, I'm sitting in there with these guys and contributing and making sure that audio never gets short shrift, if at all possible.
Craig: I guess the nice thing, though, Marty, is you're not reinventing the platform every time you're creating a soundtrack. You're utilizing content that you created for Halo 1, Halo 2, and portions of that are probably integrated into Halo 3 with any new sound effects that you've created. Is that right?
Marty: Yeah, that's right, except that, almost always, there's enough technical improvement from one game to another, or from one platform to another, where the content itself, even though we might be basing something that we've done on something previous, we kind of like to reapproach it and maybe even recreate it. So a lot of stuff is recreated, and then there's a lot of new stuff. But it's because, with each iteration of the game, fidelity and lots of other things just get better and better.
Jack: All three Halo games have made tremendous use--I mean, this is Dolbycast, so I've got to say this...
Marty: Oh, absolutely.
Jack: Have made tremendous use of Dolby Digital surround sound.
Marty: Yes.
Jack: In fact, our listeners, if you've listened to enough of our podcasts, know that there was quite some time where I was involved with our games initiative here at Dolby. And I remember when the first Halo came out on the original Xbox, which was our go-to title which we carried around the industry and said, "This is what can be done with 5.1 sound in a game." How has that advanced between those early days, when I was carrying Halo 1 around, and what you're doing now with surround sound with Halo 3?
Marty: That is a great question, Jack. Thanks for asking that.
[laughter]
Marty: And just to give a little background on that, it's amazing to me how far we've been able to come and how excited I remember being. I had done some work in the game industry. I started in about '96, '97, coming from film and television, and I was a bit shocked at what the fidelity and compression and what some of the common practices were at the time in the video industry. And it was a little depressing.
Jack: [laughs]
Marty: So, once 2000, 1999 rolled around, and we started hearing rumors of the Xbox, and then I think it was after Microsoft purchased Bungie, which was 2000, and we were always excited because we knew that there was this carrot dangling in front of our eyes that said, "It's going to have real-time 5.1 Dolby Surround." And I'm like, "No, no, it's not. There's no way you could do that."
Jack: [laughs]
Marty: I was very skeptical. And then when we came out, we saw what the plan was and how it was going to work. It was a thrill for me. I knew that if we could take advantage of it, it'd be great.
Unfortunately--and you maybe remember some of this--the hardware wasn't all completely in the box [laughs] when it needed to be. And during that summer before the launch of the Xbox, we had the specs and the code that was supposed to help us work to actually create everything in surround sound, and that it would work, but we didn't have it actually working. It wasn't quite in the box yet. So some of our engineers came to me one day and said, "Hey, Marty, I think we're going to just have to walk away from this, and we'll just do stereo. Sorry."
Jack: No! No!
Marty: Literally, I was screaming. I went running over to the Microsoft hardware guys and pounded on the desk, like, "Please! You've got to get this in. You've got to convince my engineers that it's going to be OK." And thank goodness it all worked out. [laughs]
Jack: It really did. I mean, that was such a landmark title when it came out. You guys were really making such intelligent decisions, extremely early on, about what to do with those surround channels, what to do with the center channel...
Marty: Yeah.
Jack: All of these different creative decisions. A lot of those were made, I would maintain, by you guys, with the original Halo.
Marty: Yes. I would maintain that, too. [laughs]
Jack: [laughs]
Marty: We didn't really have any model to go by, in the sense that we certainly could look at how movies were being mixed, and I talked to a lot of guys in the movie industry and other things and talked to them about mixing for surround.
The difference was that Halo, and a video game, is real-time surround sound--the fact that a rocket shooting over your head actually could explode behind you, and that's something we can have happen in real-time. However, if you happen to turn and look at it explode, now it's going to be in front of you. So all that stuff has to happen in real-time.
What's great about a game like Halo is that, because it's primarily a first-person camera view--I mean, to me, this is what surround sound was meant to be.
Jack: Right.
Marty: Jack, I think I've told you this before, but when I'm in a theater and it's a surround sound mix, you can tell sometimes when the surround sound mixers are maybe going a little too far or trying to be sort of adventurous. And I'll hear something fall behind me, and I'll actually physically turn around, because I thought like the lady behind me dropped something, right?
Jack: [laughs]
Marty: And they're like, "Oh, no, no, no. That was supposed to be the blah-blah-blah." But because the screen is in front of me, and I'm really just from a camera viewpoint--and I don't mean to diss all your work with the film industry because I'm sure it's absolutely brilliant--sometimes I think it's hard for the surround sound film mixers to really create a completely immersive surround environment, because the audience is sitting very far away from a screen, and it's hard to make that really work.
As a matter of fact, I think films sometimes work better in a home theater, where you're sort of closer and it makes a little more sense. This is just my opinion, of course...
Jack: [laughs]
Marty: But certainly, if you're playing a video game, and the illusion is you are the camera, first-person--it's looking out of your eyes, and you're in the middle of this world that you can turn around and spin and run and jump--then surround sound makes just an incredible amount of sense, and it works great.
Jack: I've always maintained that games really do make probably one of the most compelling home theater demonstrations. If you have somebody and you really want to show off your system, throw in a copy of Halo 3 on the Xbox 360 and hand somebody a controller and say, "Go!"
Marty: Absolutely.
Craig: Yeah, I think that's the first thing you notice. When you bring a game into a home media center or a home theater, the first thing you notice is that first-person perspective that, as your direction within the video game changes, so does the audio mix. And I have to ask
Marty: how does that happen? I mean, in a movie, literally, you have sound producers creating a surround track. And it's pretty static: once you create it, you create it.
Marty: Right.
Craig: But a game, literally, you have so many different interactive options in a game. When you walk past an object, like a tank, you might hear the engine in the back of the tank, and at the front, you might hear the treads or some other noise related to that.
Marty: Right.
Craig: And my perspective changes every time I play the game. How do you actually mix for that, and how do you prepare for that interactive experience?
Marty: Well, that's actually the fun part. I think, like Jack was saying, is that, to some extent, we had to kind of invent it back in 2000 and 2001. There wasn't a lot of experience to figure out "How do you do this?" The nice thing is that 3D graphics, of course, had been around for a while, and our engineers are working in XYZ space all the time.
Craig: So, Marty, for our listeners: XYZ space.
Marty: Yeah. Yeah. Well, XY, of course is just a two-dimensional graph, so Z is that third dimension.
Jack: Depth.
Craig: Right.
Marty: Let's say height, length, depth.
Craig: OK.
Marty: So it's like, now, instead of looking at a rectangle or looking at a square, you're looking at a cube. You see what I'm saying?
Craig: Mm-hmm.
[laughter]
Marty: Anyway. [laughs]
Craig: We knew that, we just didn't think...
Marty: I know, I know. I guess I had never thought about how do I explain that.
Craig: Yeah. You did a good job.
Marty: If you think about, let's say, the engine for the Warthog, there are so many things. The Warthog is really a system. It's a 3D object. And what our artists and designers do is they make sure that they have actual, defined locations on the Warthog--which is our Jeep, right--that are called markers.
Jack: What a Jeep it is.
Marty: Yes, [laughs] it is.
Jack: [laughs]
Marty: On those markers, it's essentially like little, tiny speakers on each one of those markers, and we say, "On this marker, it's the left channel of this cool engine loop, and on the right marker is the right channel of a cool engine loop." But now they're existing in 3D space.
Jack: 3D space. The sound is actually panning in 3D space...
Marty: Right.
Craig: So you're mixing possible engines, then. Is that right?
Marty: Yes. Well, actually, what we're doing is we're creating--now, of course, remember, because it has to be responsive to a user, we have to say what happens when you step on the accelerator and what happens when you change gears and how do you cross-fade between all these samples...
Craig: Wow.
Marty: So we have tons of mono and stereo samples, and now we even have actual 5.1 samples, which we can especially put in ambiance.
Jack: Yeah.
Marty: But seriously, a stereo sample is great, because we can place that on an object and give it space between left and right.
Jack: Right.
Marty: As soon as we throw it into our audio engine, it's instantly coming out of a 5.1 audio path. So the engine itself is making the calculations. Well, based on where that location of where that little marker is, what speakers should it be coming out of? And then, based on the location of the camera or the player, as they walk around or jump over, or get driven over...
Jack: Yeah. [laughs]
Craig: Get run over.
[laughter]
Marty: Get run over by the Jeep. Where are those speakers? What is happening with that audio? And it's all happening with spatialization, with occlusion and obstruction, with... Not portamento. Hello. [Makes sound of plane flying by]
Jack: Doppler.
Marty: Doppler. Thank you.
Jack: There you go. [laughs]
Marty: [laughs]
Jack: Any time. [laughs]
Marty: Thank you very much. So all that's happening, yeah.
Jack: Marty, that just sounds amazing. I mean, I've built this mental picture of what all those audio cues must look like in that virtual world of Halo...
Craig: God, it's got to be so much work.
Jack: Listen, we're going to go to a break. When we come back, I want to bring us back into the real world and talk a little bit about how you get all those great orchestral sounds inside the music, when we come right back.
Craig: And I want to know what Marty thinks when he goes to a retailer the night before the game is released and sees hordes of people waiting in line to get the game.
Jack: Yeah, totally. [laughs] All right. We're going to go to a break. We're going to talk about the day before the launch and working with an orchestra, when we come back.
Marty: Great.
[Musical interlude]
Announcer: You're listening to Dolbycast, with Jack Buser and Craig Eggers. Email them at dolbycast@dolby.com.
Craig: We are back at Dolbycast with Mr. Marty O'Donnell of Bungie. Welcome back, Jack.
Jack: Welcome back, Craig. [laughs]
Craig: [laughs]
Marty: Hello, guys.
Jack: Hey, Marty. How you doing?
Craig: Hey, Marty.
[laughter]
Jack: Before we went to break, I wanted to ask a question about the music. Well, I want to talk about the music in general, but let's begin specifically with that great intro theme. It's sort of changed over the three Halos. How did you record that? Was that with a live orchestra?
Marty: Yeah. And actually, that was recorded in 1999. I was still, at that point, doing a lot of jingles, so the monks that sing [chants the Halo theme] are me and my partner, Mike, and three of our favorite jingle singers in town that I got in. At that moment, we were just trying to get a nice three-minute piece that was accompanying a promotional piece for Halo that no one had ever seen before.
That all came together relatively fast. I had a small orchestra come in, and I overlaid them with what we call over-dubbing: layering the strings over themselves several times to get as big a sound as possible.
Jack: And the guitar in Halo 2, rumored to be...
Marty: The guitar in Halo 2. No, it was not... [laughs] I think we ended up with four different guitar players, but the main guy was Steve Vai, which was so much fun. He came up to Seattle . And Nile Rodgers is the producer who has been the distributor for all the actual CD soundtracks from the game, and he is good friends with Steve Vai. Nile actually called me and said, [speaks with gravely voice] "Marty. Hey, who do you want for guitar? I think you should get somebody."
Craig: [laughs]
Marty: That's my impression of Nile Rodgers.
Jack: [laughs]
Craig: [laughs]
Marty: He's the ultra-cool producer. And he goes, [speaks with gravely voice] "I can get you a Jeff Beck. I can get you Steve Vai."
And I'm like, "Holy mackerel."
Craig: Wow.
Marty: I was like, "Well, OK. I'll take Steve Vai."
Jack: [laughs]
Marty: I mean, I would have been thrilled with Jeff Beck, too...
Craig: Stevie Vai is one of those guys that, if you're a guitar player, you just want to stop playing after you hear him.
[laughter]
Marty: I tell you, as a musician, I felt like, why should I ever play music?
Craig: Yeah.
Marty: I mean, Steve was just so just fluid. He picked everything up, and his facility is unbelievable. And he's a super-nice guy, a pleasure to work with, and his two sons are big Halo fans, so that was fun.
Craig: So, Marty, I have a trivial question for you.
Marty: Mm-hmm.
Craig: Flintstone Kids, 10 million strong and growing.
Marty: Yeah. [laughs] [Speaks with high-pitched voice] And growing...
Yeah, I wrote that.
Jack: [laughs]
Craig: [laughs]
Jack: I love that you wrote that!
Craig: Isn't it cool?
Jack: That's so cool.
Marty: [laughs] I told you I did jingles, so that's a real jingle.
Jack: That's very cool. Now, I have one for you, and this is about the character dialog. And I remember working with you, back on Halo 1. Correct me if I'm wrong, but I seem to recall a lot of the character dialog, as you're playing the game in Halo 1, was people that you had recorded around the office and things like that. And there were even some Easter eggs or something you could find. Is that true?
Marty: Yeah. Most of the good dialog, of course, was actual professional SAG and AFTA... Actually, whatever. It doesn't matter which union it was, but it was professional union talent. But there were always a lot of extra voices that we would need. And especially for the voice of the grunt, we used a guy who is also our writer, Joe Staten.
Jack: There are some pretty funny lines.
Marty: Oh, there are a lot of funny lines. I can't even remember all of the things.
Jack: [laughs]
Marty: But with every Halo game, what we have is a system that allows you to--if you do something to a character or you walk by a character or you see a character or they see something, or whatever, there's all these what we call triggers that could cause them to choose in the suite of pieces of dialog for that trigger. There could be 10 different lines, and each line is separately weighted and randomly chosen.
Jack: Wow.
Marty: So I can have what we call a skip fraction on something that would be very, very strange and funny and make sure that that will only happen once every...
Jack: 1, 000 years.
Marty: 1, 000 times.
Jack: [laughs]
Marty: So that, as people play the game, they think they've heard everything. And they play for the fourth time, fifth time, sixth time, and suddenly they hear a character come out with something they've never heard before.
Craig: That's cool. That's cool.
Jack: That really brings it to life, I bet.
Marty: Yeah.
Jack: It makes it feel like everybody's real people around you, right?
Marty: Yeah. And we have fans out there that collect lines of dialog. And on the Internet, there's a database of Halo dialog chunks. And so people continue to look for something that's never been heard before, which I think is great.
Craig: So, Marty, the night before opening day...
Marty: Yes.
Craig: What's it like to go out there and see the lines waiting at retail, and knowing that something you created, something that, quite frankly, enriches and excites so many people, you're involved with?
Jack: [laughs]
Marty: I know. What's amazing to me is that this is really sort of my second career. As you know, like I've said, I did jingles and films and stuff like that. By the time I got to the age--I'm in my mid-40s, and I'm starting to work on games--I never thought that there would be some sort of rock star status for me.
Jack: [laughs] Right.
Craig: Yeah.
Marty: And now we're playing music at the Hollywood Bowl. I was in London, and there's the Halo music being played and a standing ovation and all this stuff.
Craig: That's cool.
Marty: And then you go to the stores before the launch. So, when Xbox launched in 2001, there were a lot of people looking forward to Halo. And I remember going to the local, I think it was the EB, and we saw this giant line. People went in, and they were buying the Xbox, but the very next thing they had to do was buy some software. And the first guy in line bought two copies of Halo.
Jack: [laughs]
Marty: So we were like, "Woohoo! A 200% attach rate!"
Craig: There you go.
Jack: [laughs]
Marty: It was great.
Jack: That's great.
Marty: It went down from there, but it was good.
Jack: Marty, I've got to say, we've got to wrap it up. It breaks my heart because it's been so wonderful having you on.
Marty: Sure.
Craig: Definitely, definitely.
Jack: I can't thank you enough. This was just too cool, too cool. Actually, the reason why we've got to end this is because I've got to get home and finish my Halo 3 game. I haven't yet beaten it, but it's on my...
Marty: Oh, good.
Jack: [laughs] Over the Thanksgiving break, that's my plan.
Marty: Excellent.
Craig: You know what? What I learned from this podcast is there's a lot of work that really goes into these video games, and a lot of thought about how you stage people, how do you interact with the whole environment. What you do is incredible, Marty.
Marty: Thank you, guys.
Jack: Totally. Totally. We're huge fans.
Marty: Thank you, guys. I appreciate it.
Jack: Thanks so much for coming on.
Marty: Thanks for your help, too.
Jack: All right. Take it easy.
[music]
Back to Top
Back to Dolbycast main page
