Kraig Scarbinsky
March 5, 2018
From 8-bit to orchestras: How does video game music affect you?
Subscribe
Have you ever played a video game? Of course you have, what a silly question.
You know how important music can be when it comes to gaming. But what if you choose to play without music? Or, what if you replace the music with your own soundtrack? How does that affect your playing?
We’re going to dig into the psychology of video game music, explain how the interactivity of video game music works and figure out what “8-bit” means.
• More from Dr. Siu-Lan Tan: What’s the secret to high scores on video games?
• And find out more about Ochen Kaylan’s 8-bit music software Chip64.
• And check out Top Score, a podcast from Classical Minnesota Public Radio all about video game music!
This episode was originally released on October 14, 2014. Listen to that version here:
From 8-bit to orchestras: How does video game music affect you?
by Brains On!Audio Transcript
Download transcript (PDF)
HENRY GOLDMAN: Should I say it now? You're listening to Brains On, where we're serious about being curious.
CHILD: Brains On is supported in part by a grant from the National Science Foundation.
[MUSIC PLAYING]
MOLLY BLOOM: Recognize these tunes?
[MUSIC PLAYING]
You probably do if you've ever played Super Mario 3D World, Skyrim, or Minecraft.
HENRY GOLDMAN: But it wasn't always a given that your games would have a soundtrack.
MOLLY BLOOM: Board game certainly didn't.
HENRY GOLDMAN: And early arcade games--
MOLLY BLOOM: The predecessor to video games.
HENRY GOLDMAN: --didn't have music either.
MOLLY BLOOM: So how did it come to be here?
HENRY GOLDMAN: And what kind of effect does it have on you?
MOLLY BLOOM: We'll find out now.
HENRY GOLDMAN: Keep listening.
[ELECTRONIC MUSIC]
MOLLY BLOOM: I'm Molly Bloom, and my co-host for this episode of Brains On is 12-year-old Henry Goldman.
HENRY GOLDMAN: Hello.
MOLLY BLOOM: So, Henry, you play video games.
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: And do you ever the first video game you ever played?
HENRY GOLDMAN: Hmm, the first video game I ever played was probably Cube Runner, back when I was four or five.
MOLLY BLOOM: What's Cube Runner?
HENRY GOLDMAN: It's this game where it was kind of like a simulation of a car, almost, except you were just an arrow. And you would have to tilt the phone or iPad or whatever device you had back and forth trying to avoid all of these just random cubes that would pop up on the screen.
MOLLY BLOOM: Was there music in that game?
HENRY GOLDMAN: There was music in that game.
MOLLY BLOOM: Do you remember it?
HENRY GOLDMAN: Oh, it went something like dit-dit-dit. I don't know. It's one of those songs where it's in your head, but it doesn't come out.
[ELECTRONIC MUSIC]
MOLLY BLOOM: So which video game would you say has your favorite music?
HENRY GOLDMAN: Hmm, my favorite music-- ah, this game called Bouncy Slime. It's on the iPad where you're this little slime ball, and you kind of bounce around. It's weird, but it's fun.
MOLLY BLOOM: And why does that have your favorite music? What about it do you like?
HENRY GOLDMAN: I don't know. It just kind of triggers something that makes it fun. The game seems more fun.
MOLLY BLOOM: So it actually enhances the game when you have the music on?
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: Because sometimes--
HENRY GOLDMAN: It gets you pumped up.
MOLLY BLOOM: Right, because sometimes you don't like the music for games, right?
HENRY GOLDMAN: Yeah, and then it bugs you, and then you don't play that game.
MOLLY BLOOM: Henry finds video game music annoying sometimes. So we want to know--
HENRY GOLDMAN: What effect does video game music have on players?
MOLLY BLOOM: Before we get to that question, let's look at where video game music came from. Producer Mac Sanchez is here to tell us more. Hi, Marc.
MARC SANCHEZ: Hey, guys.
HENRY GOLDMAN: Hello. Marc, when does music enter the world of video games?
MARC SANCHEZ: Around 1978, these four chords changed everything.
[ELECTRONIC MUSIC]
Do you know what that is?
MOLLY BLOOM: Space Invaders?
MARC SANCHEZ: Exactly. Have you ever played Space Invaders?
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: You did? You've played Space Invaders?
HENRY GOLDMAN: Yeah. Aren't you a little ship thing at the bottom?
MOLLY BLOOM: Yeah.
HENRY GOLDMAN: And then there's the little rocks, and there's all the thingies coming that are shooting at you, and you don't want them to get to the bottom.
MOLLY BLOOM: Exactly.
MARC SANCHEZ: Yep. Space Invaders-- it started out as a stand-up arcade game, and it was set in this really, really primitive version of outer space. The guy that invented the game-- his name is Tomohiro Nishikado. He decided to put those four chords throughout the game. And the longer you keep your spaceship from getting hit by an alien by one of their lasers, the faster the music would get.
[ELECTRONIC MUSIC]
And just those four notes, those four chords makes things really, really tense.
MOLLY BLOOM: I've never played Space Invaders, but I've played other games where the music speeds up the closer you get to the end. And I get really nervous when it happens. How about you, Henry?
HENRY GOLDMAN: I agree with that. I mean, I think that as more games are starting to pop up, a lot more games are starting to have music that's speeding up.
MOLLY BLOOM: And does it make you feel nervous?
HENRY GOLDMAN: Definitely.
MARC SANCHEZ: I called up a friend of mine. He's a programmer and a musician. His name is Ochen Kaylan. I wanted to ask him about these early sounds, this early music in video games. They're actually made by an 8-bit chip. I don't know if you've heard of that or not.
MOLLY BLOOM: Have you heard the term 8-bit before?
HENRY GOLDMAN: Yeah, it's like-- well, I've never heard of it referenced to the music. I've only heard of it referenced to the pixelated creatures.
MOLLY BLOOM: Yeah, it references what you see on the screen. But the shorthand for that music-- people call it 8-bit-music.
MARC SANCHEZ: Here's what he said.
OCHEN KAYLAN: Inside video games and arcade consoles, there are chips, and transistors, and capacitors, and all those things. Well, those chips run sets of instructions. And there may maybe one chip to do all the video, and so that's what makes all the pixels on the screen light up certain colors. And there may be another chip that deals with the joystick, or the buttons, or something like that. And there's often another chip that just does sound.
MARC SANCHEZ: OK, you guys got this so far? Microchips are inside the games, and each chip has a set of instructions that tells the game how to react.
MOLLY BLOOM: OK.
MARC SANCHEZ: Now, let's focus on the chip that sends sound signals to the game's speaker. We've talked about sound waves in the past, right?
MOLLY BLOOM: Yes, at our music episode. You can listen--
MARC SANCHEZ: Shout-out to the music episode.
MOLLY BLOOM: You can listen to that to find out how our brains actually hear music.
MARC SANCHEZ: And as we learned in that episode, sound travels in waves. And when you think about a wave, when part of it goes up, the other part goes down. Ochen told me that this kind of symmetry, this up and down motion, is ideal for these sounds that are coming out of 8-bit chips.
OCHEN KAYLAN: The binary nature of computers-- 0s and 1s, or off and on are really well-suited to making music because, if you think of a speaker, the cone in the speaker, it just goes in and out. If a computer says, 0, make the cone go in. And if the computer says 1, make the cone go out.
And so if the computer can just send a series of 0s and 1s in some special order, then that can make the speaker go in and out in some special order. And that special order, we call sound. And so that sound might be a violin, or it might be a piano. Or it could just be some alien getting shot.
MOLLY BLOOM: OK. So far, we've got chips getting instructions in the form of 0s and 1s.
MARC SANCHEZ: Binary code.
MOLLY BLOOM: And the chip turns those instructions into sound waves that come through the speaker.
MARC SANCHEZ: Right.
MOLLY BLOOM: But why do the old games sound so old? Why couldn't Space Invaders have a super awesome soundtrack like Skyrim that sounds like an orchestra with violins?
MARC SANCHEZ: It's all about limitations.
HENRY GOLDMAN: Limitations?
MARC SANCHEZ: Limitations and exponents. Are you guys ready to do a little multiplication?
HENRY GOLDMAN: I guess so.
MARC SANCHEZ: Compared to today's microchips like the ones in your phone, or tablet, or a computer that you might have, the chips in Tomohiro Nishikado's Space Invaders-- they might as well have been potato chips. Those early chips could only run a limited, teeny-tiny amount of instructions compared to today before they ran out of room or bits. The more bits, the better the sound.
OCHEN KAYLAN: Bit means how many-- in terms of a mathematical equation, it's 2 to the n power. And so 2 times 2 would be called 2-bit because you're multiplying 2 by 2 just twice. 2 times 2 times 2 is 2 to the third power, or 3-bit. And 2 times 2 times 2 times 2 is 2 to the fourth power or 4-bit.
MARC SANCHEZ: A lot of early video game consoles like the Atari 2600, or early Nintendo systems, or the Commodore 64 computers-- those ran on 8-bit chips. You guys want to do the math on this one?
MOLLY BLOOM: Yeah. Let's see. 8 bits would be 2 to the eighth power--
MARC SANCHEZ: Mm-hmm.
MOLLY BLOOM: So that's 2--
MARC SANCHEZ: Times 2--
HENRY GOLDMAN: Times 2--
MOLLY BLOOM: Times 2--
MARC SANCHEZ: Times 2--
HENRY GOLDMAN: Times 2--
MOLLY BLOOM: Times 2--
HENRY GOLDMAN: Times 2.
MARC SANCHEZ: OK. Multiply that out, and you get--
HENRY GOLDMAN: 256.
MOLLY BLOOM: But 256 what?
MARC SANCHEZ: Ochen?
OCHEN KAYLAN: So remember we talked about the 0s and the 1s make the speaker go in and out. Well, if you only have two states-- if you have in, and you have out-- that's 1 bit, meaning that there are only two instructions or 2 to the first power. And so you only have two states-- either all the way in or all the way out.
If you have something that's 2-bit, meaning that it's 2 times 2, or 4, that means there are four states. So it would be all the way off, or all the way on, or there would be two states in between-- about a 1/3 on and about 2/3 on. Well, 8-bit means that there's 2 times 2 times 2 times 2, or eight times, or 256 states. And so that speaker can be in 256 places.
That sounds like a lot, but music is really complicated. And so if you only allow the speaker to rest at 256 separate places, it approximates sound. When we're just walking outside, and we're hearing birds, and we're hearing trees rustle, we're hearing millions and millions of states between totally on and totally off.
[ELECTRONIC MUSIC]
MARC SANCHEZ: Among his many talents, Ochen has programmed the sounds of 8-bit computer chips into modern music-making software.
MOLLY BLOOM: We're listening to a song made with his software right now, in fact.
MARC SANCHEZ: And there's a whole 8-bit music scene out there. Some musicians, like Ochen, use software and modern computers to make their computer video game-sounding music. And others, the purists out there, they purposely limit themselves by using the original chips.
MOLLY BLOOM: Cool. Well, I guess there's just one thing left to say.
MARC SANCHEZ: Yep.
ALL: Dance party.
[ELECTRONIC MUSIC]
MOLLY BLOOM: You can find links to Ochen's software and music at our website, BrainsOn.org. Speaking of beautiful sounds, we want to hear yours. We're working on another Mystery Sound Extravaganza and want to hear some of the mystery sounds that you think will stump your fellow listeners.
You can email them to Hello@BrainsOn.org. Be sure to both record the sound and your explanation of what it is. You can also send your drawings and questions to that same email address-- Hello@BrainsOn.org. That's what Caleb did.
CALEB: Why are cheetahs so fast?
MOLLY BLOOM: We're going to answer that question during our Moment of Um at the end of the show. And we're going to read the latest group of listeners to join Caleb on the Brain's Honor Roll. That's the way we thank all of you who share your questions and ideas with us. Keep listening.
HENRY GOLDMAN: So how do we get from this--
[SIMPLE ELECTRONIC MUSIC]
--to this?
[ORCHESTRAL MUSIC]
MOLLY BLOOM: Excellent question, but we have something we need to do first. It's time for the mystery sound.
[ELECTRONIC WHISTLES AND POPS]
CHILD: (WHISPERING) Mystery sound.
MOLLY BLOOM: Here it is.
[CLICKING]
Any guesses?
HENRY GOLDMAN: It sounded kind of like a keyboard clicking.
MOLLY BLOOM: OK. OK, you're on the right track. But think about it a little bit more. We're going to come back to it later in the show. And while you and our listeners puzzle over it, we're going to get back to that question that you just asked. How did video game music evolve?
Well, as the technology evolved, so did the music. Logic came into the picture. The music went from playing at the start of levels and just looping over and over to having to make decisions about where it was going to play.
HENRY GOLDMAN: If Mario eats a mushroom, then play this music.
[ELECTRONIC MUSIC]
MOLLY BLOOM: If Mario jumps on Yoshi, then add percussion.
HENRY GOLDMAN: If Mario jumps off Yoshi, then remove percussion.
MOLLY BLOOM: So it's this logic in the software that makes the music interactive. To explain more about how that works, we turn to one of our podcasting buddies.
EMILY REESE: I'm Emily Reese.
MOLLY BLOOM: She's the host of Level with Emily Reese, a podcast all about sounds and music in video games.
HENRY GOLDMAN: It features interviews with composers about their work.
MOLLY BLOOM: That's right. Emily says video game music is fundamentally different from other soundtracks in one big way.
EMILY REESE: It tends to be interactive. When you watch a film or a television show, that's a very static experience. You hear that music once, and then it's always the same through every single viewing of that particular media.
But when you're playing a video game, there are times where you might spend an hour in one area where another player might spend five minutes. And so the music is tailored to adapt to each player, depending on how they're playing.
MOLLY BLOOM: Have you ever noticed that while playing a game-- how the music changes?
HENRY GOLDMAN: Yeah, I've noticed that. I mean, there's some games that don't even-- the music doesn't change, no matter how long you're on one level or one area. It just gets annoying, and then you don't want to play that game anymore.
MOLLY BLOOM: But then the games that do change are a little more interesting.
HENRY GOLDMAN: Definitely.
MOLLY BLOOM: So the way that this music is different is that they layer different themes on top of each other. So that's when you add 1 bit of music, and then another bit, and then another, depending on what's going on in the game.
HENRY GOLDMAN: We asked her to give us an example.
EMILY REESE: OK, so let's say we're in an adventure game, and your job is to go from one end of a cave to the other end of the cave, and along the way, certain things are going to happen. So when you first enter that cave, the music is likely going to be kind of ambient, like you would think of a soundtrack-- kind of a background hum of what's happening.
MAN 1: (EERIE SINGING) Background hum. Background hum.
EMILY REESE: And as you're moving through the cave, and as certain-- maybe goblins are coming out, or maybe there are rocks falling. Maybe you come up to a point where you can't get across. You have to figure out some way to cross a pool of water in the lake or something like that. Other layers of music will pop in.
WOMAN 1: (EERIE SINGING) Obstacles. Obstacles. Things are getting harder.
EMILY REESE: And you might not even notice it. They might just creep in so slowly, and it'll be a layer of music that works really well with whatever you've already been hearing. And then once maybe an enemy pops out, that's when the drums would kick in.
MAN 2: (EERIE SINGING) Bom, bom, enemies. Bom, bom, enemies. Bom, bom, enemies. Bom, bom, enemies.
EMILY REESE: That helps you, I think, to get in the mindset of, OK, now it's time to do battle, or now it's time to solve this puzzle, or whatever it is that that game is telling you you need to achieve. A lot of times, the intensity of the music will ramp up a little bit, and there will be many more layers added to that.
WOMAN 2: (EERIE SINGING) Everything is happening. OMG it's happening. Everything is happening. OMG it's happening.
EMILY REESE: When that task--
COMPUTER: (ROBOTIC VOICE) Beep, boop. Level achieved.
EMILY REESE: --those tend to drop away, and we're back down to layer one where the music is nice and ambient. And you're back in your cave exploring and not getting in any trouble.
MAN 1: (EERIE SINGING) Background hum.
EMILY REESE: It's that blending of these different layers that makes each experience unique, really.
MAN 1: (EERIE SINGING) Background hum. Background hum.
MAN 3: (DISTORTED VOICE) Ba, ba ba, ba, ba, ba, ba, ba, ba, ba, brains on.
MOLLY BLOOM: Let's go back to that mystery sound. Ready to hear it one more time?
HENRY GOLDMAN: Yeah.
[CLICKING]
MOLLY BLOOM: A different guest this time?
HENRY GOLDMAN: I don't know, I'm still pretty high on keyboard. But that's kind of making me think-- if we're talking about video games, and this could possibly be an old sound, maybe it's an arcade. And maybe somebody playing the video game, and that's the sound of clicking the buttons before there was music.
MOLLY BLOOM: That is a really good guess. This is actually a sound that's very familiar to me. It's actually one that I hear in my living room all the time. Ready for the answer?
HENRY GOLDMAN: Yeah. Here it is.
ANDY DOUCETTE: That was the sound of my video game controller as I was playing video games on my Xbox 360. My name is Andy Doucette. I'm an artist, a teacher, and I've been playing video games for about 30 years.
MOLLY BLOOM: What's your relationship to me?
ANDY DOUCETTE: You're my wife, and I'm your husband. We're married.
MOLLY BLOOM: And I hear that sound a lot when you play video games because you are wearing headphones. So why do you wear headphones when you play?
ANDY DOUCETTE: Sometimes it just allows you to hear the sounds a little bit better. And some games are designed to have sound move around your head, so you can hear that in stereo. There's two different experiences-- what I hear in my head and what everyone else hears, which is just silence and the clickety-clack of my controller.
HENRY GOLDMAN: I never thought about that.
MOLLY BLOOM: Yeah. People who are playing don't usually hear the sound of the clicking because they are hearing the music and the sounds from the game. So it's a little different perspective on what it sounds like to play video games.
[ELECTRONIC MUSIC]
So it's possible to play video games without the music. But will that help or hinder your playing?
HENRY GOLDMAN: To find out, we talk to psychologist Siu-Lan Tan who actually studies the impact that video game music has on us.
MOLLY BLOOM: And she teaches at Kalamazoo College in Michigan.
HENRY GOLDMAN: We started with the basics. What kind of music makes us feel certain ways?
SIU-LAN TAN: So if we think, for example, about video game music and emotions like scary, nervous, happy, if you're playing a video game, there's some music that just expresses emotion. So it sort of sets up an atmosphere around you. But then there's other music that can actually make you feel the emotion, so it induces emotion.
So, for instance, Henry, have you played a racing game before with and without music? And if you are playing it with music, do you sometimes feel like the music can actually make you feel more excited or more panicky or make you feel exhilarated, like you're flying or soaring? And that's music actually inducing or stirring up emotions inside you.
HENRY GOLDMAN: Well, why does it do that?
SIU-LAN TAN: So why music does that is actually a complex question, and there's lots of possible answers. One way that music can express or induce emotion is by mimicking what your body does when you experience an emotion in your real life. So, for example, do the classic side-scroller Mario Brothers game?
HENRY GOLDMAN: Yeah, I like that game.
SIU-LAN TAN: So you notice the music gets steadily faster and faster as you're running out of time, right?
HENRY GOLDMAN: Yeah.
SIU-LAN TAN: So the music makes you feel excited and energized. And one way it might do that is that because it sounds like it's getting faster and faster, it's just like the pounding of your heartbeat going faster and faster when you're running fast.
HENRY GOLDMAN: Does the music in video games actually help us play better, or does it make us play worse?
SIU-LAN TAN: So if you have a lot of sounds that are just doubling up what you see on the screen, it might actually distract you. But there are some sounds and music in your video game that are cues or warnings of things that you don't see on the screen. So they're your only clues about something important. And if you can pay attention to those things, you can actually play better. It can really help your game.
HENRY GOLDMAN: Well, I've started playing-- instead of using the small amounts of music that's in that game, I've been turning-- well, kind of drowning out the music with my own music played off of my phone, like just the radio. And it's been helping me. I think I've been playing a lot better with it than when I haven't had it on because it kind of-- it's like a white noise, in a way, so I only pay attention to the things in the game that really matter.
SIU-LAN TAN: That's really, really interesting because we did research on exactly that topic. We were asking about whether people might play a new game, particularly a complex game that's new to them that they haven't played before, if they play better with the sound off or the sound on or, like you were doing, playing pleasant music that they like that's not distracting in the background that's not related to the game.
And, actually, we found the same thing that you said. You could have done our research, Henry, and been a step ahead of us. But we found the same thing-- that if you're playing a new game, and it's complex, and you don't know it really well yet, sometimes it's actually better to play with relaxing music.
And you can just focus on the visual cues and what's really important, like you said. But we did find that the subsection of our participants who were the top gamers, who were getting the top, top scores-- they were different from the rest. They actually played better from the beginning if they were playing with the music and the sound from the game on.
And that's because they're able to put together the sound cues and what's happening on the screen in a really tight way and know what to pay attention to and not be distracted. But most people, it seems like it would be better to play with music that's unrelated for a short time until you get to know your game really well.
HENRY GOLDMAN: So it turns out, if you're just starting out--
MOLLY BLOOM: Or if you're an expert gamer.
HENRY GOLDMAN: --video game music can really help.
MOLLY BLOOM: But if you're a regular player of a game, it can eventually become a little bit of a distraction.
HENRY GOLDMAN: So you can make your own soundtrack.
MOLLY BLOOM: Or maybe try your own experiment-- do you play better with the sound off or on, with no music or maybe with your own soundtrack?
HENRY GOLDMAN: Send us your results.
MOLLY BLOOM: Email us at Hello@BrainsOn.org. Now, before we go, it's time for our Moment of Um.
BACKGROUND VOICES: Um, um, um, um, um, um, um, um.
[? KAVEN MILLS: ?] Hi. I'm [? Kaven ?] Mills. I live in Sacramento. Why are cheetahs so fast?
RICK SCHWARTZ: My name is Rick Schwartz. I'm the ambassador for San Diego Zoo Global that oversees the San Diego Zoo, the San Diego Zoo Safari Park, and our conservation that have around the world.
[DRUM MUSIC]
When it comes to the anatomy of a cheetah, everything about them is really built for speed. We can talk about the physiology or the structure of the spine, the legs, and the muscles that propel the cat forward so quickly. The spine is incredibly flexible.
We tell people to think of it as a giant spring. And every time you bend a spring, no matter what direction, it wants to bounce back. And with that physical force, what the cheetah's body does, as the back legs go forward, and the front legs push back, the spine curves. And then it springs back and actually arches the other way, allowing them to have a huge stride.
We look at the cat family, and cats are well known for having retractable claws. Cheetahs are the exception to the rule. They have claws that are always out, similar to a dog. So they are thick, and they are blunt.
But because they're always out, they also work very much like you have for football cleats or soccer cleats. So as they're chasing and running, it does help them to get that grip while they change directions to chase down their prey.
There's the internal part of it, as well-- enlarged lungs. It's pretty impressive the way the rib cage moves with the sprint to help force air in and out of those lungs, almost like bellows, if you would. If you compare, let's say, the cheetah's speed-- top speed. They don't always hit 70 miles an hour, but it's very close, usually, most chases-- you compare that top speed to say something like, say, a lion-- another African cat that they will share territory with-- the lion's top speed usually is around 35.
They try to rely on that element of surprise and not the element of chase. And the reason we see that the cheetah has to have this kind of speed versus the other cats can use that element of surprise is when they hunt. Cheetahs are going to hunt in the middle of the day when the other predators are sleeping. And the other predators, like lions and leopards, tend to hunt in the twilight in nighttime hours.
[DRUM MUSIC]
MOLLY BLOOM: Fast like a cheetah, I'm going to speed through this list of names. It's time for the Brains Honor Roll. These are the listeners who keep us going by sharing their questions, ideas, and mystery sounds with us. Here they are.
[LISTING HONOR ROLL]
[ELECTRONIC MUSIC]
And that's it for this episode of Brains On.
HENRY GOLDMAN: This episode was produced by Sanden Totten, Marc Sanchez, and Molly Bloom.
MOLLY BLOOM: Many thanks to [LISTING HONOR ROLL].
HENRY GOLDMAN: If you like what you're hearing, write a review on iTunes--
MOLLY BLOOM: Like us on Facebook--
HENRY GOLDMAN: --or follow us on Twitter at @Brains_On.
MOLLY BLOOM: And you can always find past episodes at our website, BrainsOn.org.
HENRY GOLDMAN: Thanks for listening.
COMPUTER: (ROBOTIC VOICE) Beep, boop. Level achieved.
CHILD: Brains On is supported in part by a grant from the National Science Foundation.
[MUSIC PLAYING]
MOLLY BLOOM: Recognize these tunes?
[MUSIC PLAYING]
You probably do if you've ever played Super Mario 3D World, Skyrim, or Minecraft.
HENRY GOLDMAN: But it wasn't always a given that your games would have a soundtrack.
MOLLY BLOOM: Board game certainly didn't.
HENRY GOLDMAN: And early arcade games--
MOLLY BLOOM: The predecessor to video games.
HENRY GOLDMAN: --didn't have music either.
MOLLY BLOOM: So how did it come to be here?
HENRY GOLDMAN: And what kind of effect does it have on you?
MOLLY BLOOM: We'll find out now.
HENRY GOLDMAN: Keep listening.
[ELECTRONIC MUSIC]
MOLLY BLOOM: I'm Molly Bloom, and my co-host for this episode of Brains On is 12-year-old Henry Goldman.
HENRY GOLDMAN: Hello.
MOLLY BLOOM: So, Henry, you play video games.
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: And do you ever the first video game you ever played?
HENRY GOLDMAN: Hmm, the first video game I ever played was probably Cube Runner, back when I was four or five.
MOLLY BLOOM: What's Cube Runner?
HENRY GOLDMAN: It's this game where it was kind of like a simulation of a car, almost, except you were just an arrow. And you would have to tilt the phone or iPad or whatever device you had back and forth trying to avoid all of these just random cubes that would pop up on the screen.
MOLLY BLOOM: Was there music in that game?
HENRY GOLDMAN: There was music in that game.
MOLLY BLOOM: Do you remember it?
HENRY GOLDMAN: Oh, it went something like dit-dit-dit. I don't know. It's one of those songs where it's in your head, but it doesn't come out.
[ELECTRONIC MUSIC]
MOLLY BLOOM: So which video game would you say has your favorite music?
HENRY GOLDMAN: Hmm, my favorite music-- ah, this game called Bouncy Slime. It's on the iPad where you're this little slime ball, and you kind of bounce around. It's weird, but it's fun.
MOLLY BLOOM: And why does that have your favorite music? What about it do you like?
HENRY GOLDMAN: I don't know. It just kind of triggers something that makes it fun. The game seems more fun.
MOLLY BLOOM: So it actually enhances the game when you have the music on?
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: Because sometimes--
HENRY GOLDMAN: It gets you pumped up.
MOLLY BLOOM: Right, because sometimes you don't like the music for games, right?
HENRY GOLDMAN: Yeah, and then it bugs you, and then you don't play that game.
MOLLY BLOOM: Henry finds video game music annoying sometimes. So we want to know--
HENRY GOLDMAN: What effect does video game music have on players?
MOLLY BLOOM: Before we get to that question, let's look at where video game music came from. Producer Mac Sanchez is here to tell us more. Hi, Marc.
MARC SANCHEZ: Hey, guys.
HENRY GOLDMAN: Hello. Marc, when does music enter the world of video games?
MARC SANCHEZ: Around 1978, these four chords changed everything.
[ELECTRONIC MUSIC]
Do you know what that is?
MOLLY BLOOM: Space Invaders?
MARC SANCHEZ: Exactly. Have you ever played Space Invaders?
HENRY GOLDMAN: Yeah.
MOLLY BLOOM: You did? You've played Space Invaders?
HENRY GOLDMAN: Yeah. Aren't you a little ship thing at the bottom?
MOLLY BLOOM: Yeah.
HENRY GOLDMAN: And then there's the little rocks, and there's all the thingies coming that are shooting at you, and you don't want them to get to the bottom.
MOLLY BLOOM: Exactly.
MARC SANCHEZ: Yep. Space Invaders-- it started out as a stand-up arcade game, and it was set in this really, really primitive version of outer space. The guy that invented the game-- his name is Tomohiro Nishikado. He decided to put those four chords throughout the game. And the longer you keep your spaceship from getting hit by an alien by one of their lasers, the faster the music would get.
[ELECTRONIC MUSIC]
And just those four notes, those four chords makes things really, really tense.
MOLLY BLOOM: I've never played Space Invaders, but I've played other games where the music speeds up the closer you get to the end. And I get really nervous when it happens. How about you, Henry?
HENRY GOLDMAN: I agree with that. I mean, I think that as more games are starting to pop up, a lot more games are starting to have music that's speeding up.
MOLLY BLOOM: And does it make you feel nervous?
HENRY GOLDMAN: Definitely.
MARC SANCHEZ: I called up a friend of mine. He's a programmer and a musician. His name is Ochen Kaylan. I wanted to ask him about these early sounds, this early music in video games. They're actually made by an 8-bit chip. I don't know if you've heard of that or not.
MOLLY BLOOM: Have you heard the term 8-bit before?
HENRY GOLDMAN: Yeah, it's like-- well, I've never heard of it referenced to the music. I've only heard of it referenced to the pixelated creatures.
MOLLY BLOOM: Yeah, it references what you see on the screen. But the shorthand for that music-- people call it 8-bit-music.
MARC SANCHEZ: Here's what he said.
OCHEN KAYLAN: Inside video games and arcade consoles, there are chips, and transistors, and capacitors, and all those things. Well, those chips run sets of instructions. And there may maybe one chip to do all the video, and so that's what makes all the pixels on the screen light up certain colors. And there may be another chip that deals with the joystick, or the buttons, or something like that. And there's often another chip that just does sound.
MARC SANCHEZ: OK, you guys got this so far? Microchips are inside the games, and each chip has a set of instructions that tells the game how to react.
MOLLY BLOOM: OK.
MARC SANCHEZ: Now, let's focus on the chip that sends sound signals to the game's speaker. We've talked about sound waves in the past, right?
MOLLY BLOOM: Yes, at our music episode. You can listen--
MARC SANCHEZ: Shout-out to the music episode.
MOLLY BLOOM: You can listen to that to find out how our brains actually hear music.
MARC SANCHEZ: And as we learned in that episode, sound travels in waves. And when you think about a wave, when part of it goes up, the other part goes down. Ochen told me that this kind of symmetry, this up and down motion, is ideal for these sounds that are coming out of 8-bit chips.
OCHEN KAYLAN: The binary nature of computers-- 0s and 1s, or off and on are really well-suited to making music because, if you think of a speaker, the cone in the speaker, it just goes in and out. If a computer says, 0, make the cone go in. And if the computer says 1, make the cone go out.
And so if the computer can just send a series of 0s and 1s in some special order, then that can make the speaker go in and out in some special order. And that special order, we call sound. And so that sound might be a violin, or it might be a piano. Or it could just be some alien getting shot.
MOLLY BLOOM: OK. So far, we've got chips getting instructions in the form of 0s and 1s.
MARC SANCHEZ: Binary code.
MOLLY BLOOM: And the chip turns those instructions into sound waves that come through the speaker.
MARC SANCHEZ: Right.
MOLLY BLOOM: But why do the old games sound so old? Why couldn't Space Invaders have a super awesome soundtrack like Skyrim that sounds like an orchestra with violins?
MARC SANCHEZ: It's all about limitations.
HENRY GOLDMAN: Limitations?
MARC SANCHEZ: Limitations and exponents. Are you guys ready to do a little multiplication?
HENRY GOLDMAN: I guess so.
MARC SANCHEZ: Compared to today's microchips like the ones in your phone, or tablet, or a computer that you might have, the chips in Tomohiro Nishikado's Space Invaders-- they might as well have been potato chips. Those early chips could only run a limited, teeny-tiny amount of instructions compared to today before they ran out of room or bits. The more bits, the better the sound.
OCHEN KAYLAN: Bit means how many-- in terms of a mathematical equation, it's 2 to the n power. And so 2 times 2 would be called 2-bit because you're multiplying 2 by 2 just twice. 2 times 2 times 2 is 2 to the third power, or 3-bit. And 2 times 2 times 2 times 2 is 2 to the fourth power or 4-bit.
MARC SANCHEZ: A lot of early video game consoles like the Atari 2600, or early Nintendo systems, or the Commodore 64 computers-- those ran on 8-bit chips. You guys want to do the math on this one?
MOLLY BLOOM: Yeah. Let's see. 8 bits would be 2 to the eighth power--
MARC SANCHEZ: Mm-hmm.
MOLLY BLOOM: So that's 2--
MARC SANCHEZ: Times 2--
HENRY GOLDMAN: Times 2--
MOLLY BLOOM: Times 2--
MARC SANCHEZ: Times 2--
HENRY GOLDMAN: Times 2--
MOLLY BLOOM: Times 2--
HENRY GOLDMAN: Times 2.
MARC SANCHEZ: OK. Multiply that out, and you get--
HENRY GOLDMAN: 256.
MOLLY BLOOM: But 256 what?
MARC SANCHEZ: Ochen?
OCHEN KAYLAN: So remember we talked about the 0s and the 1s make the speaker go in and out. Well, if you only have two states-- if you have in, and you have out-- that's 1 bit, meaning that there are only two instructions or 2 to the first power. And so you only have two states-- either all the way in or all the way out.
If you have something that's 2-bit, meaning that it's 2 times 2, or 4, that means there are four states. So it would be all the way off, or all the way on, or there would be two states in between-- about a 1/3 on and about 2/3 on. Well, 8-bit means that there's 2 times 2 times 2 times 2, or eight times, or 256 states. And so that speaker can be in 256 places.
That sounds like a lot, but music is really complicated. And so if you only allow the speaker to rest at 256 separate places, it approximates sound. When we're just walking outside, and we're hearing birds, and we're hearing trees rustle, we're hearing millions and millions of states between totally on and totally off.
[ELECTRONIC MUSIC]
MARC SANCHEZ: Among his many talents, Ochen has programmed the sounds of 8-bit computer chips into modern music-making software.
MOLLY BLOOM: We're listening to a song made with his software right now, in fact.
MARC SANCHEZ: And there's a whole 8-bit music scene out there. Some musicians, like Ochen, use software and modern computers to make their computer video game-sounding music. And others, the purists out there, they purposely limit themselves by using the original chips.
MOLLY BLOOM: Cool. Well, I guess there's just one thing left to say.
MARC SANCHEZ: Yep.
ALL: Dance party.
[ELECTRONIC MUSIC]
MOLLY BLOOM: You can find links to Ochen's software and music at our website, BrainsOn.org. Speaking of beautiful sounds, we want to hear yours. We're working on another Mystery Sound Extravaganza and want to hear some of the mystery sounds that you think will stump your fellow listeners.
You can email them to Hello@BrainsOn.org. Be sure to both record the sound and your explanation of what it is. You can also send your drawings and questions to that same email address-- Hello@BrainsOn.org. That's what Caleb did.
CALEB: Why are cheetahs so fast?
MOLLY BLOOM: We're going to answer that question during our Moment of Um at the end of the show. And we're going to read the latest group of listeners to join Caleb on the Brain's Honor Roll. That's the way we thank all of you who share your questions and ideas with us. Keep listening.
HENRY GOLDMAN: So how do we get from this--
[SIMPLE ELECTRONIC MUSIC]
--to this?
[ORCHESTRAL MUSIC]
MOLLY BLOOM: Excellent question, but we have something we need to do first. It's time for the mystery sound.
[ELECTRONIC WHISTLES AND POPS]
CHILD: (WHISPERING) Mystery sound.
MOLLY BLOOM: Here it is.
[CLICKING]
Any guesses?
HENRY GOLDMAN: It sounded kind of like a keyboard clicking.
MOLLY BLOOM: OK. OK, you're on the right track. But think about it a little bit more. We're going to come back to it later in the show. And while you and our listeners puzzle over it, we're going to get back to that question that you just asked. How did video game music evolve?
Well, as the technology evolved, so did the music. Logic came into the picture. The music went from playing at the start of levels and just looping over and over to having to make decisions about where it was going to play.
HENRY GOLDMAN: If Mario eats a mushroom, then play this music.
[ELECTRONIC MUSIC]
MOLLY BLOOM: If Mario jumps on Yoshi, then add percussion.
HENRY GOLDMAN: If Mario jumps off Yoshi, then remove percussion.
MOLLY BLOOM: So it's this logic in the software that makes the music interactive. To explain more about how that works, we turn to one of our podcasting buddies.
EMILY REESE: I'm Emily Reese.
MOLLY BLOOM: She's the host of Level with Emily Reese, a podcast all about sounds and music in video games.
HENRY GOLDMAN: It features interviews with composers about their work.
MOLLY BLOOM: That's right. Emily says video game music is fundamentally different from other soundtracks in one big way.
EMILY REESE: It tends to be interactive. When you watch a film or a television show, that's a very static experience. You hear that music once, and then it's always the same through every single viewing of that particular media.
But when you're playing a video game, there are times where you might spend an hour in one area where another player might spend five minutes. And so the music is tailored to adapt to each player, depending on how they're playing.
MOLLY BLOOM: Have you ever noticed that while playing a game-- how the music changes?
HENRY GOLDMAN: Yeah, I've noticed that. I mean, there's some games that don't even-- the music doesn't change, no matter how long you're on one level or one area. It just gets annoying, and then you don't want to play that game anymore.
MOLLY BLOOM: But then the games that do change are a little more interesting.
HENRY GOLDMAN: Definitely.
MOLLY BLOOM: So the way that this music is different is that they layer different themes on top of each other. So that's when you add 1 bit of music, and then another bit, and then another, depending on what's going on in the game.
HENRY GOLDMAN: We asked her to give us an example.
EMILY REESE: OK, so let's say we're in an adventure game, and your job is to go from one end of a cave to the other end of the cave, and along the way, certain things are going to happen. So when you first enter that cave, the music is likely going to be kind of ambient, like you would think of a soundtrack-- kind of a background hum of what's happening.
MAN 1: (EERIE SINGING) Background hum. Background hum.
EMILY REESE: And as you're moving through the cave, and as certain-- maybe goblins are coming out, or maybe there are rocks falling. Maybe you come up to a point where you can't get across. You have to figure out some way to cross a pool of water in the lake or something like that. Other layers of music will pop in.
WOMAN 1: (EERIE SINGING) Obstacles. Obstacles. Things are getting harder.
EMILY REESE: And you might not even notice it. They might just creep in so slowly, and it'll be a layer of music that works really well with whatever you've already been hearing. And then once maybe an enemy pops out, that's when the drums would kick in.
MAN 2: (EERIE SINGING) Bom, bom, enemies. Bom, bom, enemies. Bom, bom, enemies. Bom, bom, enemies.
EMILY REESE: That helps you, I think, to get in the mindset of, OK, now it's time to do battle, or now it's time to solve this puzzle, or whatever it is that that game is telling you you need to achieve. A lot of times, the intensity of the music will ramp up a little bit, and there will be many more layers added to that.
WOMAN 2: (EERIE SINGING) Everything is happening. OMG it's happening. Everything is happening. OMG it's happening.
EMILY REESE: When that task--
COMPUTER: (ROBOTIC VOICE) Beep, boop. Level achieved.
EMILY REESE: --those tend to drop away, and we're back down to layer one where the music is nice and ambient. And you're back in your cave exploring and not getting in any trouble.
MAN 1: (EERIE SINGING) Background hum.
EMILY REESE: It's that blending of these different layers that makes each experience unique, really.
MAN 1: (EERIE SINGING) Background hum. Background hum.
MAN 3: (DISTORTED VOICE) Ba, ba ba, ba, ba, ba, ba, ba, ba, ba, brains on.
MOLLY BLOOM: Let's go back to that mystery sound. Ready to hear it one more time?
HENRY GOLDMAN: Yeah.
[CLICKING]
MOLLY BLOOM: A different guest this time?
HENRY GOLDMAN: I don't know, I'm still pretty high on keyboard. But that's kind of making me think-- if we're talking about video games, and this could possibly be an old sound, maybe it's an arcade. And maybe somebody playing the video game, and that's the sound of clicking the buttons before there was music.
MOLLY BLOOM: That is a really good guess. This is actually a sound that's very familiar to me. It's actually one that I hear in my living room all the time. Ready for the answer?
HENRY GOLDMAN: Yeah. Here it is.
ANDY DOUCETTE: That was the sound of my video game controller as I was playing video games on my Xbox 360. My name is Andy Doucette. I'm an artist, a teacher, and I've been playing video games for about 30 years.
MOLLY BLOOM: What's your relationship to me?
ANDY DOUCETTE: You're my wife, and I'm your husband. We're married.
MOLLY BLOOM: And I hear that sound a lot when you play video games because you are wearing headphones. So why do you wear headphones when you play?
ANDY DOUCETTE: Sometimes it just allows you to hear the sounds a little bit better. And some games are designed to have sound move around your head, so you can hear that in stereo. There's two different experiences-- what I hear in my head and what everyone else hears, which is just silence and the clickety-clack of my controller.
HENRY GOLDMAN: I never thought about that.
MOLLY BLOOM: Yeah. People who are playing don't usually hear the sound of the clicking because they are hearing the music and the sounds from the game. So it's a little different perspective on what it sounds like to play video games.
[ELECTRONIC MUSIC]
So it's possible to play video games without the music. But will that help or hinder your playing?
HENRY GOLDMAN: To find out, we talk to psychologist Siu-Lan Tan who actually studies the impact that video game music has on us.
MOLLY BLOOM: And she teaches at Kalamazoo College in Michigan.
HENRY GOLDMAN: We started with the basics. What kind of music makes us feel certain ways?
SIU-LAN TAN: So if we think, for example, about video game music and emotions like scary, nervous, happy, if you're playing a video game, there's some music that just expresses emotion. So it sort of sets up an atmosphere around you. But then there's other music that can actually make you feel the emotion, so it induces emotion.
So, for instance, Henry, have you played a racing game before with and without music? And if you are playing it with music, do you sometimes feel like the music can actually make you feel more excited or more panicky or make you feel exhilarated, like you're flying or soaring? And that's music actually inducing or stirring up emotions inside you.
HENRY GOLDMAN: Well, why does it do that?
SIU-LAN TAN: So why music does that is actually a complex question, and there's lots of possible answers. One way that music can express or induce emotion is by mimicking what your body does when you experience an emotion in your real life. So, for example, do the classic side-scroller Mario Brothers game?
HENRY GOLDMAN: Yeah, I like that game.
SIU-LAN TAN: So you notice the music gets steadily faster and faster as you're running out of time, right?
HENRY GOLDMAN: Yeah.
SIU-LAN TAN: So the music makes you feel excited and energized. And one way it might do that is that because it sounds like it's getting faster and faster, it's just like the pounding of your heartbeat going faster and faster when you're running fast.
HENRY GOLDMAN: Does the music in video games actually help us play better, or does it make us play worse?
SIU-LAN TAN: So if you have a lot of sounds that are just doubling up what you see on the screen, it might actually distract you. But there are some sounds and music in your video game that are cues or warnings of things that you don't see on the screen. So they're your only clues about something important. And if you can pay attention to those things, you can actually play better. It can really help your game.
HENRY GOLDMAN: Well, I've started playing-- instead of using the small amounts of music that's in that game, I've been turning-- well, kind of drowning out the music with my own music played off of my phone, like just the radio. And it's been helping me. I think I've been playing a lot better with it than when I haven't had it on because it kind of-- it's like a white noise, in a way, so I only pay attention to the things in the game that really matter.
SIU-LAN TAN: That's really, really interesting because we did research on exactly that topic. We were asking about whether people might play a new game, particularly a complex game that's new to them that they haven't played before, if they play better with the sound off or the sound on or, like you were doing, playing pleasant music that they like that's not distracting in the background that's not related to the game.
And, actually, we found the same thing that you said. You could have done our research, Henry, and been a step ahead of us. But we found the same thing-- that if you're playing a new game, and it's complex, and you don't know it really well yet, sometimes it's actually better to play with relaxing music.
And you can just focus on the visual cues and what's really important, like you said. But we did find that the subsection of our participants who were the top gamers, who were getting the top, top scores-- they were different from the rest. They actually played better from the beginning if they were playing with the music and the sound from the game on.
And that's because they're able to put together the sound cues and what's happening on the screen in a really tight way and know what to pay attention to and not be distracted. But most people, it seems like it would be better to play with music that's unrelated for a short time until you get to know your game really well.
HENRY GOLDMAN: So it turns out, if you're just starting out--
MOLLY BLOOM: Or if you're an expert gamer.
HENRY GOLDMAN: --video game music can really help.
MOLLY BLOOM: But if you're a regular player of a game, it can eventually become a little bit of a distraction.
HENRY GOLDMAN: So you can make your own soundtrack.
MOLLY BLOOM: Or maybe try your own experiment-- do you play better with the sound off or on, with no music or maybe with your own soundtrack?
HENRY GOLDMAN: Send us your results.
MOLLY BLOOM: Email us at Hello@BrainsOn.org. Now, before we go, it's time for our Moment of Um.
BACKGROUND VOICES: Um, um, um, um, um, um, um, um.
[? KAVEN MILLS: ?] Hi. I'm [? Kaven ?] Mills. I live in Sacramento. Why are cheetahs so fast?
RICK SCHWARTZ: My name is Rick Schwartz. I'm the ambassador for San Diego Zoo Global that oversees the San Diego Zoo, the San Diego Zoo Safari Park, and our conservation that have around the world.
[DRUM MUSIC]
When it comes to the anatomy of a cheetah, everything about them is really built for speed. We can talk about the physiology or the structure of the spine, the legs, and the muscles that propel the cat forward so quickly. The spine is incredibly flexible.
We tell people to think of it as a giant spring. And every time you bend a spring, no matter what direction, it wants to bounce back. And with that physical force, what the cheetah's body does, as the back legs go forward, and the front legs push back, the spine curves. And then it springs back and actually arches the other way, allowing them to have a huge stride.
We look at the cat family, and cats are well known for having retractable claws. Cheetahs are the exception to the rule. They have claws that are always out, similar to a dog. So they are thick, and they are blunt.
But because they're always out, they also work very much like you have for football cleats or soccer cleats. So as they're chasing and running, it does help them to get that grip while they change directions to chase down their prey.
There's the internal part of it, as well-- enlarged lungs. It's pretty impressive the way the rib cage moves with the sprint to help force air in and out of those lungs, almost like bellows, if you would. If you compare, let's say, the cheetah's speed-- top speed. They don't always hit 70 miles an hour, but it's very close, usually, most chases-- you compare that top speed to say something like, say, a lion-- another African cat that they will share territory with-- the lion's top speed usually is around 35.
They try to rely on that element of surprise and not the element of chase. And the reason we see that the cheetah has to have this kind of speed versus the other cats can use that element of surprise is when they hunt. Cheetahs are going to hunt in the middle of the day when the other predators are sleeping. And the other predators, like lions and leopards, tend to hunt in the twilight in nighttime hours.
[DRUM MUSIC]
MOLLY BLOOM: Fast like a cheetah, I'm going to speed through this list of names. It's time for the Brains Honor Roll. These are the listeners who keep us going by sharing their questions, ideas, and mystery sounds with us. Here they are.
[LISTING HONOR ROLL]
[ELECTRONIC MUSIC]
And that's it for this episode of Brains On.
HENRY GOLDMAN: This episode was produced by Sanden Totten, Marc Sanchez, and Molly Bloom.
MOLLY BLOOM: Many thanks to [LISTING HONOR ROLL].
HENRY GOLDMAN: If you like what you're hearing, write a review on iTunes--
MOLLY BLOOM: Like us on Facebook--
HENRY GOLDMAN: --or follow us on Twitter at @Brains_On.
MOLLY BLOOM: And you can always find past episodes at our website, BrainsOn.org.
HENRY GOLDMAN: Thanks for listening.
COMPUTER: (ROBOTIC VOICE) Beep, boop. Level achieved.
Transcription services provided by 3Play Media.
