It’s Halloween night and I am a pirate, with gold rings in my ears and a wide, unwieldy hat on my head. I’m paying no attention to the colorful characters that surround me at the costume party–the SpongeBobs and Joe the Plumbers. Nothing matters except the plastic guitar in my hands, the color-coded notes streaming down the television screen, Betty Crocker on drums, and Richard Simmons on vocals. Deaf to the din of party conversation, my ears are trained on every note of the Rolling Stones’ “Gimme Shelter.”

We go on that way all night, the characters swirling in and out of our imaginary band. I play every part. My bandmates are Alan Moore’s V, Richard Nixon, Cupid, and a sexy ladybug. I don’t need to know their real names–the camaraderie is there the moment they pick up their instruments. When the party thins out, I look up to find that it is 4:00 a.m. That’s the kind of thing that can happen when you play Rock Band.

Produced by Harmonix, a Cambridge-based company founded by Alex Rigopulos ‘92, SM ‘94, and Eran Egozy ‘95, MEng ‘95, Rock Band and its sequel, Rock Band 2, are hugely successful video games that simulate the experience of playing in a band–and they’re just the latest examples of high-profile games that can trace their roots to MIT. Rock Band players use plastic drums, guitar, bass, and a microphone to play along to such songs as “Gimme Shelter” and Weezer’s “Say It Ain’t So”: the guitarist, for example, hits the guitar’s plastic strum bar while holding down sets of colored buttons on its neck in simplified chord patterns, guided by color-coded notes on-screen. The game can be set to different levels of difficulty, but whatever the setting, the accuracy with which the player presses the buttons in time to the music determines how good the song sounds. Correct play triggers the full guitar solos in the original recording; mistakes interrupt the song with silence and sour notes.

The core concept of breaking down songs so that nonmusicians could meaningfully control them was at the heart of a research project that Egozy and Rigopulos began at the MIT Media Lab. It eventually morphed into Harmonix’s first hit, Guitar Hero. (Harmonix sold the rights to Guitar Hero to another company; one of the game’s latest iterations sold 9.8 million units to become the top-selling video game between 1995 and 2008.) In developing such broadly appealing games that can serve as the center of social events, Harmonix has redefined expectations for the gaming industry, which racked up $9.5 billion in sales in 2007 and has assumed a prominent place in modern life and culture in just a matter of decades. (Sixty-five percent of American households now play computer or video games, according to the Entertainment Software Association.) And Harmonix’s products are hardly the first MIT-influenced games to shake things up. A look back at MIT projects from distinct eras in gaming history shows that the Institute has played a big part in shaping the industry from the very beginning.

Additional Information:"MIT's Game Lab"

Hacking Their Way Into History

MIT’s involvement in the development of video games dates back to 1958, when Professor Jack Dennis ‘53, SM ‘54, ScD ‘58, began letting students play around with a recently donated Digital Equipment Corporation PDP-1 (for “programmed data processor”)–an early commercial computer designed to interact with its user instead of merely reading punch cards. By 1961, ­Martin Graetz ‘58, Wayne Wiitanen ‘57, and MIT employee and Tech Model Railroad Club member Steve Russell had hacked their way into history by developing the initial concept for Spacewar, in which two players, each in control of a spaceship, battled it out in a starfield. Although William Higinbotham, a nuclear physicist at the Brookhaven National Laboratory, had created the primitive Tennis for Two game in 1958, Spacewar was much more sophisticated; most experts consider it the first interactive video game.

Tinkering away in Building 26 and at the Hingham Institute Space Warfare Study Group (Russell’s home), Russell, Dan Edwards ‘59, SM ‘66, EE ‘67, Peter Samson ‘62, and others improved the game over the next year. Eventually, a surplus jet-fighter joystick was added for the full piloting experience–likely the first use of a joystick in a video game. The final version of Spacewar accurately modeled the positions of stars, the motions of the ships, and the effects of gravity, using close to 100,000 calculations per second to do so. “And that’s the first computer game to come out of MIT!” says Philip Tan ‘01, SM ‘03, the U.S. executive director of the Singapore-MIT GAMBIT game lab. “It’s insane.” A marvel of geeky coolness, the game became popular among the few scientists and engineers with access to the 50 or so PDP-1s that were ever made. But it would be years before video games–and the computers needed to run them–became a consumer industry. The hackers who created Spacewar were just in it for fun, and never made money from their creation.

Playing With Technology

“Another routine day of drudgery aboard the Stellar Patrol Ship Feinstein. This morning’s assignment for a certain lowly Ensign Seventh Class: scrubbing the filthy metal deck at the port end of Level Nine. With your Patrol-issue self-contained multi-purpose all-weather scrub-brush, you shine the floor with a diligence born of the knowledge that at any moment, the dreaded Ensign First Class Blather, the bane of your shipboard existence, could appear.”

“Ensign Blather, his uniform immaculate, enters and notices you are away from your post. ‘Twenty demerits, Ensign Seventh Class!’ bellows Blather. ‘Forty if you’re not back on Deck Nine in five seconds!’ He curls his face into a hideous mask of disquiet at your unbelievable negligence.”

Such is one possible beginning to Planetfall, a classic ­interactive-fiction game written by Steve Meretzky ‘79 in the heyday of Infocom, a Cambridge-based software company founded by MIT faculty and alumni in 1979. In interactive fiction, the direction of the story hinges on input from the reader; the ending or other events may change depending on what the reader does. The player controls a character, typing in commands that dictate that character’s behavior whenever a decision point is reached. Often, traversing an entire game and reading the whole story requires the player to solve a series of difficult puzzles.

Nick Montfort, SM ‘98, an assistant professor of digital media at MIT whose book Twisty Little Passages traces the history of interactive fiction, calls Infocom the Shakespeare of game writing. In Planetfall, Meretzky, the company’s most prolific writer, introduced Floyd, a robot helper who traveled with the player. Unlike the wooden characters then typical, Floyd was charming, vivid, and clever. He would tease the players, who tended to save the game to disk before actions that could get their character killed; saving would prompt him to say, “Oh, boy! Are we gonna try something dangerous now?” His selfless actions also became key to the game’s plots.

In the 1980s, Planetfall and other Infocom text adventures were as revolutionary as the flashiest work of Harmonix is today. Infocom’s software made more sophisticated use of the PC’s processing power than common programs such as word processors, explains Montfort. Although the company’s games weren’t the first works of interactive fiction, Montfort says, they brought major technical and stylistic advances to the field.

Infocom had its roots in a game called Zork, built on a mainframe at MIT starting in 1977 by Marc Blank ‘75, Dave Lebling ‘71, SM ‘73, Bruce Daniels ‘71, SM ‘74, and Tim Anderson ‘75, SM ‘77. Players of Zork explored an enormous cave, searching not only for treasure but for the remains of an ancient civilization, described in chunks of text peppered with geeky in-jokes and references to MIT. While the game itself referred to its parser, which was responsible for interpreting player commands, as “fairly stupid,” it was actually able to handle more complex sentence structures than earlier games could. The group also created a fantasy world that was more sophisticated than any seen before: objects such as vehicles could move within it, and other characters could interact meaningfully with the player’s character. When the group started Infocom a few years later, the technological innovation continued.

The company developed advanced compression techniques to fit large quantities of data on disks whose capacity was tiny by modern standards. (At the time, a typical home computer had one 100-kilobyte floppy drive; today, even Apple’s smallest iPod has a one-gigabyte hard drive, or more than ten thousand times the storage capacity.) To build the PC version of Zork, Infocom also created the Zork Implementation Language (ZIL), a new programming language based on the MIT Design Language (MDL) used in the Dynamic Modeling Group at MIT. Using ZIL, programmers could write instructions that read more like English than instructions written in other languages, making programming easier.

One of Infocom’s biggest innovations was the Z-machine, the first commercial example of a type of program known as a “virtual machine.” Virtual machines mediate between a program and a computer’s operating system, so that the same program can run on different types of computers. Instead of having to retool all its games for each new model of computer, as other companies had to do at the time, Infocom just had to retool the Z-machine. Once that was done, Infocom’s entire library became instantly available on that computer. The virtual machine was a key innovation: it’s one of the reasons, for example, that online applications can run on any Web browser, on any operating system.

While Montfort says that it’s hard to trace specific Infocom technologies back to particular MIT projects, he adds, “I don’t think that somebody sitting in their garage unaffiliated with a university would have contrived the Infocom parser and the system for modeling a world.” Like Spacewar, he says, Zork and later Infocom games were influenced by attitudes toward technology that have long been pervasive at the Institute. At MIT, says Montfort, “people are very accepting of exploration and playfulness with technology. It’s okay to take a computer and create a game with it.”

Through the Looking Glass

The player enters a stone building, slipping through its darkest shadows, and watching down dimly illuminated hallways for ­people who might detect his presence. The sound of his own footsteps loud in his ears, he listens for the softer footfalls that could warn him that someone approaches. “I’m sure I saw something,” a cultured voice says, from off to the side.

The arrow strung on his bow prominent in his field of vision, the player continues to stalk. He picks off a few of the keep’s denizens–often with an arrow to the back–before they can spot him. If they manage to spread the word about his presence, they are sure to overwhelm him, ending the mission.

Thief: The Dark Project advanced “first-person” gaming both technologically and conceptually when it was introduced in 1998 by Looking Glass Studios, a company full of MIT alumni that was originally based in Lexington, MA. A group including Doug Church ‘89 designed the game.

First-person games, often called FPSs (for first-person shooters), depict a world through the eyes of the character, an important change in perspective for game play: the player, like the character, can’t see what might be behind her or around the corner. But until Thief came along, first-person games were typically about mowing down enemies wherever they could be found.

Thief, on the other hand, was one of the first 3-D first-person “stealth games.” The player’s objective was to get through a series of missions without being detected. The innovative concept, which drew heavily on the tension created by what the player could not see, required advances in technology: to make the simulation more realistic, Thief’s designers had to work out the subtleties of light and shadow, creating mathematical models that represented how well a character could hide in the darkness. Were the surrounding guards unaware or suspicious? Was the shadow too small to conceal the character? Thief was also one of the first games that tried to faithfully simulate the behavior of sound in a three-dimensional space, accounting for echoes and the damping effects of walls.

It’s no surprise that Looking Glass’s MIT alumni would create a game that modeled the physics of sound and shadow, says ­GAMBIT’s Tan. Video games, he contends, are simply a natural extension of the experimenting being done in labs across campus. “MIT’s always been about simulation,” he says. “There’s always been a sense that you can replicate a reality.” And people naturally want to play around with those simulations, he says. “Interacting with that simulated reality [becomes] the entertainment.”

Once the simulation has made the leap to entertainment, it can still invite the users to explore, experiment, and even hack. “Think about games as a box of tools,” Tan says, “a box of verbs and nouns.” This vision appeals to the engineering mind-set common at MIT. “A game is something you tinker with, not something you receive as an entertainment experience,” Tan says. Thief succeeded because the quality of its simulation convincingly immersed players in the world of the game.

But the more a simulation caters to the engineering mind-set, the more overwhelming it can be. Many game projects that originated at MIT have failed to find an audience, says Tan, “not because they’re not fun to play, but because they’re extremely complicated for non-MIT players to understand.” When a simulation includes too many factors, it loses the accessibility that incites players to pick it up and start messing around.

Although Looking Glass got it right withThief, the company was also guilty of creating simulations that only a narrow group of users could appreciate. Tan points to another of the company’s creations, a flight simulator that modeled the experience of flying so accurately that you nearly had to be a pilot to use it, he says. He believes that while the realism of such games was impressive, it held Looking Glass back from greater commercial success. The company folded in 2000, but its games shaped the sensibilities of many modern game designers.

The Drive to Innovate

A mere 15-minute walk from where the mouth of MIT’s Infinite Corridor opens onto Mass. Ave., Eran Egozy, surrounded by dozens of employees in Harmonix’s second-floor office in Central Square, listens intently to a speaker giving a talk on neuroscience and music. One month after the release of Rock Band 2 in September 2008,Egozy has a bit of a beard, and the slender face below his shock of thick, misbehaving hair bears the look of someone in semi-stunned recovery after a hard push toward a goal. He adopts a serious tone as he reflects on the company’s long struggle from obscure Media Lab research project to stardom. (As of December 2008, U.S. sales of Rock Band had hit 4.5 million units, and Rock Band 2–barely three months after its release–had sold 1.7 million units.)The rush of the new release and the company’s recent years of success haven’t diminished his memory of the lean years, when many of the company’s investors had written Harmonix off.

“We’re taking music, which is something that is older than the hills, really established, and has a strong emotional connection,” he says, “and we’ve figured out how to apply technology to that, and bring it to more people and have them experience it in this new way.” He and cofounder Alex Rigopulos shared this vision while still at the Media Lab, in Tod Machover’s computer music group, where they took on projects that included modifying a cello for Yo-Yo Ma. But, Egozy says, they decided they were more interested in using technology to help average people express themselves musically.

Egozy and Rigopulos started out by breaking down classics such as Herbie Hancock’s “Watermelon Man” and identifying aspects of the music that users could change. People are used to turning the volume of a song up or down, but what if there were a knob on the stereo that could make a song sound “more excited” or “sadder”? They decided to develop an intuitive way for a user to control these aspects of the music.

Egozy says that as the project’s engineer, he would hack while Rigopulos, the “idea guy,” played with a flight simulator on his computer and helped brainstorm ideas. One day, the two activities came together. “Alex says, ‘Hey, Eran, what do you think about hooking up this joystick to this music system that we’ve been working on?’” Egozy recalls. The result quickly became a favorite stop on Media Lab tours. Egozy remembers demonstrating it to dozens of people, including the musician Peter Gabriel.

When the pair graduated, they wanted to keep working on their project. They founded Harmonix and built a product called the Axe, a commercial version of their joystick device. The software allowed users to jam along with songs on a CD provided by Harmonix, selecting lead instruments such as piano or guitar and controlling them with a joystick. The Axe gave players real and sophisticated control over the music while preventing the off notes that a beginner would be sure to play on an ordinary instrument.

People who played with the Axe were enthusiastic about it. But without a demonstration, it was hard to explain what the system did–and thus hard to promote it. After the Axe, Harmonix created other products, such as the music games Frequency and Amplitude; they got good reviews but never took off. The company struggled to make its products appealing to a general audience.

Success didn’t come until Harmonix settled on simulating the experience of being a rock star. Before they knew anything about how Guitar Hero would work, Egozy says, he and ­Rigopulos knew that the plastic guitar needed a whammy bar. This lever, which varies string tension and produces a vibrato sound on a real electric guitar, is a key part of most people’s air-guitar fantasies. Egozy and Rigopulos also knew that the game needed to make people feel as if they were onstage, being Jimi Hendrix or Stevie Ray Vaughan. It needed to give them that feeling right away, without forcing them to learn a complicated system first. By designing a game that can adjust to players’ skill levels and convince them that they’re in control, Harmonix found the right balance between geek credibility and accessibility.

“The thing that ends up becoming successful, you only figure out in retrospect,” Egozy says. “The experiments that ended up not working too well were the ones [such as the Axe] that were a little too out there. Maybe you have to start with some things that are out there and then bring them back home. I think we did that.”

But Harmonix isn’t resting on its laurels. It has already announced plans for a new game based on the music of the ­Beatles; the founders are intent on creating a completely new game, not simply another version of Rock Band. And that quest to keep ­reinventing itself keeps Harmonix true to its experimental roots.

Egozy says, “The thing that MIT gives you–for a long time, I didn’t know that it was there–is this sort of drive to innovate, and this notion of not wanting to copy. It’s just this innovation bug that MIT really instills in you.”