Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo

 

Unsupported browser: Your browser does not meet modern web standards. See how it scores »

At the Web 2.0 Expo today in San Francisco today, Twitter’s Alex Payne discussed the technical details of the programming language he hopes can help his company handle the upswing in traffic it has experienced over the past few years. The company is leaving behind a programming language that has caused it much pain in the past, and instead embracing a new and somewhat obscure language called Scala.

Some background: Twitter, a service that allows people to post 140-character messages to friends and the public, was launched in 2006 and is now estimated to have roughly eight million unique users. When a person posts a short message to Twitter, the service posts it to the Web and sends it to people’s cell phones and Twitter software applications. The concept is simple, but under the hood, the technology is more complicated.

The popular Web programming language Ruby on Rails is responsible for the look and feel of Twitter’s user interface, as well as that of many other websites. Since the user interface, known as the “front end,” relied on Ruby, it also made sense to use Ruby for back-end operations like queuing messages. But as Twitter’s popularity grew, the Ruby-built back end wasn’t able to handle the torrent of messages that came its way–a situation that created the sudden popularity of the “Fail Whale,” the error message that Twitter sends to users whenever the service crashes.

So the Twitter team turned to Scala, a programming language with its origins in work by Martin Odersky, professor at EPFL in Lausanne, Switzerland, around 2003. During his presentation, Payne, who’s also writing a book on the language, explained that Scala has many of the benefits of other languages but without the drawbacks. Some of the characteristics that make Scala so appealing to Twitter is that it’s able to efficiently handle concurrent processing–that is, separate instructions that need to use the system’s resources at the same time. This is useful when messages from millions of people need to be sent out instantly to different devices all over the world.

It’s also flexible for programmers to use, says Payne. If a programmer wants more structure, then Scala offers structure, but if she wants more free-form programming, it allows for that as well. And importantly, for Payne and the engineers at Twitter, Scala is a new, exciting, and “beautiful” language that keeps the team engaged. There’s still room for programmers to feel like they’re contributing to the development of something new and fresh. This isn’t the case with more established languages, like Java or C++.

Scala isn’t perfect, notes Payne, but its benefits far outweigh its drawbacks. The most glaring drawback is that it’s somewhat difficult to learn because it has a ton of features and a syntax with which some programmers might not be familiar, he says. Additionally, it’s a relatively new, which means that it doesn’t have a proven track record. But Payne says he and Twitter are willing to take the risk because the language already worked well in a number of test cases.

Right now Twitter’s service is a hybrid of programming languages, Payne says. The user interface runs on Ruby on Rails, which is “fine for people clicking around Web pages,” he says. But by the end of the year, Twitter hopes to have a set of services in the back end that are written entirely in Scala. And it’s the company’s plan to make sure that all the third-party services that connect to Twitter via the application programming interface (API) go through Scala code, bypassing Ruby on Rails completely. “When you’re talking about a bunch of programs hitting the API rapidly,” Payne says, “We found we can better optimize things…using Scala.”

0 comments about this story. Start the discussion »

Tagged: Web, Twitter, ruby on rails, fail whale, scala

Reprints and Permissions | Send feedback to the editor

From the Archives

Close

Introducing MIT Technology Review Insider.

Already a Magazine subscriber?

You're automatically an Insider. It's easy to activate or upgrade your account.

Activate Your Account

Become an Insider

It's the new way to subscribe. Get even more of the tech news, research, and discoveries you crave.

Sign Up

Learn More

Find out why MIT Technology Review Insider is for you and explore your options.

Show Me