Skip to Content
Uncategorized

The Secret Behind Twitter’s Growth

How a new Web programming language is helping the company handle its increasing popularity.
April 1, 2009

At the Web 2.0 Expo today in San Francisco today, Twitter’s Alex Payne discussed the technical details of the programming language he hopes can help his company handle the upswing in traffic it has experienced over the past few years. The company is leaving behind a programming language that has caused it much pain in the past, and instead embracing a new and somewhat obscure language called Scala.

Some background: Twitter, a service that allows people to post 140-character messages to friends and the public, was launched in 2006 and is now estimated to have roughly eight million unique users. When a person posts a short message to Twitter, the service posts it to the Web and sends it to people’s cell phones and Twitter software applications. The concept is simple, but under the hood, the technology is more complicated.

The popular Web programming language Ruby on Rails is responsible for the look and feel of Twitter’s user interface, as well as that of many other websites. Since the user interface, known as the “front end,” relied on Ruby, it also made sense to use Ruby for back-end operations like queuing messages. But as Twitter’s popularity grew, the Ruby-built back end wasn’t able to handle the torrent of messages that came its way–a situation that created the sudden popularity of the “Fail Whale,” the error message that Twitter sends to users whenever the service crashes.

So the Twitter team turned to Scala, a programming language with its origins in work by Martin Odersky, professor at EPFL in Lausanne, Switzerland, around 2003. During his presentation, Payne, who’s also writing a book on the language, explained that Scala has many of the benefits of other languages but without the drawbacks. Some of the characteristics that make Scala so appealing to Twitter is that it’s able to efficiently handle concurrent processing–that is, separate instructions that need to use the system’s resources at the same time. This is useful when messages from millions of people need to be sent out instantly to different devices all over the world.

It’s also flexible for programmers to use, says Payne. If a programmer wants more structure, then Scala offers structure, but if she wants more free-form programming, it allows for that as well. And importantly, for Payne and the engineers at Twitter, Scala is a new, exciting, and “beautiful” language that keeps the team engaged. There’s still room for programmers to feel like they’re contributing to the development of something new and fresh. This isn’t the case with more established languages, like Java or C++.

Scala isn’t perfect, notes Payne, but its benefits far outweigh its drawbacks. The most glaring drawback is that it’s somewhat difficult to learn because it has a ton of features and a syntax with which some programmers might not be familiar, he says. Additionally, it’s a relatively new, which means that it doesn’t have a proven track record. But Payne says he and Twitter are willing to take the risk because the language already worked well in a number of test cases.

Right now Twitter’s service is a hybrid of programming languages, Payne says. The user interface runs on Ruby on Rails, which is “fine for people clicking around Web pages,” he says. But by the end of the year, Twitter hopes to have a set of services in the back end that are written entirely in Scala. And it’s the company’s plan to make sure that all the third-party services that connect to Twitter via the application programming interface (API) go through Scala code, bypassing Ruby on Rails completely. “When you’re talking about a bunch of programs hitting the API rapidly,” Payne says, “We found we can better optimize things…using Scala.”

Deep Dive

Uncategorized

Our best illustrations of 2022

Our artists’ thought-provoking, playful creations bring our stories to life, often saying more with an image than words ever could.

How CRISPR is making farmed animals bigger, stronger, and healthier

These gene-edited fish, pigs, and other animals could soon be on the menu.

The Download: the Saudi sci-fi megacity, and sleeping babies’ brains

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. These exclusive satellite images show Saudi Arabia’s sci-fi megacity is well underway In early 2021, Crown Prince Mohammed bin Salman of Saudi Arabia announced The Line: a “civilizational revolution” that would house up…

10 Breakthrough Technologies 2023

Every year, we pick the 10 technologies that matter the most right now. We look for advances that will have a big impact on our lives and break down why they matter.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.