At the Web 2.0 Expo today in San Francisco today, Twitter’s Alex Payne discussed the technical details of the programming language he hopes can help his company handle the upswing in traffic it has experienced over the past few years. The company is leaving behind a programming language that has caused it much pain in the past, and instead embracing a new and somewhat obscure language called Scala.

Some background: Twitter, a service that allows people to post 140-character messages to friends and the public, was launched in 2006 and is now estimated to have roughly eight million unique users. When a person posts a short message to Twitter, the service posts it to the Web and sends it to people’s cell phones and Twitter software applications. The concept is simple, but under the hood, the technology is more complicated.

The popular Web programming language Ruby on Rails is responsible for the look and feel of Twitter’s user interface, as well as that of many other websites. Since the user interface, known as the “front end,” relied on Ruby, it also made sense to use Ruby for back-end operations like queuing messages. But as Twitter’s popularity grew, the Ruby-built back end wasn’t able to handle the torrent of messages that came its way–a situation that created the sudden popularity of the “Fail Whale,” the error message that Twitter sends to users whenever the service crashes.

So the Twitter team turned to Scala, a programming language with its origins in work by Martin Odersky, professor at EPFL in Lausanne, Switzerland, around 2003. During his presentation, Payne, who’s also writing a book on the language, explained that Scala has many of the benefits of other languages but without the drawbacks. Some of the characteristics that make Scala so appealing to Twitter is that it’s able to efficiently handle concurrent processing–that is, separate instructions that need to use the system’s resources at the same time. This is useful when messages from millions of people need to be sent out instantly to different devices all over the world.

It’s also flexible for programmers to use, says Payne. If a programmer wants more structure, then Scala offers structure, but if she wants more free-form programming, it allows for that as well. And importantly, for Payne and the engineers at Twitter, Scala is a new, exciting, and “beautiful” language that keeps the team engaged. There’s still room for programmers to feel like they’re contributing to the development of something new and fresh. This isn’t the case with more established languages, like Java or C++.

Scala isn’t perfect, notes Payne, but its benefits far outweigh its drawbacks. The most glaring drawback is that it’s somewhat difficult to learn because it has a ton of features and a syntax with which some programmers might not be familiar, he says. Additionally, it’s a relatively new, which means that it doesn’t have a proven track record. But Payne says he and Twitter are willing to take the risk because the language already worked well in a number of test cases.

Right now Twitter’s service is a hybrid of programming languages, Payne says. The user interface runs on Ruby on Rails, which is “fine for people clicking around Web pages,” he says. But by the end of the year, Twitter hopes to have a set of services in the back end that are written entirely in Scala. And it’s the company’s plan to make sure that all the third-party services that connect to Twitter via the application programming interface (API) go through Scala code, bypassing Ruby on Rails completely. “When you’re talking about a bunch of programs hitting the API rapidly,” Payne says, “We found we can better optimize things…using Scala.”