Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

The Evolution of English Words and Phrases Since 1520

The digitisation of the world’s books reveals how the popularity of English words and phrases has evolved since the 16th century. And the Top 100 lists for each year are now free to browse online

  • December 11, 2012

The digitisation of the world’s books reveals how the popularity of English words and phrases has evolved since the 16th century. And the database is now freely browsable online

Last year, the Google Books team released some 4 per cent of all the books ever written as a corpus of digitised text, an event that has triggered something of a revolution in the study of trends in human thought. The corpus consists of 5 million books and over 500 billion words (361 billion in English) dating from the 1500s to the present day.

In a single stroke, this data gives researchers a way to examine a whole range of hitherto inaccessible phenomena. Since then a steady stream of new results has emerged on everything from the evolution of grammar and the adoption of technology to the pursuit of fame and the role of censorship.

Today, Matjaz Perc at the University of Maribor in Slovenia uses this data to examine the evolution of the most common English words and phrases since 1520. 

As expected, the results show various power laws at play. Power laws are thought to arise in social systems because of a phenomenon of self-organisation called preferential attachment.

This is the idea that in a network, a node with more connections is likely to attract more connections in future. That’s why it is also known the rich-get-richer effect or the Matthew effect (a biblical reference).

So it’s really no surprise that the popularity of words and phrases over time follows a similar law, given that the spread of language can be modelled by a network model.

What’s interesting about Perc’s work, however, is that he’s published the results on his website at http://www.matjazperc.com/ngrams/evolution.html.

Here you can see lists of the top 100 most popular 1, 2, 3, 4 and 5-word phrases for each year of data from 1520 until 2008.

It makes for fascinating browsing.  For example, the most popular 5-word phrase in 1520, the first year for which data is available, is “the pope and his followers”. In 2008, the last year for which data is available, it is “at the end of the”. 

Worth a look if you have a few minutes to spare.

Ref: arxiv.org/abs/1212.1709: Evolution Of The Most Common English Words And Phrases Over The Centuries

The latest Insider Conversation is live! Listen to the story behind the story.

Subscribe today
Already a Premium subscriber? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look. Exclusive early access to stories.

    Insider Conversations. Listen in as our editors talk to innovators from around the world.

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.