Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

Quarantine for Cyberbullies: The Latest Strategy in the Fight Against Offensive Social Media Content

If people post offensive content, cutting off their contact to the network can prevent their messages from spreading, say network theorists.

  • August 11, 2014

Cyberbullying and hate mail is one of the scourges of the modern age. There are numerous well documented cases where this kind of behavior has made people’s lives a misery and even lead to the victim committing suicide.

Clearly, an important question is how to deal with this kind of behavior. On many social networks, it is possible to report offensive material and to block those responsible for it. On Twitter, for example, it is straightforward to block the tweets from another individual. But that does not stop these messages from reaching a broader community.

Today, Krystal Blanco at Boston University and a few pals say there is another way. These folks have developed a mathematical model of the way messages spread through networks like Twitter and say that a quarantine, in which offensive individuals are banned from contact with the community, is an effective way of preventing their messages from spreading.

Blanco and co begin by constructing a mathematical model of the way messages spread based on a 1965 model of rumor mongering. In this model, individuals are either ignorants, spreaders, or stiflers of information. In addition to these categories, Blanco and co introduce a category of quarantined users who are unable to contact other members of the population and so are unable to spread their messages.

There are a number of free parameters in this model, such as the number of links between individuals, the percentage of people who are stiflers to start off with and the probability that a user who is not a stifler of information then becomes one.

To measure these parameters are in the real world, the team searched Twitter for offensive homophobic tweets including the words “gay” and “disgusting.” They then extracted the parameters from the network associated with the spread of these messages.

They then go on to show that the rate of retweeting in a population of users is lower when some are quarantined than when they are not. “The tweet on average dies out more quickly,” they say.

In some ways, that is exactly what you might expect. A more difficult problem is how to enforce the quarantine effectively in the first place, something that this study sweeps under the carpet.

Possibilities include using natural language filters to pick out tweets that are likely to be offensive and then quarantining the authors. Another is a peer review model in which people rate the offensiveness of tweets and those responsible for the content deemed most offensive are quarantined.

But these approaches raise all kinds of practical questions: where should the cut-off lie between people who should be quarantined versus those who should not; how long should individuals be quarantined for, and so on. And what of users simply reregister under another name?

Just whether such a system would be workable in practice is by no means clear. What is clear, though, is that organizations running social networks need to take active steps towards minimizing and even preventing the kind of tweets that can lead to some unfortunate users paying the ultimate price.

Ref: arxiv.org/abs/1408.0694 : The Dynamics of Offensive Messages in the World of Social Media: the Control of Cyberbullying on Twitter

Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.

Subscribe today

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.