Skip to Content
Silicon Valley

The mass shooting in New Zealand shows how broken social media is

March 15, 2019

A gunman live-streamed the murder of dozens of innocents in two mosques in Christchurch, New Zealand, on Friday—and the world got a terrible reminder of how flawed existing social-media policies and algorithms are for policing violent and offensive content.

In the days before the shooting, the perpetrator apparently boasted of his plans and posted an online manifesto. He then broadcast the horrific act live on Facebook. The attack left 49 people dead and dozens more injured.

Live stream: Over the past 18 months, following harassment and fake-news scandals, social-media companies have invested heavily in content moderators. But this did little to stop video of the shooting from spreading. Not only was the live stream reportedly up for 20 minutes, but the resulting video was then reposted on YouTube, with some clips remaining up for over an hour.

Several factors contributed to letting the footage slip through the filters, according to experts.

Real-time challenge: It’s vital to catch a video quickly, so that it doesn’t spread onto other platforms. But social-media moderation simply isn’t geared toward catching content in real time. It is impossible to automate the process effectively, and identifying live streams that need to be shut down manually is “like finding a needle in the haystack of data that’s flowing over the network all the time,” says Charles Seife, a professor at NYU’s School of Journalism. He adds that Facebook could require users to build up a reputation before letting them live-stream content, to reduce the risks.

Whack-a-mole: Moderators are overwhelmed at the best of times. Video of the shooting hosted on YouTube most likely spread so quickly that the humans employed to check for inappropriate content didn’t have time to catch everything. These workers typically have a few seconds to make a call. The process can be partly automated, but those who reposted the footage apparently clipped it and introduced distortions to avoid these algorithms.

Algorithmic failure: Social-media companies also use algorithmic tweaks to de-prioritize suspicious content. But Mike Ananny, an associate professor at the University of Southern California, says these algorithms were probably thrown by the popularity of the offending videos.

Not our problem: These factors reflect the key systemic problem: Facebook, YouTube, and other big social platforms do not see themselves as the arbiters of content in the first place. Research has shown that far-right sources of information can be policed more proactively to prevent violent or hateful material from spreading. “They have this attitude of being post hoc,” says Ananny. “It’s a deep cultural thing.”

Keep Reading

Most Popular

images created by Google Imagen
images created by Google Imagen

The dark secret behind those cute AI-generated animal images

Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.

biomass with Charm mobile unit in background
biomass with Charm mobile unit in background

Inside Charm Industrial’s big bet on corn stalks for carbon removal

The startup used plant matter and bio-oil to sequester thousands of tons of carbon. The question now is how reliable, scalable, and economical this approach will prove.

AGI is just chatter for now concept
AGI is just chatter for now concept

The hype around DeepMind’s new AI model misses what’s actually cool about it

Some worry that the chatter about these tools is doing the whole field a disservice.

Peter Reinhardt
Peter Reinhardt

How Charm Industrial hopes to use crops to cut steel emissions

The startup believes its bio-oil, once converted into syngas, could help clean up the dirtiest industrial sector.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.