The news: Microsoft has created an automated system to detect sexual predators trying to groom children online. The tool, code-named Project Artemis, is designed to spot patterns of communication in conversations.
Rating: On the basis of words and patterns of speech, the system assigns a rating for the likelihood that one of the participants is trying to groom the other. Companies implementing the technique can set a score (for example, 8 out of 10) above which any flagged conversations are sent to a human moderator to review. The moderators could potentially identify imminent threats and report them to law enforcement. It would also provide child protection experts with more information on how pedophiles operate online. Microsoft has been using these techniques for several years for its own products, including the Xbox platform and Skype, the company’s chief digital safety officer, Courtney Gregoire, said in a blog post.
How does it work? Microsoft hasn’t explained the precise words or patterns the tool hunts for—doing so could potentially cause predators to adjust their behavior to try to mask their activities. The tool is available free for companies that provide online chat functions, through a nonprofit called Thorn, which builds technology products to defend children from sexual abuse.
The risks: The system is likely to throw up a lot of false positives, since automated systems still struggle to understand the meaning and context of language. That means social-media companies will need to invest in more moderators if they are truly committed to tackling online grooming (and victims argue it is not clear that they are). The system also assumes that messages are not encrypted and that users consent to their private communications being read, which is not necessarily a given.
Sign up here to our daily newsletter The Download to get your dose of the latest must-read news from the world of emerging tech.
Yann LeCun has a bold new vision for the future of AI
One of the godfathers of deep learning pulls together old ideas to sketch out a fresh path for AI, but raises as many questions as he answers.
The dark secret behind those cute AI-generated animal images
Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.
The hype around DeepMind’s new AI model misses what’s actually cool about it
Some worry that the chatter about these tools is doing the whole field a disservice.
AI’s progress isn’t the same as creating human intelligence in machines
Honorees from this year's 35 Innovators list are employing AI to find new molecules, fold proteins, and analyze massive amounts of medical data.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.