Skip to Content
Uncategorized

How to Distinguish Fiction from Nonfiction

Telling fact from fiction isn’t always easy on the on the Web. Now researchers have discovered a method that could help automate the process.

Pick up a piece of text and start reading and it usually becomes clear pretty quickly whether you’re reading a nonfictional news story or a fictional novel.

Some clues come from the environment where the stories are found which provide hints, such as the presence of headlines, standfirsts and cross heads.

But even the text alone is revealing. News stories, for example, have very specific structures that give writers little room for creative manoeuvre.

But pinning down these differences in a measurable way that a computer might use to tell them apart is a little more tricky.

Now Joseph Stevanak and Lincoln Carr at the Colorado School of Mines in Golden have come up with a way to do it. They say that the key is to look at the networks that form when you examine how often words appear close together in each type of text.

The type of network they examined creates a graph in which each word in the text forms a vertex. A line connects two vertices if these words appear next to each other in the text. It is possible to explore longer range links by connecting vertices when they appear two or three or four words apart and so on.

Stevanak and Carr say that just two properties of this kind of network can help distinguish fiction from nonfiction stories. The first is the power law that describes the number of links to each vertex in the network. The second is the cluster coefficient which describes how well the vertices are connected to the rest of the network.

Measuring these two quantities alone can identify the type of story with remarkable accuracy. “Our analysis yielded a 73.8±5.15% accuracy for the correct classification of novels and 69.1 ± 1.22% for news stories,” say Stevenak and Carr.

This kind of analysis has the potential to improve future generations of text-finding algorithms that they can better classify and hunt down the types of stories that individuals are looking for, and also to identify the communities producing it.

And although it doesn’t look like a Google-beater just yet, it has huge potential. If there’s one place where the ability to distinguish fact from fiction may turn out to be useful, it is surely on the web.

Ref: arxiv.org/abs/1007.3254: Distinguishing Fact from Fiction: Pattern Recognition in Texts Using Complex Networks

Deep Dive

Uncategorized

Five poems about the mind

DREAM VENDING MACHINE I feed it coins and watch the spring coil back,the clunk of a vacuum-packed, foil-wrappeddream dropping into the tray. It dispenses all kinds of dreams—bad dreams, good dreams,short nightmares to stave off worse ones, recurring dreams with a teacake marshmallow center.Hardboiled caramel dreams to tuck in your cheek,a bag of orange dreams…

Work reinvented: Tech will drive the office evolution

As organizations navigate a new world of hybrid work, tech innovation will be crucial for employee connection and collaboration.

lucid dreaming concept
lucid dreaming concept

I taught myself to lucid dream. You can too.

We still don’t know much about the experience of being aware that you’re dreaming—but a few researchers think it could help us find out more about how the brain works.

panpsychism concept
panpsychism concept

Is everything in the world a little bit conscious?

The idea that consciousness is widespread is attractive to many for intellectual and, perhaps, also emotional
reasons. But can it be tested? Surprisingly, perhaps it can.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.