Researchers at Western build satire detector for news websites

“Hellmann’s introduces new meat-on-the-bottom mayo cups.” Not sure if this headline is fact or fiction? Just copy and paste the tricky headline and article into the satire detector, and within seconds the site will spit back the answer.

To build the tool, Victoria Rubin and a team of assistants with the Language and Information Technology and Research Lab (LiTRL) at Western University, relied on a mixed bag of expertise. Linguists read over several examples of legitimate and satirical news to note recurring grammatical, stylistic and structural properties that might distinguish one type of writing from the other. They looked for markers of absurdity and humour, and for signs of deception in the writer’s use of prepositions, verbs, articles and pronouns (research in psychology suggests liars avoid first-person pronouns, for example).

Computer and information scientists then developed a series of algorithms that “read” a text for these markers. All of it was packaged into a website that, to date, can make a ruling with up to 84 percent certainty. (It was 84 percent sure that the mayo-cup story, published by The Onion, was satire – to the disappointment of meat-lovers everywhere.)

Dr. Rubin, a professor in library and information science, started working on the satire detector nearly five years ago. Back then, “fake news” generally referred to Stephen Colbert’s news program on Comedy Central, or The Onion website. She notes that the tool has become more relevant today.

“There are more threats to democracy and more threats to our happy, truth-bias existence where we assume that everybody in the conversation is working towards the common goal of understanding each other, that nobody is a con artist,” she says.

The LiTRL team is now looking to integrate the project, funded by the Social Sciences and Humanities Council, into open and user-friendly products like a purpose-built web browser or a Firefox plug-in. She’s also working with faculty in Western’s journalism program to bring the tool to newsrooms.

People are “notoriously bad lie detectors,” Dr. Rubin notes. Unlike a person, a machine “never gets tired, never gets persuaded, doesn’t care whether you smile or not, if you’re a sociopath or not,” she says. Although she supports the adoption of computer-assisted fact-checking and satire-detection, she cautions that tools like hers weren’t built to “substitute human judgment, but assist the human either in news production or news consumption.”

And the satire detector is just one weapon in the LiTRL lab’s battle against deception in media. Dr. Rubin and company will be targeting hoaxes, native ads and clickbait next.