Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

  • A fake news article written by OpenAI's GPT-2.
  • Hendrik Strobelt and Sebastian Gehrmann
  • Intelligent Machines

    An AI for generating fake news could also help detect it

    Sometimes it takes a bot to know one.

    Last month OpenAI rather dramatically withheld the release of its newest language model, GPT-2, because it feared it could be used to automate the mass production of misinformation. The decision also accelerated the AI community’s ongoing discussion about how to detect this kind of fake news. In a new experiment, researchers at the MIT-IBM Watson AI Lab and HarvardNLP considered whether the same language models that can write such convincing prose can also spot other model-generated passages.

    Sign up for The Algorithm
    Artificial intelligence, demystified

    The idea behind this hypothesis is simple: language models produce sentences by predicting the next word in a sequence of text. So if they can easily predict most of the words in a given passage, it’s likely it was written by one of their own.

    The researchers tested their idea by building an interactive tool based on the publicly accessible downgraded version of OpenAI’s GPT-2. When you feed the tool a passage of text, it highlights the words in green, yellow, or red to indicate decreasing ease of predictability; it highlights them in purple if it wouldn’t have predicted them at all. In theory, the higher the fraction of red and purple words, the higher the chance the passage was written by a human; the greater the share of green and yellow words, the more likely it was written by a language model.

    detecting fake news
    A reading comprehension passage from a US standardized test, written by a human.
    Hendrik Strobelt and Sebastian Gehrmann
    detecting fake news
    A passage written by OpenAI's downgraded GPT-2.
    Hendrik Strobelt and Sebastian Gehrmann

    Indeed, the researchers found that passages written by the downgraded and full versions of GPT-2 came out almost completely green and yellow, while scientific abstracts written by humans and text from reading comprehension passages in US standardized tests had lots of red and purple.

    But not so fast. Janelle Shane, a researcher who runs the popular blog Letting Neural Networks Be Weird and who was uninvolved in the initial research, put the tool to a more rigorous test. Rather than just feed it text generated by GPT-2, she fed it passages written by other language models as well, including one trained on Amazon reviews and another trained on Dungeons and Dragons biographies. She found that the tool failed to predict a large chunk of the words in each of these passages, and thus it assumed they were human-written. This identifies an important insight: a language model might be good at detecting its own output, but not necessarily the output of others.

    This story originally appeared in our AI newsletter The Algorithm. To have it directly delivered to your inbox, sign up here for free.

    Learn from the humans leading the way in machine learning at EmTech Next. Register Today!
    June 11-12, 2019
    Cambridge, MA

    Register now
    detecting fake news
    A reading comprehension passage from a US standardized test, written by a human.
    Hendrik Strobelt and Sebastian Gehrmann
    detecting fake news
    A passage written by OpenAI's downgraded GPT-2.
    Hendrik Strobelt and Sebastian Gehrmann
    More from Intelligent Machines

    Artificial intelligence and robots are transforming how we work and live.

    Want more award-winning journalism? Subscribe to Print Subscription.
    • Print Subscription {! insider.prices.print_only !}*

      {! insider.display.menuOptionsLabel !}

      Six print issues per year plus The Download delivered to your email in-box each weekday.

      See details+

      12-month subscription

      Print magazine (6 bi-monthly issues)

      The Download: newsletter delivery each weekday to your inbox

    /3
    You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.