Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

  • Saiman Chow
  • Intelligent Machines

    An AI Ally to Combat Bullying in Virtual Worlds

    Startup Spirit AI says it has a solution to the growing problem of harassment in online games: a sensitive bot.

    In any fictionalized universe, the distinction between playful antagonism and earnest harassment can be difficult to discern. Name-calling between friends playing a video game together is often a form of camaraderie. Between strangers, however, similar words assume a different, more troublesome quality. Being able to distinguish between the two is crucial for any video-game maker that wants to foster a welcoming community.

    Spirit AI hopes to help developers support players and discourage bullying behavior with an abuse detection and intervention system called Ally. The software monitors interactions between players—what people are saying to each other and how they are behaving—through the available actions within a game or social platform. It’s able to detect verbal harassment and also nonverbal provocation—for example, one player stalking another’s avatar or abusing reporting tools.

    “We’re looking at interaction patterns, combined with natural-language classifiers, rather than relying on a list of individual keywords,” explains Ruxandra Dariescu, one of Ally’s developers. “Harassment is a nuanced problem.”

    When Ally identifies potentially abusive behavior, it checks to see if the potential abuser and the other player have had previous interactions. Where Ally differs from existing moderation software is that rather than simply send an alert to the game’s developers, it is able to send a computer-controlled virtual character to check in with the player—one that, through Spirit AI’s natural-language tools, is able to converse in the game’s tone and style (see “A Video-Game Algorithm to Solve Online Abuse”).

    Ally’s bot might say: “I noticed you and Player 42 seem to be strangers. They may have said something inappropriate to you, so I just wanted to check if you’re okay?” The player is then able to respond to the bot, Dariescu says. If the interaction is dismissed as friendly banter, the system registers this and updates its rules so it won’t intervene over similar behavior in the future. If the user feels abused, the system intervenes with the culprit.

    Ally is compatible with all online games, and Spirit AI is working on adding capabilities to police behavior in virtual-reality games (see “The War on the Disturbingly Real Trolls in Virtual Reality”). The tool can be configured in a variety of ways to interact with players and take different kinds of action. Spirit AI sees Ally as a way to augment a game’s community management team, which can monitor activity in a dashboard.

    Subscribe to The Download
    What's important in technology and innovation, delivered to you every day.
    Manage your newsletter preferences

    The company, which was officially formed in 2016 and currently has a staff of 15, won’t disclose which game developers it is currently working with, but the software has attracted attention in the gaming world. “On the simplest level, AI-supported community management tools can extend the team’s reach,” says Imre Jele, a veteran designer of vast online multiplayer games such as RuneScape. “But what I’m most excited about is AI systems that go beyond identifying and punishing rule-breaking behavior. Ultimately the goal of these AI team members should be to get out of policing and into shaping positive interactions and building healthier communities.”

    Spirit certainly intends for the software to offer more than a mere digitized police force that can be summoned into online games. “We’ve identified the potential for this technology to not only prevent bad behavior but also encourage good, prosocial behavior,” says Dariescu. The software could also use ambient eavesdropping to identify broader issues that players might have with a game’s design, providing insights to the developer as to how the overall player experience might be improved.

    There are, however, limitations when it comes to AI interventions in cases of harassment, according to Jele. Many interactions between community management and player support teams require deep empathy and sympathy, he says: “The players supported the developers with their money and deserve the best possible care, which only humans can provide at the moment.”

    Want to go ad free? No ad blockers needed.

    Become an Insider
    Already an Insider? Log in.
    /
    You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.