League of Legends that could separate good players from bad players

Riot also conducted a player-centric analysis of its wealth of chat log data to try

to generate an automatic profile that could separate good players from bad players.

“One of the cooler things we did is we took our whole player base and categorized

the players who are known for toxic behaviors, all the players who are known for

positive behaviors, and we can cross-correlate all the words that both populations

use,” Lin said. “Any words in common we filter out of the dictionaries. What you’

re left with is a dictionary for all the words the bad players use that good players

don’t use.”
League of Legends Image
The dictionary of common words for bad players falls along depressingly predictable

lines, with heavy weighting towards racial and homophobic slurs. The dictionary for

good players, however, turned up an inspiring data point. “The first 500 or so words

were real life names,” Lin said. “The best experiences are when you trust the other

person because they’re a real life friend.”

This kind of chat analysis also makes it possible to predict which players are likely

to run into problems in the game even before they show truly bad behavior or generate

complaints. “It turns out that if you use the dictionaries, you can predict if a

player will show bad behavior with up to 80 percent accuracy from just one game’s

chat log,” Lin said. Of course, Riot doesn’t intend to use this kind of predictive

modeling to take pre-emptive actions on players who may be heading down the path to

undesirable behavior, saying that would contradict the real spirit of the company’s


“The core philosophy of the player behavior team is to work with the players and

collaborate with the players. We want to build systems and tools to allow the players

to hold each other accountable or to provide more social norms to online society,”

Lin said.

The team’s next experiment was to add “Reform Cards” to the Tribunal system.

Previously, the Tribunal could deliver bans of various lengths to players, but the

banned player wouldn’t know why they were punished. Reform Cards show the punished

players the same chat logs and other information that was presented during their

Tribunal case and add statistics on the level of agreement among the player judges on

the recommended punishment.