Rarely cited, hardly heard of by players, Ubisoft's editorial user research department quietly tracks the way players play before, during and after a game's launch, and makes sure the development team knows what needs to be fixed, what players hate and what players love.
"We gather a lot of data based on what players do and say," said Sebastien Odasso, Ubisoft's director of editorial user research, which started in 2001. "We make them come to our facilities, to the labs, and we make them play the game and observe them through direct active observation and then we use a lot of methodologies and protocols based on psychology and ergonomics to analyze how they play."
Once the data is gathered and aggregated, it's handed off to the development team to show them what works and doesn't work.
"We give this feedback to the development team so they can fix the issues to improve the quality of a game before release," he said. "It's meant to be an iterative process."
Ubisoft has been using deep user research to track player engagement with games for about eight years. While it initially was relegated to bringing gamers into a lab to play an unreleased game, the group's work has since been expanded to include beta tests and even live games.
The key motivator behind this data-intensive approach to user research is the knowledge that everyone sees things from a different perspective.
"Our users, our players won't see the game as we intended," he said. "Every player will see the game differently and that's something that needs to be kept in mind."
The tools used by the team vary greatly. Sometimes it's something as simple as an in-game survey, asking players to press the D-pad in one direction to choose, after a specific moment in the game, whether they liked it. Other tools include using special gear to track where a player is looking as they play, to ensure that they are looking where the designer wants them to be looking. The team also combines gameplay data with graphics to do things like create heat maps to show where players are and aren't going on a map. That approach can sometimes inspire a developer to rework an area to make it more challenging, easier to find, or any other number of solutions.
Afterbeta, the user research group discovered that players weren't happy with one element of the map.
"When we gathered data from the players we observed that they liked the game, but one thing they didn't like was navigation," Odasso said. "They said it took too long to travel and was too tiresome to get from point A to point B and move between missions."
Odasso and his team gave that information to the developers.
"They said in the game you can tame animals to make them fight for you," he said. "Why can't they ride the beasts to go faster? When we playtested it again after the change was made, they said navigation wasn't an issue."
The team was also quick to spot that The Division's original compass was too complex and too loaded down with visual indicators to be useful. They've also been able to spot when a particular move in a game likeis not being used enough, allowing the developer to tweak its use or how it's performed.
"We are working with the development team very early in the process and throughout the process," he said. "Throughout production and even now after the game releases."
Currently, Ubisoft has 13 user research labs spread out around the world. The two largest are located in Montreal and Paris. In those two labs alone last year, Ubisoft held 203 play tests that were attended by 1,923 players.
As games continue to evolve, so does the methodology and tools used to track and analyze play.
"We are exploring biometrics know to see if we can go deeper," Odasso said. "We're looking at little sensors playtesters might wear on their fingers or wrists to better evaluate the emotion of players in real time, but that still in R&D."
He said the team is also already working on unique ways to gather statistics on gamers as they play virtual reality titles.
"We came up with some very interesting insight," he said, "We can predict if a player will be motion sick in a game thanks to a questionnaire and several other things."