How market research changes your games

Somewhere, probably as you read this, people are sitting in a room attempting to compile every piece of video game-related information that exists. They then turn that minutiae into data, cataloging the news, media, trivia and more, and save it on a gigantic server.

This data defines how players describe games, with modifiers for high-end analytics. World of Warcraft shows up as an "action RPG" with a "massively multiplayer" modifier. Metroid Prime is a "first-person shooter" with elements of "action" and sub elements of "platforming." If ever there is any debate about a game’s classification, a team meeting occurs to assure all tags are as close to accurate as possible.

This is market research.

Yet it's only step one of the process. Market research also studies you, the player.

Who is it that might identify as both a first-person shooter fan, and, for example, an iPad user? What interests would this hypothetical person have in common with other game players?

Market research figures out, say, what percentage of players is likely to have disposable income, how many are male and female, how old they might be and how often they play video games. And most importantly for marketers and developers, research can uncover which combinations of variables are the most lucrative.

For years, the science of mining video games and their demographics has been the invisible hand that guides your favorite video games.

And statistically, you like it that way.


The birth of EEDAR

Market research has a long history, existing for as long as there have been advertisements. You can trace it as far back as the 1920s when companies began analyzing newspaper copy and radio dialogue for effectiveness. And it has continued to adapt to new technologies, from the Bell company popularizing telemarketing in the 1970s, to research companies like the NPD Group including video game sales in their retail tracking in the late 1990s. At that time, however, only the broadest of information was available to companies that did not make data for themselves.

In 2006, two former Sony Online Entertainment employees hatched an idea — not only would they pull data centered around new games in development, they would classify the DNA of all video games. Who were these two? Greg Short, former director and product manager at Sony Online Entertainment and Geoffrey Zatkin, a developer on the original Everquest team and then lead game designer at Monolith Productions.

"Are those five Metacritic points coming from people not floating in the air? That’s a pretty bad problem. Are they coming from having eight more trees in the jungle? Nobody cares."

The ideas at their table: Game producers continue to exist in a precarious business that is steep in failed ventures, while furthermore, statistical tools like focus groups, surveys and more had long been commonplace and independently available to game makers, yet Short and Zatkin felt they were rarely used to their fullest potential. Short and Zatkin knew they could connect those dots to gather more useful data, so they opened Electronic Entertainment Design and Research that same year.

Having founded MMO fan network Casters Realm, Short knew the advantages of maintaining detailed database management. And as a creator of massive virtual communities, Zatkin viewed gamer feedback as critical to a game’s success.

Their combined insights helped create one of the earliest gaming-exclusive research companies. EEDAR is hidden away in a discreet office space in Carlsbad, Calif., and it’s a Pentagon of video game knowledge. Lacking the stiffness of a corporate office, its insides are similar in flair to the average development studio. Framed posters of video games line every inch of its wall space, employees drown their workspaces in video game swag, and meetings take place in rooms labeled "Black Mesa" and "The Mushroom Kingdom."

Head upstairs, and you will find employees mining video games for their data. Go downstairs, and you will be in the company of life-sized statues of zombies and Lara Croft hovering over the break room. A massive piece of Street Fighter art presides as the focal point of the copy room. Even the bathrooms guide patrons with Pac-Man and Ms. Pac-Man symbols.

As its Guinness World Record plaque attests, it is the world’s largest market research firm that specializes in video games. The company boasts over 100 million internally researched data points — and it's still counting. EEDAR’s business model: sell its digital "genome" of interactive entertainment to interested parties, including developers looking to create, publishers looking to invest and retailers looking to catalogue.


EEDAR thrives on a client base of who’s who in the gaming industry. Clients include Sony, Nintendo, EA, Walmart, GameStop and Mattel. Yet despite collaborating frequently with major companies, market research tends to cast a smaller cultural shadow in relation to the influence it exerts. Since the company sometimes works with nondisclosure agreements and unannounced projects, the general public’s knowledge of gaming research can often be muted.

Nevertheless, things such as subgenre elements, popular memes, advertising methodology and Metacritic scores are not trivial factoids, but the blueprints for what EEDAR creates.

"[If] you want to understand your players and your consumers, you need to have a plan in place to implement the right research at the right time, and at the right milestone," says Robert Liguori, CEO of EEDAR.

Should you spring to develop a massively multiplayer RPG? How is the market for first-person shooters versus third-person shooters? Should your company hop into the toys-to-life market? These are the "what if" scenarios EEDAR thrives on.

Developers often want answers the same questions, says Short: "What if I don’t do multiplayer? What if I don’t do a CGI trailer? What if we don’t do TV ads but we just advertise in the store? What if we push the release date out and it gets us five more Metacritic points? How much will those points help?

"The answer will depend. Are those five Metacritic points coming from people not floating in the air? That’s a pretty bad problem. Are they coming from having eight more trees in the jungle? Nobody cares. We help give information so that clients feel comfortable about the investments they make."


What are you buying? What are you sellin’?

So let’s say you’re sitting on a great idea for a video game. Or perhaps, you’re thinking of green-lighting the latest entry in your long-running, blockbuster franchise.

At EEDAR, the process begins with GamePulse, an "intelligence tool" that the company claims 90% of "top video game companies" regularly access. For a fee, you can search an enormous number of titles and sort through their classifications, view their development and marketing budgets and sales numbers and more. Access to this robust reference tool is typically the first stop in market research and often guides developers’ business decisions.


The next level of service involves honed in, analytical consultation from panels of experts. If a developer wants to understand the marketplace in relation to a given proposal, experts will analyze their game concept to figure out how to best begin or continue development. In other cases, they can analyze a title post-release and find out what went wrong, or conversely, find out exactly why a title succeeded.

"Confidentiality makes it difficult to share the specifics of clients and projects," says Patrick Walker, vice president of Insights and Analytics at EEDAR, though some details are made public for this story.

"In the beginning of 2012, THQ had a very difficult decision to make regarding their planned MMO, Warhammer 40,000: Dark Millennium Online," he says. "Over years of development, THQ had already invested a significant amount of money in the title, but the shift to free to play in the MMO landscape was in full effect."

In 2012, the market drifted away from THQ. Hit games like League of Legends, DC Universe Online and Dungeons and Dragons Online marked a change in consumer expectations and how much players were willing to pay. Being a long-term project, Warhammer 40,000 originated as a subscription model years ahead of free-to-play trends. It became vulnerable as a result.

"The research question [THQ gave] was, ‘What is the game’s likely revenue, based on subscription MMORPGs?’ [For us], it was very much an exercise on analytics ... Our reports suggested that it was going to be very difficult to successfully launch a subscription model at that time."

"Our reports suggested that it was going to be very difficult to successfully launch a subscription model at that time."

THQ’s options were to lose the money and time invested on development or to forge ahead knowing the economic outlook for the game was at a high risk to miss its mark.

"THQ announced the title would not be an MMO in March 2012," says Walker. "Although the cancellation of a title is always unfortunate, the decision helped THQ save some much needed cash that supported other releases, such as the excellent Darksiders 2 later that year."

If your company is looking for a way to market already developed titles, there are specialists for this as well. Frank N. Magid Associates is a media market research company that focuses on creating frameworks for businesses. Mike Vorhaus, president of Magid Associates, tells Polygon how their analytics helped guide one particular gaming project that allowed EA to monetize its series of public domain and licensed genre titles.

"We conducted advanced research for [EA] ... and provided our consultative advice for how to build Club Pogo and how to launch it."

Vorhaus suggests that other types of entertainment, like board and card games can be difficult to monetize because they’re everywhere, but analytics can help companies overcome those types of hurdles.


"The nature of the games in Club Pogo being similar to many readily available games did not create a problem for us. We found consumers very excited about no ads, enhanced points, special servers and chat rooms and special badges."

Their research suggested that they bring new features to old games and make the titles accessible all at once, creating a new brand name in the process.

"It was, and still is, a huge success with well over one million paying members after launch."

In addition to analytical reports and consultations, another tool in the market research playbook is the mock review.

Like consumer reviews, these outlined critiques suggest which sections of a game work well and which do not. Mock reviews can also show what can realistically be done to improve potential review scores based on consummate advice.

Market research firms like EEDAR and others often sell in-progress reviews to companies, many times even before the respective title is announced to the public. In EEDAR’s case, veteran game journalists who are kept on staff author each review.

EEDAR touts its mock reviews as having 90% accuracy to a game’s eventual, cumulative review score.


Market research as homogeneity

In a 2012 interview with Wired, game maker Ken Levine reacted to fan sentiment that the box art and marketing for his latest AAA blockbuster BioShock Infinite were notable departures from the macabre, psychologically tinged gameplay the series is known for. For the game’s marketing, the protagonist stands brooding, rugged and visibly bearing arms. This was a slap in the face to series devotees, and Levine acknowledged the disconnect.

"It's disconcerting to know it's someone's job to learn how best to extract money from frustrated players."

"We went and did a tour ... around to a bunch of, like, frat houses and places like that," he said. "People who were gamers, [but] not people who read IGN. And [we] said, so, have you guys heard of BioShock? Not a single one of them had heard of it."

To the uninformed, BioShock Infinite’s marketing appeared indistinguishable from a traditional run-and-gun shooter. This was exactly what the game’s publisher, 2K Games, was aiming for.

"I looked at the cover art for BioShock 1, which I was heavily involved with and love, I adored. And I tried to step back and say, if I’m just some guy ... what would I think? And I would think: This is a game about a robot and a little girl," said Levine.

There is a lingering sentiment among critics that games are overwrought with recurring tropes; Machismo, guns, zombies, and male characters are the lifeblood of the industry, and this is indisputable. If, then, market research is what leads developers by the hand to more tired themes, an argument could be made that their doorstep is where creativity wilts.

"I understand the concern over micromanaging a game's experience based on cold data," says Paste Magazine and Killscreen contributor Jon Irwin. "The result can be impersonal or exploitative. I have no idea how King figures out the best way to craft their free-to-play puzzle sagas, but it's disconcerting to know it's someone's job to learn how best to extract money from frustrated players.

"I'm not necessarily opposed to a more direct approach, either, casting a wide net and asking players what they've liked or disliked about games they played ... I think the larger problem is when such a reliance on data infringes on a creator's original vision."


An example Irwin cites is Ubisoft’s User Research Lab, where biological responses to game playing areanalyzed, including vital signs and visual perceptions.

Irwin continues, "The fear is that relying on such math to figure out an 'ideal' experience ultimately funnels games down a single path, leading to homogenization and a more generic experience that may run counter to the design team's ideas."

This is not how Patrick Walker sees it.

"Here is what we’ve seen in the industry ... things are way more diverse now."

From Walker’s standpoint, genre hybridization and socially connected games are logical evolutions for major titles.

"Creativity is built on things that have been done before," says Walker. "Yes, AAA games are starting to look more homogenous with RPG elements, etc. But that’s because game design is getting better. It turns out, progression and exploration are a good thing."

"AAA games are starting to look more homogenous with RPG elements, etc. But that’s because game design is getting better."

A world where market research didn’t exist, Walker suggests, would cultivate fewer risks, not more. "Market research helps companies figure out ‘the line.’ Without it, you’d think there’d be more risks. But really, companies would just play it safe."

SuperData Research is a market research firm that focuses on digital platforms and qualitative data. CEO Joost van Dreunen agrees that market research is net positive for game creativity. "Like all art forms, game design exists within a larger context," he says. "In my experience, the nature of game design is working within a framework of limitations, whether those are set by a financially conservative publisher looking to meet market demand or by the fact that you’re a cash-poor indie studio.

"There is as much glory in designing a new, cool game that no one has ever seen as there is in perfecting an existing mechanic or genre."

Those working within market research will remind you that the market could not have predicted the Nintendo Wii or Minecraft. But to those spending the greatest amount of money while trying to keep their doors open, what matters most is whether the market can consistently predict the successes of more likely ventures like another Call of Duty or Grand Theft Auto 5.

In the Wired story, Levine agreed. "Games are big, and they’re expensive ... they have to be financially successful to keep getting made."


Giving you what you want

As a reliable, profitable science, present day market research is now a permanent cog in the game making machine. With the game industry’s net worth rapidly approaching twelve digits, market research is a crucial navigation system where a lot of money is at stake.

Expert, outside opinions are worth money. From inside developer’s doors, gamers themselves are commonly recruited for their data, often in exchange for downloads, posters, or T-shirts, among other things. And the information the researchers gather helps them unearth our desires for things — things like massively multiplayer online titles, fantasy card games, and human simulations. When researchers identify a genre as in vogue, developers can more easily find their funding and their titles are more likely to be gobbled up by fans.


Databases are being stockpiled, markets are being identified, and more specific experiences are being created as a result.

For those fresh ideas that gamers don’t realize they want yet — and that market research may seem to stand in the way of — their fates lie squarely in the hands of the producers and developers, according to Short.

"We can never really directly point to a lot of instances and say, ‘We made this happen. This would never happen in this game if it wasn’t for us.’ Even though we come back to the developer or publisher with information, they ultimately have to make that decision. Our job is only to assess risk."

In addition, says Liguori, "... When you look at the earlier days of EA and Atari and Sega and even Microsoft ... these guys were definitely making decisions based on their knowledge and passion of what they believed in and what they wanted, and it happened to resonate very well with consumers at the time. Technology has very much changed things ... consumer expectations has changed things [from] where they were many years ago."

Creators are passionate, and they’re often hungry to take their great ideas and run with them, but they are also often aware of the need for outside opinions — especially those informed by hard numbers.

"You can build a game that you love, or 10 or 20 people love, but at the end of the day, you want to make the game people want to play."

Continues Liguori, "... You can’t just build the game that you want to build. You can build a game that you love, or 10 or 20 people love, but at the end of the day, you want to make the game people want to play ... what do they want?"

All this adds up to a reality where the more people there are interested in video games, the less varied larger games can realistically become — but at the same time, a larger variety of games can exist.

Because of this, over the course of market research’s evolution, many shocking risks have been taken in the field, and plenty of games have unabashedly copied one another. Both AAA titles and indie games alike have shown great promise and failed. Gamers have had the year of the bow, and the year of Luigi. Trends caught fire then fell into obscurity.

History suggests that future research results will reflect a mix of marketplace trends and gamemaker imagination, however timid or wild they both may be. Consumers will continue to reinforce what gets created, even if they don’t realize when researchers are paying attention.

Market research is ultimately the study of us. And the games that we play are, in many ways, a reflection of what we suggest we want to see. Babykayak