GlassLab: where game designers and teachers come together to fix the way we teach our kids.
By any reasonable measure, Michael John made it big.
Mention the name of the character John helped create and almost anyone who's held a video game controller will remember it: Spyro, the purple dragon that became a sensation when it launched on PlayStation. Now, 16 years later, Spyro is one of the Skylanders, and he's everywhere. Thank Michael John.
John and a team of four other designers at game developer Insomniac created Spyro the Dragon in 1997, and it has since become a 13-game franchise. Insomniac, bolstered by the success of its purple dragon, went on to other blockbuster game projects. John moved on to start his own studio and worked for several others, including game design luminary Mark Cerny's Cerny Games. John did well for himself. Like most game developers, a lot of his work was behind the scenes, or never got released. He kept going. Making games. Making money. He bought a Lamborghini. It was green. Success story.
In his own words, John was doing all the expected successful game developer things, in all the expected ways. Living large. Playing the rock star designer shtick. Driving the green Lambo.
"It was a classic," John says. "What a noise that thing made. ... I have redlined a V12 with my foot. OK, now I can move on."
Then John had a kid, and everything changed.
John wanted to raise his kid right. He took an active role in her education. When it was time for her to start school, John met directly with her teachers to understand how and what they wanted to teach his child, his daughter, his Lily.
It was in one such meeting he had his epiphany. The theories his daughter's teachers were presenting to him sounded familiar. They talked about teaching at one level, then moving to a next, then a next — a process they called "scaffolding." To John, it sounded almost exactly like game design.
"That's what I do every day," John says. "That's my job. It's to do exactly that, to relate to something that seems impossibly difficult, let you bang on it for a while, let you step away from it for a minute or try another route or something like that, and then you come back and now you can get past it.
"So I'm hearing this from a teacher. ... Different language, but so [many] of the same ideas. And the truth is, [scaffolding is] why kids are addicted to games."
And that's when the boulder started rolling down the hill, dragging John, rock star game designer, to where, as he says now, he was headed all along: to use the power of games to help make the world a better place.
John sold the Lambo. He became an advocate for using games to teach kids. Now he's a creative director at EA and the lead designer at GlassLab, a little game company tucked away in a disused corner of EA's campus south of San Francisco. GlassLab is part learning institute, part game company. And it's trying to change the world.
There's a problem with learning in America, according to Jessica Lindl, and it's probably not what you think.
"If you think about the world of assessment, it looks exactly like it did 100 years ago," says Lindl. She's general manager of GlassLab.
"Every middle school kid in the country, and perhaps the world, is playing Minecraft right now."
Lindl is sitting in the grand foyer of the large, corporate office complex at 650 Townsend, right on the edge of San Francisco's Tenderloin District, where GlassLab borrows office space for game testing. The building consists of two parts: one occupied entirely by the mobile game company Zynga, and the other, owned by Zynga, but filled with nonprofits and startups, a sort of incubator. GlassLab occupies one such incubator suite, on loan from Zynga and shared with another company.
GlassLab makes its games to the south of San Francisco, at EA's Redwood Shores campus, and that's where team members bring dignitaries visiting to offer grants or learn about serious games. The kinds of people who don't know anything about game design, but know that EA is a big deal. Here at 650 Townsend, though, is where the studio talks to game developers. The kinds of people impressed by the startup culture. And it's where the group conducts game tests like the one Lindl is about to oversee.
The company has invited a handful of teenagers to play Minecraft and run through some basic tasks designed by GlassLab's testers to evaluate the game's effectiveness for learning applications. Specifically, how effective a tool Minecraft might be for assessment — the art of learning how well students have learned.
According to Lindl, the assessment is just as important as the learning and the teaching, but it's an art that's languished for a century — as she describes it, "kids sitting with number-two pencils filling in bubbles on a high-stakes test, or in a classroom doing worksheets."
Enter: video games.
"Every middle school kid in the country, and perhaps the world, is playing Minecraft right now," Lindl says. "It's an incredibly promising game as far as what kids are already doing in the game."
What attracted GlassLab to Minecraft is its open-ended design and creative construction angle. Kids playing the game are exercising mathematical functions, solving problems and collaborating, concepts teachers have been struggling to get kids to learn in the classroom. Playing Minecraft, these come naturally, because it's fun.
"What we'll be doing in the playtest is watching how kids are working with the game, what they think they're already learning or experiencing in the game, and getting a better understanding of, from a learning and assessment designer's perspective, what are the competencies we think are naturally covered by the game," Lindl says.
Lindl and her team will take the data from the test, analyze it and see how hard it would be to bend the existing game around principles their learning designers might want to teach. If the test results show there's little or no lag between instructing the kids to do a thing and determining how well they learned the lesson, it may someday lead to a partnership with Minecraft developer Mojang.
The result would be a partnership like the one GlassLab forged with EA and game developer Maxis, to create SimCityEDU. But for now, GlassLab is focused on making its own games from scratch — such as a just-released mobile game called Mars Generation One: Argubot Academy.
"People like their children to be educated," says John. "They have a thing about that. They also dislike the way they're currently being educated, by and large. The only thing standing between profit and that idea is just a bunch of bureaucracy. That's formidable, but that's all it is."
Ask anyone how GlassLab came about and the answer is always: "Michael John."
For years prior to GlassLab, John had been one of the most prominent game developers thinking and speaking about the educational opportunities for video games. So when the Gates & MacArthur foundations were looking for someone to kick off an initiative to create learning games in 2011, John got the call.
"Whenever you hear 'Gates & MacArthur,' with the ampersand, that's pretty unusual," John says. The two foundations, while often engaged in similar endeavors, don't necessarily have the same mission. And they're run by different people.
The Gates Foundation, established by Microsoft founder Bill Gates and his wife Melinda, focuses on increasing the health and productivity of individuals worldwide. The MacArthur Foundation, founded by a grant from billionaire banker John D. MacArthur after his death, focuses primarily on the arts and awarding grants to creative pioneers or media entities. For the two foundations to align behind video games as education suggests something profound about how both organizations view the world, and video games' place in it.
Hoping to enlist the support of the Gates & MacArthur foundations, game designer, educator and author Tracey Fullerton gathered a small but diverse group of thinkers and game designers at USC, including John.
"I called it the Games and Learning Workshop or something like that," John says. "They had a bunch of educators there, a bunch of game designers there, all of whom had no idea why they were there. It was just an intentional mash-up."
The meetings hosted lasted for two days, during which the assembled minds gathered into small groups and workshopped specific ideas. Then they all went their separate ways. It seemed to John like, just as with many similar initiatives, nothing was going to come of it. Eight months later he got an invitation to visit EA.
"I sat down in a room, and [former EA CEO] John Riccitiello is there, and a couple of the big brass from EA," John says. "The ESA has brought a couple folks out from D.C. And then the Gates & MacArthur people are in the room. I sit down. They're all, 'Hey, how ya been?' The EA people were very shocked. 'How do you know each other?'"
"This is the first game I've played that has told me my decisions matter. That I might be important as a person. This game reminds me of the real world."
The Ampersands had been paying attention after all, and had cooked up a plan to connect John with EA and tie in the passion displayed at the sessions eight months prior with the resources available south of San Francisco. The result was GAIL, the Games and Assessment Innovation Lab, funded by the foundations and hosted at EA. Eventually GAIL became GlassLab.
EA donated office space and also donated John’s time to set up the GlassLab team (he currently splits his time between EA, GlassLab and teaching at USC) and gave the GlassLab team the source code for the SimCity Glassbox engine, the powerful world simulator driving the new SimCity. GlassLab's first product was a version of SimCity designed to teach simple lessons about pollution and city management, called SimCityEDU, distributed to educators through traditional textbook and educational resource channels in the fall 2013.
"One of the things we found with SimCity was, we were hearing from kids who playtested for us," says GlassLab's lead systems designer Erin Hoffman. "They said, 'This is the first game I've played that has told me my decisions matter. That I might be important as a person. This game reminds me of the real world.' They've never experienced that before, because they're surrounded with games that are constantly putting them in a fantasy place.
"It was fascinating, and very touching, to see that a kid at that age is ready to hear, 'Your ideas are important. Your decisions have an impact.' That empowerment, it really reaches them."
While SimCityEDU was ultimately successful, in that it was released and people played it, the experience taught the designers at GlassLab a few hard lessons about how to create a game designed for learning, and what that type of game could and couldn't do. GlassLab's designers assumed that SimCity, as a deep, data-driven game, would naturally be educational. The reality is that it both is and isn't. While the core game does teach some very basic lessons about resource management and city building, bending those lessons to address anything else is a challenge.
The team decided that in order to effectively teach a specific thing, it would have to build a game entirely from the ground up. Hence, Mars Generation One.
But perhaps more importantly, the experience of building SimCityEDU taught GlassLab that putting game designers in the same room with learning designers to make a better learning and assessment tool was a possibility.
"It seems obvious I guess to us now that it is, but five or six years ago when this conversation was happening, it wasn't obvious at all, that the answer [to assessment] was video games," says Lindl. "Games that the kids were playing outside of the classroom were teaching them an incredible amount of skill. It's just that the data wasn't being served up in a way that was easily understandable to a teacher or to a parent."
The team at GlassLab consists of game designers and learning designers in almost equal measure. The game designers come from all walks of the game industry, representing all disciplines. There are level designers, systems designers, artists and coders. The learning designers are different, but in their own field just as normal. They make tests and textbooks; tools for teaching, learning and assessing.
Just as the people who make games are slowly coming to the realization that what they do might have applications outside of creating fun, the learning designers have also come to realize that the skill they rely upon the most — assessment — is falling radically behind the times. Seeing how their need for a better assessment tool can overlap with the desire of the game makers to create a more meaningful game is what drives the whole mission.
people who make games are slowly coming to the realization that what they do might have applications outside of creating fun.
But building a game development and/or learning company comprised of people from radically different fields was a challenge in itself. Certain ways the two fields conducted business might have been similar, but they went about achieving their goals in radically different ways. And, even more immediate: They spoke completely different languages.
"Game designers speak in prototypes. They speak in stories. Assessment researchers speak in white papers," says Lindl. "A game designer will never read a white paper. Now some of ours may, but it's not traditionally thought of as a mode of communication."
Early meetings at GlassLab revealed key differences in how the game designers and learning designers tackled challenges. The game designers would arrive with working game prototypes to illustrate their points. The learning designers would come armed with pages of research, which no one on the game-design side had read.
"Now I think we're at a point where it's almost flipped, and to my concern, too much," says Lindl. "I have the game designers doing a ton of writing, and the assessment and learning designers doing a lot of sketching. We're trying to find this happy medium that we can all communicate around and use the same language for.
"I think what we're finding is that once these parties get introduced to each other, they do want to talk to each other and find the common ground and the common connection. It's just a lot of Venn diagrams being superimposed on each other. At the center is still the kids. People want to make an impact on them."
"Learning is not broken. But testing is pretty broken. The way that we measure learning has been pretty broken.
"A lot of game developers are parents now, and they're starting to see their kids learn. They're starting to see what their kids are encountering in the education system. The education system is starting to realize, we feel like we're competing with this barrage of entertainment that's around kids, nonstop, all the time. I think initially that was shocking to them. TV is the enemy. Heavy metal is the enemy. Video games are the enemy. They're starting to ask, what if we could harness that instead of making it the enemy? So it's actually a parallel maturation. Both the education system is maturing into not seeing video games as the enemy, and game developers are maturing into seeing that there are things we can do with games other than just pure entertainment."
That's Erin Hoffman, lead systems designer at GlassLab. Hoffman might be the last person you'd expect to see at EA's campus. She made a name for herself in the early oughts as the author of a blog chronicling EA's abusive work practices. Titled "EA Spouse," the blog detailed the experiences of Hoffman and her then-fiancee, who at the time worked for EA.
Hoffman was just out of college, and the game industry was just coming into, as she calls it "a critical mass" where startup business practices were no longer viable and actually harmful to the people caught up in them. Things like long hours without overtime pay, stressful crunch for months at a time, and the assumption that, because people were making games, which were supposed to be fun and creative, the ordinary rules of how to run a business and treat employees didn't apply.
After seeing her fiancee get swept up into pressures of working at a major AAA game publisher, Hoffman says she "kind of lost [her] mind."
"I wrote this blog post that got picked up by the LA Times and the New York Times and [it] became this sort of moment where I think game developers started to look at the way that the industry was being run, the way that people outside the industry were seeing it. It was this weird mirroring effect, where they were seeing how shocked the outside world was that this was happening. It made them question whether it was right. It was kind of a reflection moment."
The blog post led to a class-action lawsuit, which resulted in EA having to pay out millions to its employees. Ask Hoffman about being at EA and she shrugs. Things change. And besides, EA isn't calling any shots at GlassLab. It just donated a room.
The blog post led to a class-action lawsuit, which resulted in EA having to pay out millions to its employees.
"It was weird at first, but I think ... EA is so big that it's impossible to avoid [the company]," Hoffman says. "This is a thing I've learned over the years ... I've just met so many people and seen so many of the different ways that EA has an influence on the industry. What it primarily has, for the mainstream world, is actually a sense of identity and quality. If EA wants to lend itself to something so powerful, and so powerfully good, as GlassLab, then why wouldn't we do that?
"I wouldn't say that everything is perfect now, and it never will be, because of the structure of the system itself, but it's awake. Something like GlassLab is a very awake place. It's very much aware of the outside world. It's very much thinking about the impact that we can have. And I think that's cool. It is something that's changed."
Hoffman's career, like John's, spans many years. But unlike John's, Hoffman's trajectory through the game industry has a bit of a wobbly edge to it, as if she hasn't yet found her niche. She was most recently working with Brenda and John Romero at developer Loot Drop. Before that, at Zynga as a systems designer. Before that, GoPets and other places. She's now the lead game designer at GlassLab and running her own mobile game company on the side. She also writes novels.
"I get bored, probably too easily," Hoffman says. "I always have liked when I've found something in the video game industry that very few people know about. That's very exciting. ... if the educational world is ready for video games, and really honestly ready to appreciate video games for what they are and what they do, then that's massive, because that's not just one new kind of game. That's a dozen different new kinds of games ... and that's very exciting to me."
Hoffman's game, Mars Generation One, is designed to teach argumentation, part of the Common Core academic standards adopted by 44 U.S. states since 2009. The Common Core was created, in part, by the National Governors Association to standardize grades K-12 learning and assessment, and address what the NGA perceived as a growing gap between what was expected of high school graduates by employers and what was actually being taught in schools.
Argumentation is a foundational skill for the English language arts standard.
"Everything is built on being able to put your thoughts together in a reasoned way, and then express them in different way," says Hoffman. "In ELA there's reading and then writing and speaking. All those are different ways of expression, but at the bottom of it is putting thoughts together. That's argumentation."
Mars Generation One aims to teach argumentation through Pokemon-style robot battles.
"[Mars Generation One is] a game that teaches argumentation, and especially the construction of a claim-data pair," says Hoffman. "This is the education-speak for what we're teaching. But what the player experiences is, you're a kid from Earth who's just arrived at the first school on Mars, where all the other kids in this school were born on Mars. They're part of the first generation to grow up on Mars.
"Here in this city on Mars, they have a convention where the way they make decisions in groups is based on reason. They build these arguebots, which are matched to different types of argumentation, so you have an Authoritron, which is the argument from authority, and a Consequencebot and an Examplebot, all the different types of robots. You can master their different forms of argument and equip them with evidence and battle them against each other. That's how you win arguments in this place. Because they make decisions based on arguments, it's also about conflict mediation and decision-making and social group peacekeeping and how to live in a society, especially a small community, as Mars would be, with high stakes.
"It is super nerdy, yes."
Meanwhile, back at 650 Townsend, the kids are settling in. Two boys and a girl. They look like average middle school kids. They know their way around a video game.
The play test goes like this: The kids sit in front of a laptop running Minecraft. Then they're left free to play a while, to get comfortable in the game and in the test environment. Each child is assigned a tester, who guides them through a series of tasks and asks simple but probing questions about why they're doing what they're doing. There are clipboards.
The experience is a little odd for the kids, but they get over it quickly. They're playing Minecraft, after all. It seems so far removed from traditional learning that they start having fun quickly. And when the lessons come at them, they hardly even blink.
The lead tester asks the kids to start building a small garden. He specifies the number of blocks wide it should be, by the number of blocks long. Then he asks for a certain number of items to be inserted into or removed from the garden. It is, in essence, a math problem, but one conducted in a video game, where the child's ability to add and subtract is immediately apparent, and the actual function of working the problem is being conducted in concrete terms, not with funky symbols or through a word problem.
This, says Hoffman, is the key to effective learning.
"Why would you gate a competency that's completely unrelated to reading with reading?" Hoffman says. "And yet that's the way we teach math. That's what a word problem is. It's a test of your reading comprehension. ... And so this is partly why word problems became so infamous in teaching. The purpose of a word problem is very good, but what it's trying to do, it's doing through written language, which is not the best tool for it."
Not only is a video game like Minecraft a better tool, in Hoffman's opinion, for teaching math, but it's an even better tool for assessment. The players' responses are immediate, and immediately observable. The GlassLab testers, with their clipboards, are able to see how each child responds to the math problem, evaluate what this says about their ability to grasp the problem, and redirect the learning immediately.
Not only is a video game like Minecraft a better tool, in Hoffman's opinion, for teaching math, but it's an even better tool for assessment.
Over and over throughout the test, the testers lean in to ask questions about what the children are seeing and doing, and the lesson is adjusted, incrementally, based on the immediate visual feedback provided by the game. This is what GlassLab is building toward. Not just using games to teach math, but using games to teach math (and other subjects) at a level that other products simply can't match, and for that teaching to automatically adapt to each child's efforts, and provide deep feedback to the designers so that they can make modifications on-the-fly.
"When kids do well on tests, what they're really doing is they're playing a game that they've learned the rules for while sitting in the classroom," says Seth Corrigan, one of GlassLab's learning designers.
What GlassLab aims to do is create a learning environment that provides assessment within the lesson, so that the separate step of testing — and all of the drawbacks that come with that, such as rote memorization and teaching to the test — is no longer required. The result, they hope, would be as if the bubble on a test form automatically informed the tester of not only what a child had learned, but how they'd learned it, how well they'd learned it, and could provide instant feedback to the child to help them learn more.
"Games are a nice environment ... in terms of the evidence we can gather," says Corrigan.
The kind of data that's generated when someone plays a video game is comparatively deep, compared to what testers learn from scoring a standardized test. In a testing situation, the results are binary: Either the student answered correctly, or they didn't. But in a video game, the telemetry designers can pull from the experience can tell them not only whether a player completed a challenge, but how long it took, how many times they tried, what they did that worked, and what they did that didn't.
"If you've never seen the telemetry streams that come off of some of these games, they're very rich. Think about the kind of information we get on you as a player and compare that to you as a bubble-test taker. The vector is very different. ... Imagine having access to maps or tables [during a test]. We want to know when you're accessing that information in order to start putting together a piece of how you're thinking about that world within the game. ... I get a better picture of what you're capable of doing as a learner than I would sitting you down in front of a bubble test."
Michael John is sprawled in a chair at a cubicle in the GlassLab space. He and Lily leave for a flight back to LA in just a few hours.
John has spent an entire day attempting to describe what Glasslab is and how it does what it does, and you get the sense, listening to him, that he's been having that same conversation, about learning and video games, for years. There's a tiredness in his voice due not only to the late hour and the long day traveling up and down the South San Francisco highways. But there's also a hopefulness, that maybe some of the final pieces are falling into place. That the intersections of his life's two great works — creating video games and raising Lily — are finally bearing fruit.
"Lily struggles at math," John says. He blurts it straight out, without preamble. As if this one story will explain everything. And in a way, it does. "A few weeks ago she was really cooked on some math homework — 'I can't do this, Dad. I'm not capable. I can't do it.' I say, 'Here's what we're going to do. You're going to take a break and go play Mario, the new one on Wii U.'
She says, 'Thank God.' She loves the Mario game. ... So she goes and plays Mario for a while. Beats a couple levels, gets some stars.
Then I say, 'That's enough Mario. Now, what just happened? What did you just do?'
She says, 'I played Mario.'
I say, 'Did you get through any levels?'
'Yeah, I got through two levels.'
'How were they?'
'They were pretty hard. One of them was one I couldn't beat before, but I got through it. I didn't use the leaf suit, Dad.'
I say, 'OK, how did that feel?'
She says, 'Well, I wasn't sure I could do it, but I kept trying at it, and I did it.'
I say, 'Well, let's go back and do your math. And I want you to think about how it felt when you were halfway through that Mario level you didn't think you could beat. Think about that, and let's go back and do your math.'
Then she was OK. It didn't change everything, but it changed her mindset enough that she could do it."
The exchange reminds John of making Spyro the Dragon for PlayStation. His team at Insomniac created a system for pulling telemetric data off of memory cards plugged into the PlayStation. They had to physically remove the cards and plug them into a separate machine, because the console had no internet connection. But when they did, the data they discovered was immensely helpful in designing the game.
After play test sessions, they could tell how long a child spent playing a specific level, or attempting to surmount a particular challenge in that level. They made heat maps for when specific players went to specific areas, then used that data to either increase or decrease the challenge level.
It was, in essence, assessment of the kind that John would later further develop with the GlassLab learning designers almost 20 years later.
"I remember accumulating this data," John says. "I still have some of these spreadsheets. I kept them. They're these big spreadsheets with the kids' names and their ages. I could develop a whole profile of Joey and Sammy and Lily, of what type of player of that game this kid was."
John's team then developed archetypes based on similarities between the ways certain kids played, and worked to make sure Spyro would be enjoyable for each type of player. Not to monetize, just to make it more fun. And in some cases to make the game respond on-the-fly to how kids were employing it, either increasing difficulty or reducing it based on how well the player was playing.
"It never occurred to me to tell the kids," John says. "Now I feel like an idiot. I could have just gone to Georgie ... I remember this kid. One of them, his name was Georgie. He was this really small kid, undersized for his age. Some of the more high-dexterity challenges, he just totally was not good at. But man, he would do it like 40 consecutive times, and on attempt 41 he would get through it. That's a kid who's going to do all right.
"I wish now that I could go out and tell his parents: 'You're going to look at Georgie and see a kid who fails a lot. Guess what? You're seeing the wrong way. You're seeing a kid who will succeed on attempt 41. And when he's a lot older, he'll be doing better than his peers because of that.' Dan, the kid who succeeded on attempt two, is going to think life's easy. Georgie knows it's hard, and he knows that the way to get through is to keep at it."
John stares off into the distance. He's lost in either his own exhaustion or in pondering the possibilities of a learning device that can not only judge and score, but also encourage. Perhaps it's both. But after a meal and a nap, the tiredness will probably fall away. The goal to create a better learning game will only get closer.