The artistry of game making explored at Tribeca Film Festival

The room is packed so tightly it's starting to get hot. People are murmuring excitedly — somewhere in the room is actress Ellen Page, star of movies like Inception and Juno, and she is about to discuss a project that is unlike anything else she has ever done. Then the lights dim and the screen flickers to life — and we're watching a video game.

For 35 minutes the audience sits transfixed, silent, through every clip of dialogue and every moment a button command prompt appears alongside the action. They are watching footage from developer Quantic Dream's video game, Beyond: Two Souls.

Earlier in the week, Tribeca hosted a panel featuring a developer from studio Naughty Dog, where panelists discussed evolving technology and cinematic techniques and how it affects film and game development. Topics ranged from the benefits and drawbacks of implementing new gadgetry to the importance of sweeping, cinematic scenes. Developer Josh Scherr discussed Naughty Dog's upcoming The Last of Us, and how cinematic techniques influenced its development.

The Tribeca Film Festival is an annual New York City event that aims to bring the film community and public together through critical discussions of the industry and showcasing upcoming works. This isn't the first time the Festival has hosted video games: Rockstar Games and Team Bondi's crime title L.A. Noire made an appearance at the festival two years ago.

In 2011, L.A. Noire became the first video game to be recognized by the Tribeca Film Festival. An hour of the game was screened to attendees, and was followed by a panel discussing the game's story and the technology the studio used to create the title. This year video game makers didn't simply draw parallels between the technical aspects of video games and movies, but also discussed the artistic elements the two mediums draw from one another.

Speaking on a panel about evolving cinematic techniques on Wednesday, Naughty Dog lead cinematics animator Josh Scherr said that video games are just as affected by new technologies — like the Oculus Rift and 3D — as film. However, there are additional constraints to working with these new technologies that don't always apply to film as well.

Scherr said when making Uncharted 3, which was rendered in 3D, everything had to be drawn twice, once for each eye, and tweaking every small detail to run at 30 FPS made the challenge that much harder.

"It's a technological constraint," he said. "And when we were trying to do rich elaborate environments and rendering in 3D, we had to dial things back a bit. Ultimately, while an interesting experiment, and 3D is very cool, we ended up not going with 3D for The Last of Us because we didn't want to sacrifice that graphic fidelity. As systems get more powerful, I think it will become less of an issue."

The panel's discussion turned to storytelling in cinema, and whether media with lower graphical quality can be overlooked if the narrative is strong. Scherr noted that the "cost of entry" in the video game industry is becoming lower, with small indie studios flourishing in the same market as blockbuster-budget projects like The Last of Us due to their stories.

"The Last of Us is costing tens of millions of dollars to make with a couple hundred people, and indies are still making successful, beautiful, interesting games with small teams that are selling quite well too," he said. "The way technology is coming along doesn't matter, so long as we are getting those new narrative experiences."

Striking a balance between gameplay and stopping the action for cinematic moments requires a delicate balance, according to Scherr. The more cinematic you make a scene, the more authorial control you exert over the player. The less players feel they have control over a game, the greater the risk you'll lose your audience, he said.

"One of the things that is a difficult balance to strike is, we try to make our games very cinematic because we want you to feel like you're having a cinematic experience, but at same time it's a game that has to be first and foremost about player agency," Scherr said. "At Naughty Dog we have our cinematic moments that aren't story-based and those where we progress the storyline, and we save [the latter] for instances in the story that are difficult to lock down as part of gameplay."

Making a heartfelt story as interactive as possible is another challenge, Scherr said, and it's a challenge to decide what parts of a game should be cinematic or playable.

"Lots of people argue games shoudn't be cinematic to begin with," he said. "Minecraft doesn't have to be cinematic. But for the story we're trying to tell, it's very difficult to make a rich emotional story be 100 percent interactive all the time, and the trick to that is for us to become cinematic in a subtle way."

For The Last of Us, playable character Joel is deliberately placed on the left side of screen and framed in an over-the-shoulder shot. Whenever certain things become important in the game — a change in the environment, lighting or sound — the camera shifts to call attention to where the player should look. According to Scherr, this helps players immerse themselves deeper into the world, pointing out the next step rather than dragging them to it in a cutscene, and allows them to invest in the story at their own pace.

After the panel, Scherr told Polygon that he feels action movies are borrowing more and more from video game cinematics.

"Games tend to focus on the visceral, 'oh shit oh shit oh shit' moments," he said. "So much is happening at once, and it's overwhelming to the player. One of the things we try to do at Naughty Dog and what we've done with The Last of Us is try to have some good modulation in that. You want those high explosive exciting points but you don't want them constantly.

"What I see in film is, people are starting to borrow that visual language of games."

"What I see in film is, people are starting to borrow that visual language of games," he added. "We borrow a lot of our framing techniques from filmmakers like Steven Spielberg and the Coen Brothers, and then you go to see the most recent Spiderman reboot and the part where it's a first-person point of view of someone running across buildings, swinging around everywhere, feels like Mirror's Edge. They borrow from each other."

For David Cage and his team at Quantic Dream, film and game development techniques were melded together for production on upcoming title Beyond: Two Souls. After the screening, a hushed room of viewers sat rapt, listening as Cage along with actors Ellen Page, Kadeem Hardison and Eric Winter spoke of their experience making Beyond as lifelike as possible.

Voice recording for games is typically done in an audio booth, with one actor at a time going down a list of their lines and recording them individually. They don't speak or act with the other actors, and everything is kept separate until it's all cut together. For Beyond, however, Cage's cast was suited up in motion capture gear and placed together in a 20-foot-by-20-foot stage surrounded by cameras, where they were free to interact with each other and perform as though they were on stage.

"By having them all do motion capture together, they could just focus on the script and their role," Cage said in a panel following a screening of Beyond: Two Souls footage. "They didn't have to look into cameras or stand on markers. They were completely free to move as they want.

"The problem with recording movement after you capture audio is, you lose that body language in the process," he added, explaining his reasoning for shooting motion capture and recording audio all at once — to allow the actors to immerse themselves and draw out a truly emotional experience. "When you shoot one actor at a time, they don't get to play together."

Following the screening, Hardison and Winter told Polygon that while acting for the game and acting for traditional film are very similar, the former allows more freedom.

"The big difference is the set and props," Hardison said. "On film they try to make everything as authentic and real as possible, so if you're holding a high-tech gadget in the film, you're getting a high tech gadget that works. For a game, you get a cardboard box with duct tape and maybe some chicken-scratch with a sharpie saying 'high tech gadget,' and you have to make it work. It's not so much about props, it's all about the emotion and locking in with the other actors.

"It takes you back to the roots of make-believe when you were a kid, jumping between cushions pretending the floor is lava," he added. "You have to go back to that."

Winter agreed that acting for games is a "completely different ballgame," not only in terms of the technical complexity of shooting in a motion capture sound stage, but in that you have to go back to the roots of make-believe.

"You're stripped bare with no props, wardrobe, no green screen, no nothing," he said. "It's acting in its purest form. You're a little kid in the backyard with your makeshift props playing cops and robbers. You're free in that space to do what you want to do, and the camera will find you instead of you having to find it."

"It's acting in its purest form."

In Beyond, players will get to choose dialogue answers and what actions they perform. These branching conversations and deeds were a challenge to capture for Cage's team, as every possible action and outcome had to be recorded. Actors had to seamlessly perform each scene with every outcome.

"They all had to [record every option] and still be convincing," Cage said, "whether the answer was yes, no or maybe. We made sure it was all consistent with characterization and the scene's moment. Every action was broken down to very small pieces, so players can play any part of them and make their own choices. It was quite technical."

Winter is especially impressed by the presence of games at this year's festival.

"It's mind blowing to know what we did, and then to see this crazy-ass production that looks nothing like that we shot on screen," he said. "We shot in a room with crates and boxes and now we have these crazy action sequences that players can control. It was surreal to see it on screen and have a film festival bring a game in and give it that kind of attention."

Correction: This article previously stated Uncharted 3 runs at 60 FPS and incorrectly defined how 3D was used. The game runs at 30 FPS and 3D requires all assets to be drawn twice, once for each eye. The story has been updated with these corrections.