Researchers bring their steampunk-inspired virtual reality lab to the Sundance Film Festival, looking for clues about how to make VR moviemaking more accessible to everyone.
A great movie on the silver screen can make audiences suspend reality. At this year’s Sundance Film Festival, however, the latest visual storytelling trend pulled audiences into plots and put them in control of the action using virtual reality (VR) filmmaking tools.
Many festival-goers strapped on VR headsets to dive into more than 30 VR experiences at Sundance this year. Some even stepped into a bird bodysuit that made them feel like they were flying above San Francisco.
The festival even released its own VR app so those who couldn’t attend the event could immerse themselves in new 360-degree movies.
All of this led entertainment industry pub Daily Variety to dub this year’s Sundance the coming-out party for VR.
“A year ago, VR still seemed like a futuristic technology inaccessible to most consumers,” wrote Janko Roettgers of Daily Variety. “That changed this time around, with Samsung releasing its Gear VR mobile virtual reality headset last fall and Oculus getting ready to release its Rift headset in two months.”
When Sundance, the largest independent film festival in the U.S., launched its New Frontier program 10 years ago, it focused attention on the intersection of film and information technologies. Since then, that focal lens has widened beyond personal computers and smartphones — devices that make and play a movies — to a new world of technology that engrosses audiences with emotion-stirring VR.
One of those immersive experiences at the festival’s New Frontier Lounge was the Leviathan Project. Based on Scott Westerfeld’s WWI steampunk novel series, Leviathan gave the festival crowd a sense of what it’s like to control a giant flying whale.
These VR tech-equipped viewers felt as if they were part of a world where genetically modified creatures and antiquated objects could be controlled with a turn of the head or hands.
— Andrea Mandell (@AndreaMandell) January 23, 2016
“Through my headset, I became a 19th-century scientist aboard a flying whale, bio-engineering a crazy creature of my choice,” wrote Andrea Mandell in USA Today. “It was fun but made me a bit queasy (in fairness, 3D gets to me, too).”
VR movies debuted at Sundance last year, but it was evident to Tawny Schlieski that the VR filmmaking is still nascent. Schlieski, a research scientist for Intel who studies VR experiences, was at the festival showing new tools like Intel RealSense camera technology. This technology gives depth perception to computing devices and can help filmmakers create VR experiences.
Schlieski said that filmmakers face challenges because there isn’t a set of standards — for gear, editing software and VR media players.
“You can create a system to make VR content, but it requires some hacking,” said Schlieski. “Right now, creators are taking existing tools and clumping them into these new forms.”
This DYI approach to 360-degree video making is evident in film schools and on the Internet, where creators share open source designs that help people 3D print their own synchronized spherical camera rig.
Schlieski believes that this makeshift approach to VR filmmaking will soon evolve as new equipment and digital tools are made specifically for VR creation. To speed this transition, her research team is studying how visual storytellers are adapting existing technologies to make VR experiences.
“We want to discover and articulate a content creation set of VR tools that creators can have so they don’t have to hack something together,” said Schlieski (more in this audio podcast interview).
“The core purpose of my research is to understand what that content stack of tools would be and how Intel technologies can contribute to it.”
Schlieski’s work with digital storytellers could lead to technological efficiencies that lower today’s barriers, making it easier and less expensive for professional and amateur VR creators. The Leviathan Project is a test bed for this research.
Some VR aficionados might remember Leviathan from the 2014 Consumer Electronics Show, when the flying whale soared above enormous crowds. The project is led by USC professor Alex McDowell, director of the World Building Media Lab.
The Leviathan trilogy of novels comes to life through an immersive mix of real and augmented reality, along with haptic feedback and motion tracking. The experience makes users feel like they are actively taking part in a fantasy world where they can ride on a whale-like flying warship.
“In the new iteration of The Leviathan Project, we are extending the relationship between users and the narrative space by making them participants in the outcome,” said McDowell. “No longer are the stories fixed. Now they are changed by the action of each participant,”
Many digital artists, especially the independent filmmakers, turn to their PC to sketch ideas, edit and share their videos on internet sites like YouTube or Vimeo.
Filmmakers at Sundance got to see how an Intel RealSense camera-equipped tablet could be used by viewers to follow and interact with characters in a story. The tablet allowed viewers to interact with the Leviathan whale in a virtual world that was overlaid on the physical world.
“You’re able to play with the narrative of the story in this world,” said Leviathan art director Bryan Zhang, describing the experience at Sundance. “In the book you’re confined to the story, but here you can go different places and have different interactions.”
One of the project’s lofty goals is to progress its virtual world so that it’s self-sustaining.
“As a research project, we’d like to develop it to be intelligent, to populate the whale and gondola with all the people who live on it,” he said. “And you’d be able to see it with augmented reality or a VR headset and jump in to experience a number of different kinds of stories.”
While there’s a lot of potential to bring a wide variety of tales to life, the researchers emphasize that what they’ve created is very much an experiment.
“Leviathan is a living lab,” said Schlieski. “We’re exposing filmmakers, creators and students — the future of media making — to virtual reality. We’re learning from them, so we can build up understanding about how people make and interact with stories in a VR environment.”