Pixotope launches Pocket Mobile app to teach scarce virtual-production skills to students

Virtual-production platform Pixotope on Wednesday will release a mobile app that allows students to produce immersive video content using just their phones. The company is rolling out the virtual production app in concert with 14 film schools on multiple continents to help train a new generation of creators in transformative technologies that have been much talked about but still struggle to find enough trained crews and creators.

The Pixotope Pocket app represents a different approach by the company, said CEO Marcus Brodersen from the company’s Oslo, Norway headquarters, as the company tries to get accessible versions of virtual production into the hands of young creators who can use it to make all kinds of shows.

“What we make, it also requires high-end cameras and camera tracking and equipment that’s very expensive and difficult to get your hands on,” Brodersen said. “And typically, it’s just part of a lab, which will have limited time and access (to virtual-production technologies) for the students. But they actually do have really nice cameras right in their phones. And now with the new computer vision technology from Apple, Google and others, they also do (camera) tracking. It’s built for another purpose initially, but we realized we can leverage this to essentially give the students the missing piece of the puzzle to allow them to start just doing this on their own phone.”

The app relies on the augmented-reality capabilities built into the system software for AppleAAPL -1% iPhones and phones built on Alphabet’s Android mobile software to provide the data needed to synchronize virtual overlays and backdrops with the real-world video it captures. The resulting images can then be further enhanced and extended on a typical PC or Mac computer, using Pixotope’s software in concert with Epic Games’ widely used Unreal Engine graphics software.

It’s part of the Pixotope Education Program, a broader initiative to train more talent in virtual production.

“We believe what we call virtual production today is just going to be more and more necessary for everyone,” Brodersen said. But the lack of enough trained crews in the technologies has been “a blocker” for companies who might otherwise invest in the tools that Pixotope and other companies make.

Universities have been moving into the sector in many ways; the University of Southern California’s highly rated School of Cinematic Arts, for instance, announced about a year ago that it would transform one of its sound stages for virtual production, using a Sony-made LED wall and other needed technologies.

Crucially, however, Brodersen said the Pocket app is designed to be part of a more holistic approach to virtual production training in film school curricula. Instead of tacking on a virtual-production module at the end of a semester-long class, education about the technologies needs to be integrated into each part of the production process. And the app allows students to play with the technologies in their own phone and a personal computer.

“The nature of our production is that it touches all of these disciplines, everything from the production planning to the set design to the camera operator to the director of photography, lighting, grip, everything. But not everyone needs to know everything; they just need to know how virtual production affects their particular area of expertise. This is kind of the next step in that evolution.”

As part of this week’s announcement, the app will be integrated into the production program at University of Gloucestershire in the United Kingdom, which has been part of the education program for some time. Another 13 schools are expected to announce their involvement in coming days, Brodersen said. He declined to name the schools, so they could each make their own announcements, but said they include schools in the UK, United States, Europe, and especially India, where Bollywood and other production centers pump out hundreds of movies and TV shows every year.

Disney+ hit show The Mandalorian and Disney’s live-action version of The Jungle Book, both overseen by pioneering producer/director/writer Jon Favreau, set Hollywood tongues wagging about virtual production’s potential to reduce or eliminate the cost and time of location shooting with large crews, while also enabling new kinds of cinematic story-telling possibilities.

The Jungle Book won both an Oscar and a BAFTAFTA -0.4% in 2017 for best visual effects. The Mandalorian has won 14 Emmys since its 2019 debut, including two for visual effects and another for cinematography.

High-end projects like those have associated virtual production in its early years with prodigious investments in tools such as vast walls of high-resolution LED screens, VR headsets for directors and cinematographers, brightly lit body-capture cages of high-resolution cameras and the like. Numerous companies have invested heavily too, such as AmazonAMZN +0.7%, which last winter debuted what it called the largest LED production stage in the United States.

But Brodersen said the sector’s continued growth has been hampered by the lack of technical crew who know how to use the technologies, partly caused by those pricey tools.

“It’s a general problem in the industry, access to talent,” Brodersen said. “This will resolve itself over time, because a big part of this is Unreal Engine, which is almost ubiquitous in a lot of verticals. Telling stories with pictures is kind of still the same. It’s just that you’re adding this tool to it. And that changes your process a little bit, but at the end of the day, your editing is still editing and, and framing shots and composition is still the same thing. Most of the artistic part of it is really unchanged.”

The biggest changes may actually be on the front end, where filmmakers need better pre-visualization, more planning, a more nuanced and extended relationship with set and production design, and other changes in mindset and approach, Brodersen said.

“if you want to make it easy for yourself, you do need to plan even more, and just have a clearer idea about what you want to do,” Brodersen said.

The app will be provided free to students in film programs, but Brodersen said, “We do obviously see that this has the potential outside of that for prosumers and creators and streamers, but all the way up to professionals as well. We were at (the recent National Association of Broadcasters convention) showing it a little bit to some select visitors to the booth and everyone gets very excited because you can immediately start imagining use cases for this.” Forbes