Virtual production is revolutionizing the M&E industry

Virtual production, massive LED walls, and the ability to track and render very quickly (real time in many cases), without a physical building full of hardware, is expanding the visual look of production.

Over the last 18 months, the boom in virtual production has shaken up the way filmmakers, broadcasters, mobile video designers and more conduct their creative business.

New technologies enabling virtual production include augmented reality (AR), virtual reality (VR), extended reality (XR) and mixed reality (MR). From sports coverage to the lecture hall, new options abound.

The virtual production market is poised to grow by USD 2447.47 million during 2023 to 2027, accelerating at a CAGR of 17 percent during the forecast period. The market is driven by the growing implementation of virtual production in gaming industry, the success of movies and

TV shows due to high use of animation, and the rising adoption of LED video wall technology.

The leading virtual production market vendors include 360Rize, Adobe Inc., Arashi Vision Co. Ltd., Autodesk Inc., BORIS FX Inc., Brompton Technology Ltd., Deloitte Touche Tohmatsu Ltd., Epic Games Inc., HTC Corp., Humaneyes Technologies Ltd., Mo Sys Engineering Ltd., NVIDIA Corp., Panocam3d.com, ROE Visual Co. Ltd., Side Effects Software Inc., Technicolor SA, The Walt Disney Co., Unity Software Inc., Vicon Motion Systems Ltd., and Weta Digital Ltd.

The year, in almost every hall at NAB 2023, one could see offerings related to virtual production, virtual sets and real-time performance capture mapped to real-time animation – all at a fraction of the price of the year earlier and the year before that. And while a physical matte painting may be a thing of the past, those images, landscapes and worlds (and the artists now working in digital) have no boundaries.

What was clear was the continual evolution of compute, networking, storage, visualization, and incorporation of data types necessary for virtual production resulting in creating believable and economically achievable content. Those technology factors, again, represent an evolution that has been playing out for decades.

Almost all of the hyperscalers were seen hosting their respective myriad of independent software vendors (ISVs) whose applications are running in some form of bare metal, cloud-ready, or cloud-native. Similar infrastructures that enable viewers to consume streaming video services over-the-top are being utilized to re-create and transform the live-to-cloud broadcast experience.

How it works. Virtual production is a seamless mix of live action and digital/virtual sets that allow to shoot multiple environments and variants at pace from within a single studio space on a giant LED screen (not green) that can essentially become anything you want.

It can turn a two-week, multiple-location shoot into a two-day studio shoot, while also decarbonizing the production process. And, every virtual set, background, and asset is reusable, again and again. And with a real-time 3D engine powering everything, all of these assets can be created virtually.

Scenes and shots with total precision, according to the exact creative vision of the director, can be planned out. And the flexible technology allows for on-the-fly iteration and experimentation without posing a significant risk to either the production or the budget, therefore, allowing complete creative freedom.

By combining LED camera tracking technology and real-time 3D render engines, creators can play back, and keep adjusting as they proceed, getting it right on the day – cutting costs and time.

Unforeseen elements, such as weather, unavailability of location, or a green-screen effect failing to work as intended can no longer hold the team to ransom. With no storage needs, a reduced need for travel to locations for shoots and the ability to re-use virtual sets and environments, the process remains within the financial budget. This is also for post-production, as there is no need to budget for post-production fixes because of color patterns caused by loose hair or green spill.

Virtual production is not the novelty that it was, say during Cannes Lions 2022 – not only is it becoming increasingly familiar for agencies and brands, but AI is also about to truly… well… turn up the volume.

Big brands are using it to transform their marketing, produce bigger ideas, reduce their carbon footprint and, yes, save money. Marketing teams are racing to block book facilities and holding companies are stepping up with their own volume studios, and it is up-ending traditional production and VFX pipelines.

That is not to say that virtual production is the be-all and end-all, and there will still be a need to escape the volume for certain kinds of projects that require cinematic scope or documentary storytelling. Moreover, in the film and TV space, virtual production may have pushed a certain kind of static and stilted filmmaking.

As for any technology, there are some cons too. The biggest struggle is the idea of changing workflow. It is a mindset and culture change that sometimes comes with resistance. While you can edit on the go and remove the need for fix it in post thinking, this takes a lot of planning on the front end. The process needs to start weeks/months earlier than it usually would.

The upfront costs are also a barrier to entry sometimes. Renting an LED wall (which is the key component for virtual production) can be expensive, and might not be great for smaller shoots. And there is no guarantee that one will find a wall to rent. The studios that have them are few and far between and often booked up well in advance. There is also a real dearth of talent. There is a limit to the number of people who are skilled up in virtual production.

For the creative teams, virtual production is a beautiful blessing, when used correctly and at the right time. It has moved on significantly since it first came on the scene in terms of hardware, software, and even new kinds of talent and job roles emerging, like AI director. Keeping up with this constantly evolving technology is no mean feat and not cheap but what is clear is that, although virtual production has reached a new stage in its development, we are still very much in the foothills