Welcome to Adversity & Co. Auckland’s 14th most creative agency. Today, Senior Creative Dan is pitching his ideas for the Easy Insurance account. Dan is desperate to use this TVC as a reason to travel to the most exotic overseas locations. But when the agency’s ECD, Head of Production and Business Director all point out that his ideas are all too expensive, too impossible and too shit, Dan is forced to consider the possibility of shooting in a place not overseas and not exotic at all.
In 2022 we teamed up with a group of Christchurch creatives to prove large virtual productions are possible. This is the story of the future of filmmaking, giving you total creative control to shoot any location imaginable in an immersive studio set.
It began in early 2020. The first wave of Covid was sweeping the globe and businesses were nervous. Our phone ran hot as people canceled shoots, then it stopped ringing, and for the first time in years we found ourselves with time on our hands. What better opportunity to embark on an ambitious virtual production on a scale nobody in New Zealand had attempted before?
The catalyst for this experiment was our first glimpse of content using Unreal Engine, a 3D game engine which responds to players in real time. Imagine this applied to a filmmaking scenario, where instead of reacting to a player, the environment generates in response to the movement of the camera. Unreal Engine had found a way to apply the power of their 3D engine to a filmmaking technique that can be used for virtual productions, projecting real time, responsive 3D worlds onto a set background of LED walls. When we saw the quality of content Unreal Engine could produce, our minds started racing.
Virtual productions require three key technologies:
1. An Unreal Engine environment that responds in real-time to input from cameras.
2. 3D camera tracking to connect the 3D environment to cameras’ movement.
3. An LED wall to act as the backdrop to the virtual set.
We knew Ōtautahi had all the pieces of the puzzle to bring a virtual production environment to life. Game developers Cerebral Fix have the largest crew of Unreal Engineers in New Zealand. While LED display company Pixel have the South Island’s biggest stash of vast LED screens. And the Resonate team have the production experience to pull the whole thing off. Cerebral Fix and Pixel were as excited about the vision as we were. All we needed was a location to shoot, and a wodge of funding to make it possible.
This production was about proof of concept. It was never about making money for us, or our two partners. Our goal was to bring a new technology to life in New Zealand and show that Christchurch had the talent and infrastructure to do it first. We went hunting for partners to bring our dream to life. Enter the University of Canterbury. Film production is a key strategic focus for them. The announcement of their new digital screen campus was planned to coincide with our virtual production shoot, as an exemplar of the innovation they plan to bring to the film industry in New Zealand. They funded the virtual production and provided us with a location for a studio. Screen CanterburyNZ also came to the party. This project was one of the first recipients of their new production grant.
In January 2022, we wrapped a week’s shoot where we filmed in five locations and never left the studio once. But it wasn’t all smooth sailing getting into the studio. We struggled to get camera tracking to work in such a large volume of space. We scrapped a solution we’d spent a year working on only 10 days before we began filming, and came up with something new. Meanwhile, Cerebral Fix artists built bespoke 3D art for five locations, no small task in itself. Pixel built the vast screen for the backdrop, 175m 2 and 270° of wrap around LED screen. This was our third LED set prototype and our biggest yet. The 3D engine renders the 3D locations onto the LED screens in real-time. The inner screen responds to the position and rotation of the camera, rendering the 3D scene in ultra-high quality. While the outer section of the screen helps light the scene, mimicking environmental light with ambient light reflected from the environment represented. Ian Sweeney from Vision Thing wrote and directed two scripts to showcase virtual production possibilities. The first is an ad agency spoof, gently mocking the tribulations agencies go through balancing tight budgets and champagne expectations. The second is a dystopian love story to show how a virtual production can accommodate commercial and artistic visions. It was a hugely technical shoot. Many of the cast and crew hadn’t worked together before, but the spirit of collaboration was exceptional with much goodwill and a great sense of shared camaraderie in creating something special. Come shoot day, everything came together. We walked into the studio, had total control from beginning to end, and wrapped each day knowing we’d nailed great footage. It was an eye-opener for everyone involved.
It’s not an exaggeration to state that virtual productions will entirely disrupt the film industry. With every location able to be reproduced as a responsive 3D environment, and real-world locations competing with imaginary 3D worlds, are we seeing the death of the location shoot?
Here are five reasons why virtual productions are the future of film.
1. Total creative freedom. Creatives can script scenarios they could never have dreamed of shooting due to budget limitations. In two days, we shot in Auckland, Fiji, Antarctica, and on the red desert of Mars.
2. Real immersive environments for actors, directors, and crew. For actors, virtual productions beat green screen hands down, giving them a real environment to engage with. For directors and crews everyone can see the creative vision they’re bringing to life. Unlike the guess work that goes into green screen productions, we can make decisions based on the environment right in front of us.
3. Total creative control. Unlike in a real location, you have total control over your setting. We were able to move trees and change lighting. We shifted Auckland’s Sky Tower with a click of the mouse to get a better view of it from the boardroom window. You can play God with your location in a way we could only dream of before.
4. Total production control. All the factors that make location shoots challenging: weather, available daylight, health and safety, ambient noise, permits, and access, become irrelevant. Instead, you’re shooting in a controlled, safe environment, with the amenities of a major city on your doorstep.
5. It helps you tell a better story. Pre-production becomes focused on elevating the creative vision, as you no longer need to spend time-solving problems and mitigating risks. The shoot process becomes more efficient, so you’ve time for more takes, different angles, and creative experimentation.
As the tech gets better and the cost of entry lower, virtual productions will democratize how we make advertising and film. It can only mean better storytelling for us all.
This tech is the next generation of virtual production, the evolution of greenscreen. Here, you have all the control of a studio shoot, but a far more immersive experience than greenscreen, which strips all the magic away, reducing a shoot to puppetry against a fabric background. This isn’t the first virtual production in the world. If you watched the Mandalorian you’ve seen what virtual production technology can achieve. Major disruption is on the way for the film industry, but the good news is that this experiment proves New Zealand can be leaders when it comes to virtual productions too.
We’re only starting to scratch the surface of what virtual production means for our industry, but of this, we’re sure. Within a couple of years, virtual production will play a role in every major shoot. Huge opportunities exist in virtual production for the New Zealand film industry, capturing our spectacular landscapes as 3D virtual locations, equipping and producing virtual shoots, or pioneering the new location-studio hybrid shoots that will become industry norm.
We’d love to see a permanent virtual production studio right here in Ōtautahi Christchurch. But the thing we’re most excited about is how this tech will help us tell better stories, giving us complete creative flexibility and control, and more time to explore storytelling. If you’re keen to explore what virtual production can bring to your advertising or film, we’re keen to talk with you.
