Mariana Acuña Acosta, Co-Founder and Chief Product Officer at Glassbox reflects on the power of game engines and the opportunities for real-time, virtual visualization tools for virtual production.

VFX and production are on the verge of a seismic shift toward real-time, virtual visualization or virtual production, with inexpensive and efficient tools like game engines shifting pipelines and creative workflows both on set and off. Developers are just starting to uncover what will be possible with filmmaking when creators can visualize and collaborate virtually in real-time. Judging from talks this year at GDC and SIGGRAPH, this trend will continue to take off during the year as the possibilities and workflows become clearer.

Collaboration is critical at all stages of filmmaking, but content creators have always been plagued by various challenges: from restrictions with what you can see on the live action set, to slow wait times to view renders in post. The biggest thing we hear at Glassbox is that everyone, at all stages of production, wants to be able to more effectively collaborate.

Leveraging the power of game engines and the visualization capabilities of VR, we are working on solutions like our Beehive virtual directors toolkit, that addresses some of the roadblocks currently in place. Beehive combines real-time motion capture and real-time rendering technologies to let filmmakers see their entire virtual set, characters, and assets as though they were live action, thereby significantly reducing iteration times on set. Perhaps more importantly, it also allows for virtual collaboration, where multiple members of the crew can all experience this virtual set as a shared environment in VR. This lets the director work side by side with people like the DP, art director, or production designer within the VR space to adjust the overall set and elements such as lighting, materials, and surfaces in real-time as needed.

Thanks to VR, the team can still collaborate on the fly even from multiple locations – the crew can be shooting on a stage in Vancouver while the art department updates the virtual set from Los Angeles while the director is in an office in London.

This real-time, virtual visualization capability on set trickles down to the post process as well. As more iteration and adjustment can be done on set, the VFX team gets a head start in knowing what the final CG environments and other elements should look like. By incorporating a game engine into their VFX workflow, artists can also iterate and get to final frame more quickly.

Game engines are powerful and inexpensive, and as the building blocks for effective real-time visualization, they will do more to streamline and democratize content creation than any other tool. So, what barriers to adoption remain? Right now there is some misconception throughout the industry about how game engines actually work. Unreal Engine and Unity don’t just spit out photoreal, final frame renders – you still need the traditional tools in your arsenal such as Maya and Nuke. But game engines do provide significant time and cost savings on the way to final frame, while allowing for more informed creative decisions. Education throughout the industry will continue as the benefits become impossible to overlook.

Movies like The Jungle Book and short films like Neill Blomkamp’s Adam are some of this year’s notable examples of what can be created with real-time, virtual visualization. Furthermore, VR is now changing the way stories are told and consumed. Real-time visualization technology is essential for making interactive VR content possible, and as VR continues to take hold, this will also help to educate our industry on how real-time tools like game engines can fit into a studio’s workflow.

Ultimately, there is a bright future ahead for content creators as real-time visualization will transform how we work at all stages of filmmaking. I believe that real-time visualization tools will bring us into a new era defined by unrestricted creative freedom.