Fraser toldICG magazine, “The results on screen have a lot less moire, which is the trickiest part of working out the shooting of LED screens. If the screen is in very sharp focus, the moire can come through. That factored into my decision to shoot as large a format as possible, to ensure the lowest possible depth of field.”

To capture this background, they shot with the ARRI ALEXA LF. The reason was that Panavision built the Ultra Vista lenses that had a fast fall-off, allowing them to duck the moire issue and to shoot anamorphic. 

They used Panavision's prototype lenses for the 75mm 100mm. And they also used T2.5 50mm, 65mm, 75mm, 100mm, 135mm, 150mm, and 180mm lenses to cover every focal length. 

Fraser said, "Combining the LF sensor with the 1.65 squeeze on the Ultra Vistas, you get a native 2.37 [ aspect ratio]. Those lenses have a handmade feel in addition to being large format, and a great sense of character. We didn’t use any diffusion filtration at all.” 

But even with all this testing, they still needed StageCraft to be greenlit. 

Credit: Francois Duhamel, SMPSP

Greenlighting StageCraft 

There was a lot of concern around the project that it was going to be expensive and look cheap. We know now that that was not the case, but in a universe determined to mix the practical and digital, it felt like everything was at stake. 

Lucky for them, and us, ILM greenlit the project after seeing it in action. 

But that was not the hardest battle. 

That came in mixing tech from a ton of different outside sources for one internal strategy.

ILM creative director Matt Madden, who served as The Mandalorian’s virtual production supervisor, says “UE machines handled all content rendering on the LED walls, including real-time lighting and effects, and projection mapping to orient the rendered content on each LED panel, deforming content to match Alexa’s perspective. For each take, the StageCraft operator was responsible for recording the slate and associated metadata that would be leveraged by ILM in their postproduction pipeline. ILM developed a low-res scanning tool for live set integration within StageCraft, taking advantage of Profile’s capture system to calculate the 3D location of a special iPhone rig, recording the live positions of a specific point on the rig to generate 3D points on live set pieces. The Razor Crest set, a partial set build, used this approach, with the rest of the ship a virtual extension into the LED wall. Controls on an iPad triggered background animation, giving the impression the physical Razor Crest set was moving. Once imagery had been rendered for each section of LED wall, the content was streamed to the Lux Machina team, which QC’d both the live video stream and the re-mapping of content onto walls, [which] Fuse Technical Group operated and maintained.”  

You want to know something nuts? 

They were able to control all of this remotely, using just an iPad. 

The team that controlled all of this was called the "Brain Bar."

To pull off the rest of the shoot, they came up with a new way to do business on set. 

Let's got through some steps: 

• The Brain Bar loads a 3D environment matching the physical set build into Unreal. 
• They use an iPad containing a UE4 interface that allowed us to remotely update the LED wall imagery
• Go over the look with the DP
• Dial in the position and orientation of the virtual world for the first camera setup
• Adjust lighting based on notes from the DP
• Make final adjustments to the 3D scene itself based on input from the VFX super (such as color-correcting the dirt on the virtual ground to match the physical set) 

Sheesh. 

Credit: Francois Duhamel, SMPSP

StageCraft in Action 

Series Director of Photography Barry “Baz” Idoine was used to challenging shoots, but nothing could prepare him forThe Mandalorian. Mostly because everything they were doing was new. 

He told ICG, “Seeing this 3D rear-projection of a dynamic real-time photoreal background through the viewfinder is tremendously empowering. It’s phenomenal because it gives so much power back to the cinematographer on set, as opposed to shooting in a green screen environment where things can get changed drastically in post.” 

They mess up, make changes, and do reshoots, but they're all in this together. And the team embraces that. 

So many teams have to come together to get this kind of tech to work on screen, but everyone involved understands that they're doing something new, cool, and important. 

They're proud. 

Fraser resoundingly supports them working in uncharted territory. “Technology serves us; we don’t serve it. So, it doesn’t make sense, to me, to embrace something just because it is new. But if it can help us do things as well as if we were doing it for real, but more economically, that makes good sense. Every day [on The Mandalorian] we were making decisions on how to go forward with this process, so it was like history evolving as we worked. Each of the various departments works independently, but at the end of the day, they had to stand together.”  

Credit: Disney+

Where is StageCraft Going? 

As Unreal Engine gets better and better, there's no telling where they can take the tech. As far asThe Mandaloriangoes, the plan is to try it on more character works. Right now the engine isn't great at doing masses of people. They'd like that to change. 

They also think the tech can be hard to change when the story changes. Right now, visuals get approved months in advance, so if the story changes or ideas change, that can be a big setback. 

Still, every day the tech gets better and more powerful. 

Season one challenges won't be the same in season two or beyond. 

Count me in as one of the most excited to see how this is used in Star Wars and across all Disney's franchises. 

It's an exciting time to be a movie and TV lover. 

What's next?Is the Mandalorian a Western or Samurai TV Show? 

The Mandalorian is all the rage on Disney+, but there's a growing debate over whether or not it's a western or samurai show. So which is it?

Click to find out. 

Learn how to turn your score into a character.