How Dune's visual effects made an unfilmable epic possible

Frank Herbert’s sci-fi epic Dune was long considered an impossible adaptation, too weird to bring to the screen in a truly faithful way, and too beloved by its fans for any filmmaker (even David Lynch) to change.

And then along came celebrated Arrival director Denis Villeneuve, who has a knack for handling complex themes and storytelling elements within the sci-fi genre, as well as one critically acclaimed “unfilmable” project, Blade Runner 2049, already on his resume. His adaptation of Dune — the first chapter of a two-part story — not only won over critics, but it managed to perform well at the box office amid a theater-throttling pandemic.

Among the team Villeneuve assembled for Dune was two-time Oscar winner Paul Lambert, his visual effects supervisor on Blade Runner 2049, who took home an Academy Award for his work on that film and then repeated that feat the following year for the Neil Armstrong biopic First Man. Lambert’s talent for grounding sci-fi spectacle in relatable elements of the real world was put to the test in Dune, and he spoke to Pro Well Tech about bringing a project many believed impossible to the screen.

A sandworm consumes a spice mining craft in a scene from Dune.

Pro Well Tech: Dune is such an epic project that’s been considered such a difficult one for so long. Was the challenge part of the appeal to you?

Paul Lambert: I love the challenge. But having worked with Denis before and knowing his approach was going to be very photoreal and realistic, I knew what that challenge was going to be. I like to try to make things as invisible as possible so that nothing takes you out of the movie. Even with building these massive worlds, with all these spaceships and things, the goal is that you actually believe this could actually happen — basically, trying to keep it as grounded and as realistic as possible.

What were some of the early meetings like when you were hashing out the film’s look and how you’d achieve some of the big, VFX-fueled elements?

Denis and Patrice [Vermette, production designer] had spent most of a year developing all of these visuals and all of the concepts. They had designs for the worms and for Arrakeen [the fortress on the desert planet Arrakis], and we basically built physical sets which matched these images, as well as virtual worlds and ornithopters and everything else based on those designs. We had a really good grasp on what everything was going to look like, so it allowed us to come up with different techniques on the set to help produce the best visuals we could. We were never in a position where we shot something and had to add something else in the background and were like, “We’ll just fix it in postproduction.” We always knew there was going to be a very specific structure or element behind the actors at all points, so we could make decisions based on that certainty.

An early shot of the desert landscape used for a scene in Dune.

Characters from Dune look out over the landscape of Arrakis.

How did that plan take form while you were filming?

Well, for example, we used sand-colored screen rather than green or blue screen in a lot of shots. For everything on Arrakis, we used the sand color, because we knew that behind the characters, it was always going to be Arrakeen or it was going to be the desert. … So, for shots inside the ornithopters, you’d traditionally film in a studio surrounded by green or blue screen and replace everything outside the windows. But for this, we found the highest hill outside of Budapest and put our ornithopter on a gimbal on top of that hill, surrounded by a sand-colored wrap. On a sunny day, the light would bounce off the sand into the cabin, so when you look at the footage and the focus is on Paul (Timothée Chalamet), it feels — visually — like you’re flying high over the desert, because you have all this light and sandy brown environment, and over that, just blue sky.

We also shot hours of footage out in the United Arab Emirates, flying over dunes with an array camera, which is basically six cameras attached to the front of a helicopter. Once we had all this high-resolution imagery, the compositors could take that imagery and blend it with the Budapest footage, producing really natural-looking shots. It was a very different way of working — instead of filming, then extracting the foreground and adding the other elements later, it was a blend that gave us a far more believable shot. The movie was peppered with all kinds of different techniques to help with the visual effects in postproduction.

A helicopter flying over sand dunes while filming Dune.

Winged ornithopters flying over sand dunes in Dune.

Working with so much sand seems like it would get really complicated, getting into everything both metaphorically and literally. Was that the case?

Oh, for sure. Gerd [Nefzer], who was our special effects supervisor, told me recently that he used around 18 tons of sand and dust. We built practical, full-size ornithopters [without wings] and shipped them out to Jordan. The idea was to use them for certain shots and add digital wings to them later. We picked them up by cranes and blew dust everywhere around them [to simulate] the wings flapping. There were days when crew who had spent a little too much time in front of these fans would end up looking orange, and it would take a couple of showers to get rid of it.

In one particular setup with Paul and Jessica (Rebecca Ferguson) flying in the two-seat ornithopter through a sandstorm, we set up a gimbal, put the ornithopter on it, and enclosed it in a black box. We filled the box with dust using fans, so it was swirling around them. Sometimes people had to go in there, and one of the funniest images I remember was our first assistant director, Chris [Carreras], coming out entirely orange. That stuff got absolutely everywhere.

Characters from Dune look down at the sand in an early VFX scene from Dune.

A black-and-white VFX shot from Dune with a ship in the foreground and worm in the background.

Sharacters from Dune look at the sand as a worm swallows a vehicle.

Did all of that sand complicate things digitally?

Absolutely. I’m a huge believer that when you’re shooting visual effects, rather than trying to create them in different layers — like shooting whatever the foreground is going to be, then shooting the smoke and such you want in the shot separately — I like to get everything in one pass. It does make it harder when you’re trying to change things in the background later on, but having come from the compositing side of the process myself, I know you’ll have a much more believable shot in the end that way. That’s because you have all of these elements authentically interacting with each other in the shot. So even though we used 18 tons of of dust, we also added hours and hours’ worth of digital dust to the shots, too, to help extend the effect. But having that base of real interaction with all of the elements is the key to a believable visual effect.

The body shields are such an interesting part of the story that have been interpreted in some interesting ways over the years. What was their evolution like for this film?

It was very much a trial-and-error process. I had brought two artists with me during preproduction because I wanted to do immediate tests for different ideas. We actually landed pretty quickly on a look for the shields. We started with a clip from Seven Samurai — specifically, the famous fight scene — and one of my artists processed an image from the film to blend past and future frames together. It created this shimmery look around the characters, with people moving all the time, and we played with that idea a bit. I then had the artist go in and paint back the original frames a bit or paint in some of the effect, because I wanted it to feel more analog. We showed Denis and he absolutely loved it.

Timothée Chalamet and Josh Brolin fight with shimmering body shields in a scene from Dune.

That’s always a good sign.

Right? So we got there pretty early with the shields, but once we got to the edit afterwards and were processing the footage of Paul and Gurney (Josh Brolin) fighting, we found that there were times when you couldn’t quite figure out what was going on because the action was so intense. So that’s where the idea for the blue and red colors came from. When the shield protected someone, it would be blue, and when something was slow enough to penetrate, it turns red. That’s when we added the sound effect as well. So suddenly, we had a visual the audience could understand.

Is there a visual effect you’re particularly proud of from the film?

One sequence when we came up with a great idea was when Paul hides within a hologram. For that one, I knew he was going to be embedded in this hologram and I wanted to avoid doing a digital close-up of him. That takes a while and can be very, very expensive — especially to get that beautiful, interactive light on people’s faces in CG. So how can we actually get a practical, interactive pass on Paul on the set that we can use?

What we came up with, which actually worked out really well, was to get the holographic bush he hides in approved by Denis early on, and then slice it up hundreds of times. We then got an old-fashioned projector and projected each slice on him, based on where he was on the set. So as Paul moved, you would get a different slice being projected on him and around him. And as he moved forward, you would get one slice after another, as if he was moving through the branches.

And because this shot wasn’t left to postproduction, Timothée could position himself in the best ways to interact with the light. Then we just had to put the bush behind Paul and in front of him later on. But because we had the ideal interaction with the light, it felt believable. It felt as if Paul really was inside this massive hologram. I’ve heard from various people that is actually their favorite scene, and it’s always good to hear.

Timothée Chalamet hides inside a hologram in a scene from Dune.

What went into creating this film’s version of the sandworms?

We had designs as to what these creatures would look like early on, but obviously, it didn’t move, so that’s what we had to figure out. We had a fantastic animation department at DNEG. They spent a long time trying to find reference as to how a worm or snake moves, but what we found over time was that as things became more scientific and biological, it wasn’t very cinematic. It didn’t look good on the screen. That’s when we transitioned to the idea of a whale going through water.

So as the worms move and the dunes splatter, it feels like the waves of sand are going up and down, like ripples of water. That’s what we ended up playing on: The idea that these worms are moving like they’re in the ocean on this hot, arid planet. Even the design of the worm’s mouth is based on the baleen of a whale, as if it’s sifting through the sand for something like krill as it goes. These worms travel through these deserts like whales in the ocean, sifting through the sand to get all the nutrients to produce the spice.

The mouth of a sandworm in a VFX shot from Dune.

The mouth of a sandworm in a finished VFX shot from Dune.

And that brings us back to digitally manipulating sand again …

It does. One aspect of the worms I was nervous about early on was coming up with how to displace so much sand when they surface. In the computer, it’s quite tricky to do, because it’s super complex, and computationally expensive to figure out how one grain of sand connected to another grain and so on will act on a massive scale. The key to a good visual effect is always having reference, though. At one point, I suggested to our special effects supervisor that we set off some explosions in the dunes for reference, but I was reminded that we were in the Middle East and it probably wasn’t the best thing to do. So, needless to say, I didn’t get that footage.

We started doing iteration after iteration to figure out how to make this colossal amount of sand feel real. You can’t do a grain-by-grain simulation, so you have to cut corners a bit to manage it. But that risks losing the physicality of it. It will be too fast, or the scale won’t be quite right, or it looks beautiful, but the worm is too small. So it’s an iterative process, and it takes a long time to actually simulate all of these particles. It took nearly a year to get to a place where Denis was turning over shots and we were able to process them quickly.

A monstrous sand worm rises from the desert in an early VFX shot from Dune.

Striking a balance between what’s done in-camera and visual effects is a recurring theme here. Was it difficult to walk that line on a film like this, which really leans into fantastic elements?

The philosophy of the shoot was always to try and get as much in-camera as possible, obviously. But this is a sci-fi movie, so there are always going to be things which are all CG. And we do have shots which are all CG. For example, some of the battle where the two armies come together are entirely CG. When Duncan (Jason Momoa) is flying through the city, chased by the Harkonnens, that’s all CG. But we always try to ground that with shots both before and after that use practical elements, like the practical ornithopter. There’s always a mix and match. But if we’ve done our job well, people don’t know which one is which.

Given all of the build-up to production and all of the buzz leading up to the premiere, is it a relief to finally have Dune out there and be well-received?

Yeah, it’s great that people really enjoyed it. It was a fantastic collaborative approach with all the heads of departments, and one of those rare occasions where everything just works. It was a fantastic experience.

Denis Villeneuve’s Dune remains in limited release theatrically and is available for on-demand streaming.