Behind the VFX of Finch and its Adorable Android, Jeff

Using a robot as a featured character in a movie is always a bit of a gamble, particularly if you want the audience to form a sincere emotional connection with your nonhuman android. Nevertheless, director Miguel Sapochnik went all-in on his sci-fi drama Finch by making a robot one of the film’s three main characters, right alongside an adorable dog and two-time Oscar winner Tom Hanks.

Set in the year 2030, Finch follows titular engineer and inventor Finch Weinberg, one of the few survivors of an apocalyptic solar flare that turned most of the Earth into an uninhabitable desert blasted by deadly UV radiation. Dying from radiation exposure and forced to abandon his laboratory, Finch sets out on a cross-country journey with his dog, Goodyear, and the android he created to care for his canine pal.

While Hanks delivers a powerful performance as a scientist confronting his own mortality, it’s the android named Jeff that provides much of the heart and humor in Finch, thanks to a brilliantly nuanced motion-capture performance by actor Caleb Landry Jones and a blend of practical elements and visual effects. Pro Well Tech spoke to the film’s Oscar-winning visual effects supervisor, Scott Stokdyk (Spider-Man 2, Hollow Man), to learn how the film turned Jeff the robot into such a remarkably human character.

Pro Well Tech: It’s always difficult to tell where the performance-capture work ends and the visual effects begin with a character like Jeff, so how much of Caleb Landry Jones’ performance do we see on the screen?

Scott Stokdyk: That’s a complicated question to answer. I look at all the shots in the film and feel Caleb in every shot, because I saw him for so long in his displacement suit. That’s what we call it, by the way: It’s a lycra motion-capture suit. Legacy Effects built pieces that were Velcroed on to it in order to give him the bulk and physicality and lighting reference, and they also built a real robot that was able to be puppeteered. It was part animatronic, part puppeteered, and that was the starting point for us.

When I came on to the film, I was very into the notion of artificial intelligence and machine learning and what that would mean in the year 2030. Artificial intelligence is all about accumulating massive amounts of data. So Jeff is always watching Finch. That’s where he’s getting his data from. There’s nobody else around, after all.

And that’s why we see so many scenes of Finch telling Jeff to imitate and copy him …

Exactly. He’s very mechanical at the start, and we had to show his growth. We had that discussion with Caleb early on and we were like, “During the course of this movie, you’ll start out with a very restrictive range of motions, and as you watch Finch, you’ll learn more and more, and your motions will get more fluid.” There’s this notion of style-matching with artificial intelligence that we felt was important to show here.

So we said to Caleb, “Basically, if you’ve seen Finch do something, you can imitate it. If you want to scratch your head, you have to sort of do it in the style of Tom Hanks acting as Finch — not just Tom Hanks in any movie, but as Finch.” We actually motion-captured Tom Hanks, too.

You mapped out Tom Hanks’ performance, too?

Yeah, we initially felt that at the end of the movie, we would start from Caleb’s interpretation and the physical robot, and end up with what was essentially Tom Hanks. But we went away from that as we saw Caleb do his thing. We tried to shoot in story order as much as possible to make it easier to wrap our heads around that evolution, but when we saw Caleb perform, we were like, “OK, this guy is embodying this role. He was born to do this role.”

He was such a great physical actor in the role. We had this crazy costume on him and sometimes he’d have the mask on and couldn’t perform with his face, so he had to be more physical. He ended up doing a lot with his hands, particularly with his fingers. I don’t know if you noticed that, but …

Wait, that’s where you got Jeff’s finger motions?

Yes! That came from Caleb. That was purely him. I loved it so much and was like, “OK, we’re going to use his real hands.” Caleb had on these gloves that were designed to be gentle for the dog, so they’d connect with something soft, not a metal robot hand. And as I saw him doing this stuff with his hands, I said, “OK, we’re going to paint out everything except for the gloves he’s wearing, and then we’re going to track the gloves and make the rest of him CG.”

Caleb had such subtle interactions — the way he touches Finch and the dog, for example — and we wanted to keep that connection. It made our lives harder to do it that way, but I think it paid off. I feel like it added some warmth and realism to everything we were doing.

Along those lines, what was the evolution like for Jeff’s overall visual design?

I came into the project after the production designer and the director, Miguel, had already gone down a lot of that path. They had done research at University of Oregon, and the idea that was translated throughout the whole movie is that this robot was built for the dog. So what is the dog going to respond to? He’s going to respond to touch and to voice. He’s not going to read lips. He’s going to see the physicality of the silhouette.

So Finch, as a designer, is thinking about the dog. Even the fact that Jeff is tall means that if he goes away from the dog, the dog can see him from afar. For his feet, Jeff has something like Crocs. Finch wears Crocs, so it’s like the smell of the gloves and the Crocs are all part of his design. Miguel is an extremely thoughtful director. He doesn’t just jump into things. He thinks it through. I feel like that paid off, because it is unique, and it’s a thoughtful design.

I was really amazed by how much expression and emotion we got from a character with minimal facial features. How did you find that balance in getting emotive elements out of Jeff while retaining his not-human look?

I think it’s a little bit of a magic trick in any movie, whether it’s a real actor or a CG character, to make the audience care about them. I credit a lot of that to Miguel’s direction and Caleb’s acting. But we worked hard to grow him into something that was very human and less mechanical, too. We wanted you to believe he was really there, first, and also believe he was evolving. Once you force yourself to not rely on the usual emotive sources and do it from the rest of the body, you have to work hard at it, and that connection is earned. But it’s not earned in just one shot. It’s over the course of a lot of shots and a lot of care.

It’s tempting to be like, “Oh, we’ll just use motion capture to do it,” but here’s the thing: It’s not always a one-to-one translation. Caleb gave an emotional performance, and we tried to keep what we could, but the proportions weren’t always the same. We tried to keep his hands and his head positions and other things that read strongly and emotionally to the audience, and everything between that was a lot of choices we had to make. How fast a movement should happen, for example, or whether his silhouette could read differently with just a couple of little shoulder motions or tweaks.

It’s interesting you mention shoulder movements, because I noticed Jeff starting to “breathe” as the film went on, his chest subtly rising and falling …

I’m so glad you noticed the breathing, because in the campfire scene, we made the decision to start showing a tiny bit of breathing with Jeff. A robot wouldn’t typically do that, but he’s mimicking and becoming a little humanlike — or affecting being human, in this case.

Was there a particular scene that was more challenging than others with Jeff?

At the Ship Rock location near the end, when they park the RV and they’re having the picnic, that was the trickiest part to put our finger on, because what would Jeff be at that point? Is he Tom Hanks now? And then we realized, “No, he’s actually made the choice to become something else.” That’s reflected in the dialogue of the movie, in terms of him pushing back a little bit on Finch in some conversations they have. That’s where Jeff kind of splits off into a different entity.

So when he’s breathing, it’s because he likes some aspect of that action. It’s partly for the dog, and it’s partly because of Finch, but it’s really his own hybrid. There’s also the parka he finds and wears. At the end of the movie, he embraces that as part of his identity. He first picked it up because Finch had one, and then he put it away for a while, and in the end it comes back — not because he wants to be Finch, but because he has this new identity pulled from different pieces of his experiences.

So those emotional scenes with Tom Hanks were tricky because we had to know where he was at that point in his evolution, and also because he’s acting against Tom Hanks. That’s every actor’s dream and also a nightmare, because how do you act against a legend like him? For us, we had to present a performance based on Caleb, but one that also holds up against Tom Hanks in those moments.

No pressure or anything …

Right? So I think we can be forgiven earlier for having Jeff be more robotic and not really connect, but in that late scene, you absolutely need to connect with him.

Outside of Jeff, what were some of the other elements you were proud of in the film?

Well, I want to give credit to Legacy Effects, who built the physical version of Jeff that we could all pull on and push on and work with. And there was also that little robot, Dewey, which was a real, working robot. You can go online and find working robots, but Dewey had to actually have a personality, too. He had a cruder A.I. that was basically five or six puppeteers, so he’s as smart as five or six of us. In comparison, Jeff is as smart as 60 or 70 of us by the end of the movie. I’m really proud of the work they did.

We also had to be very judicious and clever about how to make use of our resources. This is basically a drama where we’re allowed to have this great CG performance, so it’s not a giant Marvel movie. We had to consider where we would put our money. Shooting on location in this beautiful New Mexico desert definitely helped us. We didn’t shoot with a blue screen if we didn’t absolutely have to, and we took that RV on the road like it was a road trip movie. We tried to treat this as a great buddy road trip movie, and I hope it felt like that.

Directed by Miguel Sapochnik, Finch is available now on Apple TV+ streaming service.