A big hand for Elijah Wood

David Lewandowski encountered personal as well as technical difficulties while making the music video for Flying Lotus' brilliant new track Tiny Tortures. Computer Arts caught up with him to find out more.

Computer Arts (CA): Talk us through the video, and where the idea came from?

David Lewandowski (DL): "We see a recent amputee and prescription drug addicted who is coping with survivor's guilt – wilted flowers in the room and bills scattered on his desk, someone who's clearly lost something. The idea is he had killed his girlfriend in a car crash, and was coping with that loss by tripping out all day on painkillers.

"We see that many things in his former life, all of his hobbies, involved things requiring two hands – video games, a rubiks cube, baseball. We watch him overdose and he begins to hallucinate that all of his electronics and objects around the room are forming a new arm so that he can experience his old life again.

"He takes too much and is crushed by the heavier objects in his room, and begins to warp through time and space where he regrows his real human arm. He experiences a brief moment of relief, where everything is euphoric, before coming back to reality, where we see none of these things really happened; he was tripping, undid his bandages and crammed a bunch of garbage and utensils into his wound."

CA: Our readers would be interested to know how the video was made – how you created the animation needed to build Elijah Wood's new arm, which programs were used, your workflow from start to finish?

DL: "Well, my team and I had some experience doing object capture from the last video we did together for Friendly Fires. We realised that an arm is really just two object tracks, with a joint solver to figure out things like elbow rotation. So we designed some straps that could be applied to the arm, and then tracked and matchmoved using only the plate-camera with very little rotomation needed.

"Because we shot in such a dark environment, it was important to upgrade the armature rig to one with dimmable strips of LEDs so that we could always have good visibility on them even in the shadows. We used home-made LED 'throwies' in the corners of the room and on walls for additional parallax information for the tracking software.

"Elijah had a green sock underneath the shirt sleeve on his bicep so we could key an edge of the shirt easily to put over the CG. In some shots, we used a prosthetic arm stump, and we digitally squeezed the pixels to remove where his real arm was concealed.

"We shot on RED Epic, edited an assembly in Final Cut Pro and later revised inside After Effects, 3D matchmoved and tracked in PFtrack, roto and painted cleanplates in After Effects with the help of the 2D tracking in Mocha, modeled our assets in Cinema 4D, also rigged and animated with C4D, fractured with Xplode, rendered with Vray for the arm shots and C4D's physical renderer and Turbulence4D on some shots, and composited the EXRs back linearly with After Effects and a variety of plugins.

"We worked mostly in EXR and prores linearly, and colored and graded the final prores using Divinci, and went out to web using a combination of Compressor and QT7pro. The 'stump' that was designed was scanned using an DSLR and a piece of russian software called Agisoft Photoscan, then retopoligized inside of 3dCoat, the UVs were fixed with body paint inside of C4D, then imported back into Photoscan to have the photographs auto-projected back onto the arm, to get nice high resolution texture maps for the new geometry."

CA: What, if any, difficulties did you encounter making the video, and how did you overcome them?

DL: "We originally were tracking with Syntheyes, which is another affordable package that had gotten us through the Friendly Fires video the previous year. My VFX supervisor and I used it constantly for months, and made everyone on the team learn it to track the 60 VFX shots on that project.

"With Flying Lotus, it turned out to be a bit of a tracking nightmare. Shots like that one with the airplane that breaks itself in midair were a total mess that lacked enough parallax information to get a good solve.

"After blowing three weeks tracking only a handful of the VFX shots and getting terrible solves, we had a meeting and made a huge leap to a different software package that none of us had used before. The entire team crammed tutorials and help documentation for two days and picked up PFMatchit and PFTrack from scratch. It completely saved us and made tracking a dream.

"The CG arm rig was quite heavy in the end, there was a tremendous amount of asset creation involved. We modelled and purchased many objects for the arm, but the wires were created by the technical supervisor from the ground up. It was a complete beast of a rig and I spent many nights banging my head against the desk because of how cumbersome it was to use at times. It was a brilliant piece of rigging, but slow and buggy.

"Ultimately, what made the rig manageable was using XREF – reference files – as well as a careful management of a feature inside C4D called the layer browser. It allowed me to work on or render specific parts of the arm without having to cut it up into multiple files or toggle too many buttons."

CA: And the overdose scene?

DL: "The overdose trip sequence presented many design challenges. I went through many revisions and versions, trying different colors, feelings, speeds. It took me weeks of lookdev to be happy with it, ultimately having a breakthrough reverting to what my DP and I shot using macro lenses of things like egg yolks, hand sanitizer, food coloring, milk and syrup as textures on 2D cards and blending them together in After Effects.

"Also I gained around 10 pounds from stress-eating during post-production, which certainly classifies as a difficulty. That I overcame with a combination of fasting, reckless crash dieting, and exercise..."