Plenty of summer blockbusters and event television shows are of the effects-driven variety, filled with incredible computer-generated characters that still retain a sense of photorealism.
But in recent times, more and more films and television series are featuring effects of a different kind – invisible ones. They are the effects you might not always notice, although sometimes they involve just as much effort to ensure they remain unseen. Invisible effects can be used to place characters on a fast-moving train, to seamlessly replace the facades of old famous buildings, or even to make a Hollywood star skate on ice.
In fact, these were just some of the tasks given to studios Cinesite, Eight VFX, Union VFX, Rodeo FX and Framestore as they delivered invisible effects work for The Commuter; I, Tonya; Three Billboards Outside Ebbing, Missouri and more. Here we explore how a whole wave of impressive seamless effects made it into those films.
01. Piecing together a storm in Adrift
Baltasar Kormákur’s Adrift, a true story of two lovers who suffer a major catastrophe on board a yacht in the open sea, required both stormy ocean shots and other out-at-sea composites.
Production VFX supervisor Dadi Einarsson tapped Milk VFX as one of the studios to handle visual effects for the film. Some of Milk’s shots are likely to go by completely unnoticed by the audience, while others that relate to the larger storm sequences and its aftermath were highly complicated, lengthy frame challenges for the studio.
“The most complex VFX sequences both technically and creatively were the storm shots,” outlines Milk visual effects supervisor Sara Bennett, “which included a terrifying bespoke 100-foot wave that hits the yacht and ‘pitchpoles’ or capsizes the boat, ending with the character Richard underwater, disappearing into the depths of the ocean. Leading up to this complex ‘finale’ shot, we created 40 CG storm shots, which were a mix of live-action and full CG as they pass through the developing cyclone."
“We approached the pitchpole sequence in stages, starting with the open sea. We used a procedurally generated ocean layout as a base, created by our animation team. This included 21 individual hero waves that were placed by hand and then simulated to sync with the key moments in the shot to sell the intense drama of this pivotal moment in the film,” says Bennett.
Get the Creative Bloq Newsletter
Daily design news, reviews, how-tos and more, as picked by the editors.
Milk animated the main wave by hand, but also used Side Effects Software’s Houdini for ocean shots. The pitchpole sequence involved piecing together several different live-action elements and adding in CG ones. In recent times, too, the studio has moved to rendering in the cloud, via both Amazon and Google solutions.
“The project would not have been possible to render without the cloud – 140 shots of which 70 were stormy ocean,” says Bennett. “During processing we peaked at around 130,000 cores so we needed to be able to scale up as required.”
02. Fleshing out locations in Big Little Lies
If you watched the first season of Big Little Lies, the HBO television series about three troubled mothers in Monterey, California, you might not have noticed any kind of visual effects at all. Which is exactly what VFX supervisor Marc Côté from Fake Digital Entertainment hopes.
He oversaw a multitude of seamless work on the show, much of which was to flesh out locations that needed to appear as if they were along the coast. Several dramatic moments, including a car crash and a dream-sequence cliff fall, also fell within the invisible effects approach.
Côté suggests there were two kinds of VFX work done for the show. The first he calls ‘advanced cutting’, “where you’re creating elements that give you the right emotion, the right timing. These are things like split-screens, time warping and clean-up. More story-based effects work.”
Côté adds, “The second was more classic visual effects with greenscreen to bring the actors into different locations and to enhance the locations they shot at. One was the location of the Blue Blues, the place where all the characters go and talk together, which was a marina cafe but it was actually shot on greenscreen. And there’s also the house that’s right on the ocean, but it was not shot there.
“There’s even another kind of editorial effects type of shot where, because of the production schedule, the actors can’t all be there at the same time or place, so we will shoot them separately and just put them together in the same shots. It solved schedule issues and problems where we could end up paying a lot of money to make it happen.”
03. Populating a stadium in Three Seconds
Sporting films now heavily rely on invisible effects to tell their stories. That’s because it’s expensive – and logistically difficult – to fill stadiums with cheering spectators. Achieving this via digital allows filmmakers to not only widen the scope of the story, but also control what is happening in the crowd itself.
Visual effects studio CGF carried out virtual stadium and crowd augmentation for the film Three Seconds (also called Going Vertical), about the stunning USA versus USSR basketball match at the 1972 Munich Olympics.
The Olympic stadium and others seen in the film were recreated by CGF in CG, down to the finest details. “We were looking for real prototypes for each of these gym halls,” says CGF visual effects supervisor Pavel Bezborodov.
“All of the gyms have a vast number of small different parts: wiring, switches, and interior finishing. In order to make everything true-to-fact and easy for the artists to work on textures, there were thousands of elements like locks, labels, switches, door handles, wooden panels and other numerous details.”
The game-play scenes in the film were generally filmed in a pavilion surrounded by bluescreen. CGF then inserted its CG stadium pieces where necessary. For crowds, a 3D approach was considered but it was realised early that this would require the creation of more than 20,000 virtual people.
Instead, a ‘sprite’ solution was used where crowd members were shot separately with a six-camera setup that then allowed them to be placed on ‘cards’ and positioned randomly in the stadiums.
“We used only 300 real people,” notes Bezborodov. “They were the audience who were seated in the first row, and the rest of the crowd that stand behind them were CG crowd. We changed their clothing and we made them act with different behaviours.”
04. Compositing old footage in The Last Movie Star
Sometimes, invisible effects are aimed at just being a whole lot of fun, such as in the film The Last Movie Star. Here, Burt Reynolds plays an ageing movie star in the throws of realising that he is no longer the attraction he once was.
The film called for scenes of Reynolds appearing in some of his previous key movies, including Smokey and the Bandit and Deliverance. The invisible effects work was handled by Trick Digital, which typically took green-screen footage of the actor and composited it into the original films.
“The biggest challenge,” outlines Trick visual effects supervisor Adam Clark, “was that these original films were not shot with the intention of having the actors removed from them, and the process to cleanly remove Sally Field and Jon Voight from these shots required the removal and replacement of more in the frame than you might expect.”
The studio utilised Blackmagic Design’s Fusion Studio for the compositing – a lot of this was simply grunt work to make the shots feel seamless. “From a technical standpoint, the biggest impact was on tracking,” says Clark.
“We had separate node branches that blurred, de-grained and re-coloured the image upon which we would then track and then use that tracking data elsewhere. We opted to retain the original grain and match, versus removing the grain across the shot and then re-graining the entirety.”
05. Enhancing the action in Sicario: Day of the Soldado
Sicario: Day of the Soldado is the sequel to the 2015 film, Sicario, and traverses locations on the border between the United States and Mexico. The invisible effects work, crafted by Rodeo FX, ranged from adding in military vehicles in various scenes to altering landscapes and even enhancing make-up wounds.
“From day one,” says Rodeo visual effects supervisor Alexandre Lafortune, “we knew our mandate was to produce invisible effects that would enhance the story told in Day of the Soldado. To keep our work as realistic as possible, we had to do a lot of research, especially for muzzle flashes and guns. The director had something specific in mind for these shots and we did everything necessary to match his vision.”
Rodeo’s list of military vehicles crafted for the film was extensive, and included a drone Predator, Black Hawk and Seahawk helicopters, Humvees, Ospreys, as well as Mexican police vehicles and other cars. But these were all made simply as dressing for airport or army base shots. That kind of ‘in the background’ work is the ultimate in invisible effects.
Another kind of invisible effects ‘staple’ in the film are shots that go by without many realising that any kind of intervention was involved at all. One of these is a car pile-up that involves a small explosion. Rodeo assembled three separate plates into one for the final scene.
“We extracted the characters and the action, following the timing we were given,” explains Lafortune. “To ensure the continuity, we added smoke and shadows on the walls of the buildings with compositing. We also did matte painting to show the alteration of the ground, and broke the rear window of the car. This is a clever mix of assembling details!”
06. Liam Neeson on a train in The Commuter
While director Jaume Collet-Serra’s train-ride thriller The Commuter starring Liam Neeson includes its fair share of carriage crashes and explosions, the film also contains a surprisingly high number of invisible effects shots involving digital trains, set extensions, plate stitches and exterior environments.
That work was overseen by production visual effects supervisor Steve Begg, with Cinesite delivering 860 shots as lead vendor. A large portion of these shots were exterior composites of the environment the train is travelling in, often seen through the windows of the carriage.
Neeson and fellow actors were filmed on a bluescreen train set on a hydraulic rig to provide movement, while a massive effort was required to acquire environment reference and then to rebuild that digitally.
“To create these environments and help us plan the visual route, Steve Begg and his team went in and around New York and filmed 360-degree elements from the back of a truck, as well as from trains and a helicopter,” outlines Cinesite visual effects supervisor Stephane Paris.
“Whenever we see the exterior of the moving train in this section of the movie it is CG, as well as the environment it’s travelling in, apart from a few shots filmed from a helicopter. The daylight environment shots of the train formed the main body of our work – approximately 400 shots!”
At one point, Neeson’s character becomes embroiled in a fight against his adversaries. It is played out in a single long shot that was actually 14 different takes stitched together. “An effort was made by the crew to line up subsequent shots to match the positions of the actors and camera in previous shots,” explains Paris.
“However, the perspective of the carriage, camera zooms and background deformation were often not smooth and it was necessary to re-project the carriage interior onto geometry to give more camera control for the multiple required transitions.”
Some of the weapons seen in the fight, including a guitar and an axe, were CG, as were smashed glass and window elements. This was in addition to the digital exterior environment and even portions of the train.
“Also,” notes Paris, “digital versions of the characters fighting were created to aid the transitions between the plates, as well as the environments and train infrastructure in many instances, so that the camera could move freely through the action and the divisions between the carriages. The stunt actors’ faces were replaced with the actors in several, more physically demanding shots.”
07. Face replacement in I, Tonya
I, Tonya, directed by Craig Gillespie, also employed face replacement, which was crucial to telling the true story of the notorious Olympic figure skater Tonya Harding (Margot Robbie) implicated in the ‘hit’ against rival Nancy Kerrigan. First, there would need to be some extensive skating scenes with Robbie, who is not a professional ice skater, portraying Harding on the ice and pulling off her classic triple axel move.
That’s where Eight VFX came in. Their methodology was to employ a seamless face replacement approach for Robbie, and then also fill out arenas with crowd extensions.
“Margot actually did five months of skating training and was able to do a huge amount of the skating herself,” says Eight VFX visual effects producer Juliet Tierney, who worked with visual effects supervisor Jean-Marc Demmer on the film.
“For the more complex parts of the routines and jumps, there were skating doubles who were filmed with tracking markers on their face and head. For some shots, after the double had skated, a bluescreen area was set up near the rink so that we could film Margot’s facial expressions for those tricks. For other shots, where no bluescreen takes were available or the skating was too fast, full CG face replacement was used.”
This involved facial capture using a photogrammetry rig supplied by EISKO. Robbie was captured with four different make-up looks. Once the data was processed, it provided Eight VFX with both facial geometry and textures. This was then brought into Maya for animators to rig and hand animate Margot’s facial expressions.
Careful editing also meant that shots transitioned from Robbie starting and ending a routine while stunt performers and the face replacement was used for different parts of the scenes. Still, face replacement was only half the challenge. Harding’s routines take place in different locations, including at the 1994 Winter Olympics in Lillehammer, Norway.
Eight VFX had the rinks used for the shooting scanned, and then recreated them in CG. That way they could be populated with digital crowds and signage for the final sequences. Tierney is particularly proud of one three-minute shot that was created from three different shots.
“The first and last shots have Margot skating, and the middle shot has face replacement. Because this is one final shot the middle plate had to be perfect so you can’t feel the difference between the real Margot and her CG double. We also rebuilt the stadium in CG, added CG crowd for the audience and gave the sequence its own look with anamorphic lens flares.”
08. Fawns and flames in Three Billboards Outside Ebbing, Missouri
Best Picture Oscar nominee, Three Billboards Outside Ebbing, Missouri, is director Martin McDonagh’s tale about a woman who, after her daughter is murdered and no culprit is found, calls out the police for inaction via three large advertising signs near her town.
The small film still required some invisible visual effects work by Union VFX, including for when Mildred (played by Frances McDormand), encounters a fawn, and as opponents to her signs try to burn them down.
A tender moment occurs as Mildred is tending to some flowers at the billboards and sees a fawn nearby. McDormand and the fawn were shot separately, with Union compositing the two elements together.
“The trickiest thing was the fawn,” outlines Union visual effects supervisor Simon Hughes, “because basically we had to go to a different location where the animal trainers had deer and we had to set up a bluescreen there. And then we had to wait around for the right kind of lighting direction and for the deer to do things.”
A more complicated sequence has Mildred and her son trying to extinguish the billboards after they have been set on fire. Here, a combination of practical fire elements and digital fire made the shots possible. “Frances stood in position and we had some lighting rigs out of frame for interactive lighting,” says Hughes.
“We also manage to actually squeeze a little bit of fire towards the edge of frame away from her. But then we also took Frances out and did an element shoot matching that camera position, setting the actual billboard on fire using flame bars and some paraffin on the face of the billboard. For the interaction with the fire extinguisher we resorted to CG fire, which we did in Houdini, and some smoke to come from the extinguisher.”
09. Not-so-showy VFX in The Greatest Showman
Several VFX vendors contributed shots to Michael Gracey’s debut feature, The Greatest Showman, which tells the story of P.T. Barnum and his founding of the Ringling Bros and Barnum & Bailey Circus.
Production visual effects supervisor Mark O. Forker oversaw these studios, including Rodeo FX, which took on sequences involving a mix of fantastical cityscapes and scenes that required placing characters in sets that didn’t exist, and even burning down P.T. Barnum’s theatre.
As part of his stylistic approach to the film, Gracey decided to partly realise the cityscapes with miniatures. Rodeo then augmented the shots with painterly skies. “All the miniatures were built in 3D in the proper scale and then 3D printed,” states Rodeo visual effects supervisor Martin Lipmann.
“We placed them on a huge table that was lit from underneath; as live footage was going to be added to these shots, everything had to be to scale. We shot this sequence with a 30-foot Technodolly. We reprogrammed the moves from the previs on set. With the repetitive movement we were able to create different lighting scenarios, from dark to light to moonlight and sun.”
The burning Barnum theatre used a number of invisible effects tricks. A mix of custom practical fire on gas pipes matching the right window measurements served as elements. The strength of the flames on the pipes was controlled to re-create the progression of fire through the sequence.
“We also used CG fire for the lighting interaction with the CG extension of the building, and CG ashes and embers to connect all the shots together,” says Lipmann. “The collapse of the building was done in Houdini and comp’d in Nuke.”
10. The portrayal of war in Darkest Hour
In Darkest Hour, Joe Wright’s account of Winston Churchill’s (Gary Oldman) time during the early days of World War II, visual effects had a sparse, but again, crucial role in the storytelling process.
The filmmakers, which included VFX studio Framestore, looked closely to 1940s archive imagery as reference for shots of buildings of the day, as well as a number of classic war-time scenes.
But this wasn’t a film full of massive battle scenes. One initial challenge for Framestore was adding dirt to period buildings. “We Lidar-scanned all of the main buildings, including Downing Street and The Treasury, as it was the only way to get a very precise template,” explains Framestore visual effects supervisor Stephane Naze.
“Then we built a proper CG model and applied textures on it. The tricky part was to film everything in camera without any bluescreen – it meant a lot of work in roto for all the characters and extras to be able to modify the buildings in the background.”
It took three weeks to render the shot with more than 50 per cent of the machines allocated for this film – no room for mistakes!
Stephane Naze
The studio’s other main challenge came from battlefield shots, mainly aerials showcasing carpet bombing with banks of explosions. The timing of these was locked in 2D to provide quick iterations, with the assembly of all the elements finished in CG. Naze says the toughest shot to achieve was a scene on the French battlefield, which finishes on the face of a soldier.
“Everything is CG until you finish the camera movement on the dead soldier. The goal was to get a very transparent transition and not be disconnected by this dramatic moment in the story."
"Also technically, the shot was very challenging by the nature of the effect – big explosions, flames, fire, a lot of research to be accurate. It took three weeks to render the shot with more than 50 per cent of the machines allocated for this film – no room for mistakes!”
This article was originally published in 3D World, the world's best-selling magazine for CG artists. Buy issue 241 here, or subscribe here.
Related articles:
