Discover the secrets behind ultra-realistic CG

Reality is open to interpretation, in most cases. Accurately simulating how much weight a bridge can safely support during an earthquake could mean the difference between life or death. A military training simulation might involve a fake car explosion with a hired cast of screaming blood-drenched amputees and fake limbs scattered, or forcing troops through a mock city filled with the synthesised stench of rotting corpses.

In reality, a car falling from that bridge would rarely blow up; in fact they have been carefully designed to avoid such reactions. James O'Brien, professor of computer science at UC Berkeley says, "They might be crumpled, might be munched up into an unrecognisable pile of junk, but they almost never blow up."

The same is true with an exploding hand grenade. "When you throw a grenade in a film it makes this fantastic eruption of fire. In the real world, it's actually scary how little it seems to do. There is a little puff of smoke that looks very unimpressive, but ten feet away, someone will just fall down dead." But from the audience's point of view, that doesn't make a very exciting blockbuster movie.

Entertainment tends to push reality, not recreate it. And ironically, after a century of film, humans are conditioned to expect the spectacle, often to the point where this false information is perceived as true and real-world spectacle is seen as inaccurate and boring.

Yet regardless of how reality is tweaked, if a film is live-action, each effect has to be believable, and doing so means understanding the physics of the real-world, controlling simulations that attempt to mimic proper physics, and then altering those simulations to get the proper dramatic results.

The beauty and the geek

Imagine this scenario: the simulation is flawless, behaving precisely as it should in the real world. The fabric flutters and stretches as the cape whips in the wind. Superman rockets over the ground, pulling up at the last second, slams to the ground and skids to a stop on both feet. It's a dramatic moment, a pivotal scene that has the audience clutching the theatre armrests. And as Superman menacingly places a fist on each hip and stares down his enemies the cape continues its movement, flipping over his head.

Obviously this is not the desired effect, but what alterations are needed to maintain the sense of realism while faking the cape’s response? Superman flies forward in high velocity and comes to an abrupt stop. The cape’s momentum would keep it moving forward, as it has not experienced the same deceleration forces as the body; it is connected at the shoulders, so only this point of connection has been slowed down and the rest of the cape is free to keep going. To resolve this, the TD working on the simulation would need to apply artificial forces, such as wind or pins or simply counter-animate the forward motion.

Simulation fixes

A building collapsing might be obscured in the dust, requiring a simulation that manipulated the debris coming towards camera, keeping the building in view. A continuity cut that requires an object to land in a specific location where it would not land if left to physics alone. The camera and gravity make objects look like they are falling too fast or too slowly.

All of these scenarios require some sort of cheat. While the physically based simulation might be scientifically accurate, more often than not, the physical models work against the process. Ben Cole is the lead developer of Kali, MPC's simulation software. Kali was built on the back of Pixelux’s software, DMM (Digital Molecular Matter).

Kali uses a finite element-based approach, and is a very physically accurate simulation system. "The Pixelux solver is definitely designed to work with quantities that correspond to real-world values and give a highly realistic result," says Cole, "but the work we did integrating their API wasn't just about the numbers. It was about creating a tool that was usable and controllable by technical directors in a VFX context, while giving them access to all this rich and powerful physics."

Destruction development

There are two kinds of physics simulation used for destruction effects in film: rigid body and finite element.

Rigid body destruction is not physically accurate; the simulations can't handle flexing, bending or realistic fracture of materials. Rigid body constraint links can be broken based on forces or scripted connectivity. With only six degrees of freedom - three dimensions of movement, three degrees of rotation. Rigid bodies do not allow for deforming or bending. For rigid objects, such as stone, the behaviour is acceptable, but when the material is expected to deform, the results are less appealing.

Finite element destruction, by comparison, has a variable degree of freedom. A Finite Element Method, or FEM, system that has a particular set of material parameters may be able to capture a wider range of phenomena, but depending on the assumptions behind it there will still be other things it can’t easily model. "Having simulation tools with more degrees of freedom and larger parameter spaces is inevitable in my opinion," says Cole, "as they enable the production of a much broader range of effects.

However, gathering the information needed and tuning the parameters of these simulations becomes an increasingly complicated process. Also, we often need to make things behave in ways that they might not in reality because reality might not deliver the director's vision. At the end of the day it's the eyes of supervisors and TDs which guide us the most."

Consider the cape example: at this time, it is often easier for TDs to add artificial constraints to their simulations, such as unmotivated forces or other keyframed or procedural modifications, rather than try to work with increasingly accurate representations of nature.

Many studios still do much of their simulations with rigid bodies, for instance, and choose to cheat or approximate the properties not represented, rather than working with a strictly physical model. Even with a FEM simulation, the additional degrees of freedom, and more accurate physics, might not be doing what the technical director wants and so will need to be constrained or otherwise adjusted.

Deformable fractures

It started with a 1999 PhD thesis published in Siggraph written by Dr. James O’Brien and his graduation student advisor Jessica Hodgins. That paper, Graphical Modeling and Animation of Brittle Fracture developed the original fracture sim technique used inside DMM and Kali. Pixelux co-founders Eric Parker, Vik Sohal and Mitchell Bunnell contacted James, by now a professor at UC Berkeley and the most advanced in this area of study, about building software around his work.

At the time the process was very slow, taking days to run a simulation. "When I did the work at Georgia Tech as a grad student, I was using a half-million dollar SGI workstation on my desk, and the sims were running in the machine room on the big Origin machines; the size of a refrigerator with multiple processors and were in the multi-million dollar range. I had five gigabytes of disc space,” says O’Brien. But the new goal was to run in real-time video games on an Xbox 360, plus build a slightly different optimised version of the engine, the same MPC uses now. And thus DMM was born.

Sucker Punch and DMM

MPC’s client, Sucker Punch director Zack Snyder, had sequences requiring an advanced finite element system with an unusually advanced destruction technology. MPC considered writing its own solution, but research led to a paper by Eric Parker and James O’Brien that described Pixelux's DMM software, and the studio grabbed the opportunity to potentially save a lot of time.

According to Vik Sohal: "We had developed DMM for video games, but MPC wanted to use it for film." This was no small task, one that necessitated porting DMM to 64-bit Linux and changing some of the internal data structures to make it viable for very large number of tetrahedral elements.

"For games, DMM could simulate around 3,000 elements in real time. Offline, MPC's Maya plug-in could handle around 300,000. But for its high-end film destruction MPC wanted to push that to a million elements, with virtually no limit on CPUs or memory."

Tetrahedral Finite Element

Any 3D artist is familiar with a mesh, the 2D manifold wrapped around a 3D object representing the surface of that object, but finite elements represent the interior; the volume within. A tetrahedral finite element method is a type of simulation method that is a simple solution to a more difficult problem.

A single tetrahedron (tet), is simply a type of element - a pyramid with a three-sided base, the simplest 3D shape possible. According to O'Brien, "A finite element refers to breaking the object up into a finite or limited number of distinct, individual elements, thousands or even millions of separate tetrahedra, each with equations telling them how to move and behave. Whatever material properties are assigned to the individual tetrahedra, be it rubber, metal, or whatever, the structure formed from those tetrahedra will behave as if it were a solid object made out of that material."

Each tetrahedron can deform in limited ways, but joined together you have a structure that can deform and bend in a fairly complicated set of ways. For example, if you stretch a tetrahedron into a different shape, you can describe how it responds to the internal forces trying to push it back to its original shape. If each tetrahedron didn’t allow for flexibility, they would be rigid and therefore the results would be rigid, with no elastic energy.

This would be rigid-body simulation. When you use rigid-body simulation and there is no modelling or storing of internal elastic energy, the rigid body methods give you results akin to blocks falling into a pile. One looks like a real-world effect, the other like an effect you’d find in a Tom and Jerry cartoon. Jerry whacks Tom in the teeth and they break loose and drop out of his mouth to the floor. It has no tension.

Another example might be gradually bending a wooden pencil, bending it further towards a bowed shape until it hits the breaking point. When it breaks, the pieces that were previously bowed are now free to flip and fly outwards as they try to straighten themselves out. This is because the energy that was stored in that bowed shape is released, it comes up as elastic energy flinging the pieces, flinging the pencil, snapping in multiple locations as splinters and pieces fly outward.

The same logic applies to glass, and (other than safety glass) is the reason it’s so rare to see a crack stop part way through instead of continuing through to the other edge. The reason is, at a microscopic level, the glass is reacting in the same way as the pencil. When glass is deformed enough that it starts to break, the energy stored in the bent glass flings the pieces from either side of the break, driving the crack farther through the glass.

Now imagine whacking a sheet of glass with a hammer. Even if that stored energy is not visible at a macroscopic level, if you don’t have that kind of bending behavior modelled in your shape the reaction wouldn’t be the glass flying, as you would expect. Instead the glass would fall apart into little pieces that drop neatly into a pile on the floor. There is a lack of dynamism in the resulting effects, and they don't have quite the same feeling of energy.

Why is this important? Going back to the pencil example, if you bowed that pencil towards your face, you would be aware of that stored tension and know splinters flying towards your eyes would be dangerous. You would flinch, squint, turn your head in anticipation. If that energy wasn't present your response would be radically different.

In the same way, you want the audience to have the same awareness of danger, picked up in the subtle visual queues that are hardwired into the human subconscious, even if they can't quite put their finger on why one situation has more impact than a nearly identical situation.

Without adding that tension, the result is the effects tend to look a bit bland. In one scene it’s flying shrapnel, in another it's flying cardboard. Like the millions of tetrahedra, the reasons can be barely distinguishable, but the different results add up.

How does DMM work up

DMM simulates the stress inside each tetrahedra, then calculates that stress based on a number of parameters. For example, the 'toughness' material setting is what determines how much stress a tet-tet node intersection can take before it separates.

Plasticity has multiple aspects too. Objects can behave according to real-world physics or be adjusted to any number of unexpected behaviours. Much like a film set might have a 'stone' wall made of Styrofoam blocks, adjusting parameters such as density or toughness can have the same effect. How it collapses can also be altered, causing it to bend or expand strangely as it falls.

With DMM, if you tessellate the volume enough, you will get a closer approximation to the kind of material you are simulating, including a more natural fracture pattern. This is an intuitive result, akin to building a model with smaller and smaller parts. With big parts you lack the detail, with smaller parts the detail is implicit.

In the case of DMM, finite element destruction offers the rotation and movement degrees plus the 14 different material parameters, which regulate how each tet reacts to stress. The number of tets also affects how realistic your simulation is. Where a rigid-body simulation may have seven degrees of freedom, a finite element simulation may have thousands or even millions.

In 2009 a Parker and O’Brien paper, Real-Time Deformation and Fracture in a Game Environment was published in SCA covering the DMM technology.

O'Brien's research

O'Brien continues his research at UC Berkeley. He and a team of advanced students are working on triangular simulations using triangle elements. For example, destroying a model that is not a solid volume - perhaps a surface such as a glass window - this will be faster and more efficient than tetrahedra, and possibly better than anything currently available.

Currently designed with film in mind, the open source software is called ARCSim (Adaptive Refining and Coursing Simulation) and is available online to use here. It has been discussed in two different papers, one, Adaptive Anisotropic Remeshing for Cloth Simulation (Siggraph Asia 2012) that focuses on cloth simulation and the other, Folding and Crumpling Adaptive Sheets (Siggraph 2013) that focused on modelling that requires crease formation such as folding paper.

It doesn’t limit the user to a constant triangulation; instead it dynamically adjusts and refines the surface as the character moves, adding triangles as needed, representing wrinkles and folds that are higher resolution and higher detail.

Using the same simulation code with added extensions, the second paper accomplishes realistic crumpling with sharp creases, such as in paper or aluminum foil. Several other universities are already building their research on top of this code.

As-of-yet-unpublished work by a post doctorate and a student is using the same adaptive mesh-refinement engine to get very high-resolution crack propagation in thin shells, such as an eggshell or Christmas decoration, that will be indistinguishable to reality.

ARCSim calculates where the cracks, folds or wrinkles should occur and adds detail where necessary without an underlying discretisation attached. "One of my goals in this set of work is to not see the triangles, so you don’t realise it’s a polygonal mode," says O'Brien. "Simulation time is very fast; a simulation of the ornament might be half an hour.

The reason is, other methods might require a ridiculous number of triangles - for example, 500,000 - while this method might use 10,000 or less. Most of the triangles are large and cover a wide area, and only refine the area required during the period the high level of detail is needed. Once there is no longer a need, those fine triangles will be coarsened back up to large ones."

Words: Renee Dunlop

Renee Dunlop has over 20 years working as script analyst, creative and technical writer, 2D and 3D artist. She is editor of the book Production Pipeline Fundamentals for Film and Games. This article originally appeared in 3D World issue 180.