Live Face is Reallusion's latest markerless facial mocap solution for iClone 7. It's an app specifically for the iPhone X that uses the phone's depth sensor and tracking software to track the user's face and stream the data into iClone through Reallusion's Motion LIVE plugin to create realistically animated 3D art.
Facial animation can be previewed and recorded in real time on single or multiple characters. It can also be performed whilst capturing animation (for the body and hands as well as the face) from other mocap systems.
I've used many different facial mocap systems over the years, marker based and markerless, and while none are perfect, the iPhone X/Live Face approach is remarkably stable and easy to use straight away. The depth sensor approach means it's tolerant of uneven and low lighting conditions, and the fact that the capture itself is performed on the phone rather than the PC means it's efficient and doesn't hog resources.
Live Face: Effective tracking
Head, eye and general tracking in Reallusion's app is effective and while expression distortions can occur when the user's head is rotated away from the camera, these are less noticeable than when using other markerless single camera systems. There are some expression limitations due to tracking accuracy – notably with asymmetric mouth shapes such as snarls and also eyebrow differentiation, which means that raising an individual eyebrow is currently not possible with Live Face. Limitations aside, tracking is quite robust and even covers cheek expansion – something I haven't seen with other markerless systems.
Lipsync with Live Face works well by default, though users can also change lipsync qualities by calibrating with their mouth slightly open (to emphasise mouth closure), as well as by adjusting strength sliders on the Motion LIVE UI.
Live Face for iClone 7: Worth the price?
There's no denying that the iPhone X is expensive, and assuming you already have iClone 7 and the Motion LIVE integrated mocap plugin, it will cost about £230 for the software to use the phone for facial mocap in iClone. Even with the cost of the iPhone itself, this is good value for a robust facial mocap solution. In fact, the combined cost of the phone and software is about the same as for iClone's existing Faceware system.
iPhone X/Live Face is a stable, efficient and easy-to-use facial mocap approach, and is a practical addition to the animator's toolbox. While it's not perfect (Apple, please fix the separate eyebrow tracking!), it can produce decent results quickly and with minimal post cleanup.
This article was originally published in issue 240 of 3D World, the world's best-selling magazine for CG artists. Buy issue 240 here or subscribe to 3D World here.
