If you want perfect lipsyncing, then you may want to try Reallusion's iClone AccuLIPS, which detects and extracts text and viseme (the position of the face and mouth when saying a word) from audio, or imports a prepared script for precise lipsyncing. You can create natural speak with the co-articulation design, and further fine-tune every word’s visemes and strength level.
AccuLIPS contains an English dictionary with 200,000 default words, which are easily customised by adding new words, or changing others. In the video above, and outlined below, I show you how to use it. I believe that my mini-guide will prove useful, as existing resources focus on AccuLIPS and Face Puppet separately without much integration. With this short piece, I aim to emphasise the effective combination of these tools to achieve a more dynamic and dramatic character performance, moving beyond simple lip-syncing techniques.
(Image credit: Reallusion)
AccuLIPS is part of Reallusion's iClone, which is just really simple. The pipeline between Character Creator 4, iClone 8, and Unreal Engine is pretty seamless once you’ve done it a few times.
(Image credit: Reallusion)
How to make a lip-sync animation
01. Enhance the AccuLIPS clip
(Image credit: Reallusion)
Before I do anything else, I like to enhance the AccuLIPS clip by going into the lip options and setting the smoothness to 1 and the strength to 1.2. I find this tends to make the movements look a bit more natural.
(Image credit: Reallusion)
As well as setting the smoothness and strength, you can also set the talking style.
02. Set the base emotion
(Image credit: Reallusion)
I then like to set what I call my 'base emotion' for the clip, which can be done with an animation from the Digital Soul pack of pre-animated facial expressions, or by capturing my own expression from my iPhone via Motion LIVE for facial mocap.
03. Make another pass
After that, I make another pass, often with face keys, to enhance and finetune particular moments in the dialogue.
04. Make more passes in Face Puppet
(Image credit: Reallusion)
Then I'll make another number of passes with iClone's Face Puppet, adding more emotion at key points and generally moving the mouth in a more expressive way.
05. Add some head motion
Finally, I add some head motion with Face Puppet, and then whatever body motion the character requires for the scene.
Peter Haynes is a filmmaker based in New Zealand and a prior winner of the Epic MegaGrants award. For this piece, he takes time out of his busy schedule to generously provide a quick tutorial on his process for enhancing lip-sync animation using Reallusion iClone.
