Lip Sync

Hi,

I’m working on the final chapter of my next video on Newton’s third law of motion and just about to record the vocals for that chapter before doing the animation. At the same time I am also finishing the lip sync on the other chapters. This is potentially a time consuming part of the animation process, especially if you want to achieve physically accurate mouth animation. However it is possible to get a good result without having to spend a crazy amount of time if you are prepared to sacrifice some of the physical accuracy.

The way I do lip sync is very simple as I don’t use detailed mouth shapes. Instead I use jaw position to sync with the audio voice recording. This enables me to quickly produce the necessary animation. One of the potential problems of this type of lip sync is that if you don’t get the range of jaw position right you can end up with the impression of a flapping mouth opening and closing in a very unrealistic and mechanical fashion. To avoid this I use ten jaw positions; from a fully closed mouth, to fully open with a neutral mouth pose between these two extremes. Although the resulting effect is not physically accurate it produces an acceptable result with the benefits of speed.

I use a combination of Audacity®, spreadsheet and a pose library in Blender to do the lip sync animation. Audacity® is free, open source, cross-platform software for recording and editing sounds. I use this to identify frame numbers for the jaw positions. For example when I say “Force” in the video dialogue I identify the frame numbers in Audacity for the “F” and “o” relative to the beginning of the sound file for that part of the dialogue. The spreadsheet enables me to create a list of the frame numbers for these points in the Audacity file and through simple maths convert this to the frame number in the Blender file based on the known position of the sound file on the Blender timeline. I can then quickly go through the Blender file and enter a pose for the jaw bone at each of the frame numbers in the spreadsheet list. By using a pose library of jaw positions I can quickly go from frame to frame and enter the pose.

So for now it’s back to the lip sync activity. I’m looking forward to getting the video finished in the next few weeks…

Mac