ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

Automated Lip Sync

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Cody Kyle
Automated Lip Sync
on Apr 7, 2010 at 8:34:57 pm

I was wondering that for AfterEffects is their an effect that lip syncs a .wav audio file to an still image on the screen and manipulates the images lips on the the screen with it or is their an outside program that does this when you import it into AfterEffects. I am currently am working with AfterEffects CS3 and (I may be mistaken) a previous employee was able to do this some way. Now I am not talking about the lips syncing up exactly with the audio, more like a jibber jabber kind of lip moving with the audio. Has any one heard of anything similar to what I am saying? Any feedback would be appreciated.


Return to posts index

Dave LaRonde
Re: Automated Lip Sync
on Apr 7, 2010 at 10:06:04 pm

[Cody Kyle] "I was wondering that for AfterEffects is their an effect that lip syncs a .wav audio file to an still image on the screen and manipulates the images lips on the the screen with it or is their an outside program that does this when you import it into AfterEffects."

Two words: dream on.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA

Return to posts index

Byron Purvis
Re: Automated Lip Sync
on Aug 16, 2010 at 8:58:19 am

"Dream on"? Your comment is not at all helpful. You should just refrain from commenting unless you have something useful to say. It's a very valid question, and something I've been researching for quite a while...

There are a number of lip sync applications that can take an audio file, analyze the phonemes and help automate the process. Papagayo and the Lipsync widget for Mac do a fair job of syncing and making after effects keyframes, but it takes some fidgeting and expressions to get it to work! There's also a java app called JLipSync, but I didn't have much luck with it since it would only take 8-bit mono wav files as input.

As far as I know, there is no completely automated solution for after effects yet, but I'm sure it's only a matter of time/money.

If you want to step away from After Effects, applications like Anime Studio Pro 7 or ToonBoom Studio might be better solutions, but they cost between $200-400.

The current solution I'm using is not completely automatic, but it does speed up the process significantly...

I use a combination of Aaron Rabinowitz's tutorial and a technique I found on youtube, where I speed up the mouth selection by pick-whipping the +/- keys (which normally control rotation) on a null layer, to a time remapped pre-comp containing my mouth shapes.

See the following links for more info ...

Return to posts index

Rob McLendon
Re: Automated Lip Sync
on Oct 18, 2012 at 10:41:44 am

This response has been EXTREMELY helpful. Thank you, thank you, thank you!!!

Return to posts index

Todd Kopriva
Re: Automated Lip Sync
on Apr 7, 2010 at 10:25:50 pm

You got an answer to this on another forum. It is good etiquette to cross-post the answer when you cross-post a question.

Todd Kopriva, Adobe Systems Incorporated
putting the 'T' back in 'RTFM' : After Effects Help on the Web
If a page of After Effects Help answers your question, please consider rating it. If you have a tip, technique, or link to share---or if there is something that you'd like to see added or improved---please leave a comment.

Return to posts index

Tudor "Ted" Jelescu
Re: Automated Lip Sync
on Apr 8, 2010 at 8:46:24 am

Straight from the cow's mouth... actually from Aharon Rabinowitz

Return to posts index

Richard Shaw
Re: Automated Lip Sync
on May 25, 2010 at 12:12:56 pm

This is probably way too late to help you out, but will hopefully be of use to someone else.

Obviously perfect lip sync can't be magically done for you, but it doesn't sound like that's what you're after.

I think you mean something along these lines:

This can be done by converting the audio to keyframes (right click, keyframe assistant). This creates a new null (Audio Amplitude). Parent the mouth layer's position to both channels and alter the expression slightly to something like this:

temp = thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider");
value+[0, temp]/2

Adding "value" uses the layer's original position. Use "+" or "-" to push the mouth down or up (down makes sense here, but up could be used to raise the eyebrows in time with the mouth, for example).

Changing the first "temp" to 0 prevents x axis (sideways) movement.

Dividing or multiply by the required amount will limit how far the mouth layer moves (here I've arbitrarily divided by 2 just as an example).

In a profile view of the character, the mouth layer's rotation could be parented to the Audio Amplitude instead of position.

If the mouth is moving a bit too erratically, use the smoother (window, the smoother) on the Audio Amplitude layer. Click on "Slider" to select all keyframes, Choose a tolerance level (probably 2-4 or so) and hit apply. There will no-longer be a key on each frame and all keys will now be easy ease.

Hope this is helpful.

Return to posts index

eddio pina
Re: Automated Lip Sync
on Jun 5, 2011 at 4:32:36 pm

NO... Don´t Dream on...
Wake up insted...

Ae is able to do automated audio animations. And YES without plugins.

Firts. You must understand about Lip Sync. Our friend Aharon Rabinowitz did an excelent job explaining this in this post.

Once you know how to lip Sync yo are ready to work with automated lip syncing.

You must convert your audio file to Keyframes. to do this. select your audio layer in the AE timeline, then go to Animation Menu > Keyframe Assistant > Convert Audio to Keyframes.

You´ll get a new layer in your timelime, this is a keyframe representation of your audio file. It has 3 layers, you can delete 2 of them (if you want to) and work with only one layer

Click on the "Slider" option of the keyframe layer to select it and them in the Graphic Editor (over the timeline) to see what you got.

Remember you must have done a time-remap of your animation the way Aharon shown above.

Now you can start asigning values to your Animation. You can "link" your keyframe layer to the Lips layer and watch if it works

Parhaps you must perform some adjustements, but this is it

Not so hard...

Best reggards...

Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2018 All Rights Reserved