tracking a mouth to a jpg
by Oskar jensen on Oct 8, 2009 at 4:08:18 pm
Hey I am doing a remake of a music video.
And I need to figure out if I can take the mouth and movement of the original singer and mask it on to a JPEG of another singer
for example sinead oconners mouth...
I guess I need to ad the mask to the tracking movement? but how do I do that?
Re: tracking a mouth to a jpg by Dave LaRonde on Oct 8, 2009 at 4:48:42 pm
[Oskar jensen]"I need to figure out if I can take the mouth and movement of the original singer and mask it on to a JPEG of another singer... "
Your YouTube link took me to "Sinead O'Connor - Nothing Compares 2 U".
In all of the five minutes and fifteen seconds of the video, I saw absolutely no example of what you're talking about.
But here's an obvious fact: singers move in three dimensions when they sing. They move their heads up and down, left and right, back and forth, and they TURN THEIR HEADS.
You might be able to track the large movements of the singer, but it's extremely difficult to to track the head turns. That's just one difficulty.
Here's another difficulty: since singers move their mouths when they sing, you'll have to animate a mask around the mouth, somtimes on a frame-by-frame basis. Are you prepared to do that?
Let's assume you can overcome the first two difficulties. I haven't even mentioned the the most fundamental difficulty:
Have you contemplated how you will handle the change in perspective when the singer does an involuntary head turn during singing? Have you used your mind's eye to visualize how that change in perspective will look composited onto a static JPEG Image?
I think you need to do a little more than say, "Hey! I have a great idea!" I think you need to actually THINK ABOUT HOW IT WILL LOOK. If you haven't done that yet, you haven't done a complete job.
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA
Re: tracking a mouth to a jpg by Greg Neumayer on Oct 9, 2009 at 5:20:45 pm
Dave's right. Unless you're going for the Saturday Night Live look (there's a reason they do it so crudely!), you'll probably just have to do it frame by frame. If I were doing it by hand, I'd pick a pore on the tip of her nose (I know, there aren't many to choose from) or something else that will give me an approximation of total movement that excludes the nuances of the lip movement. If it didn't track well using the tracking tools, I'd lay my new layer (maybe just a null for now) on top of the nose, and use my page up/down to click through frames, and use my arrow keys to nudge my null for each frame to my reference pore. (you could also start by tapping to every 5th frame or so, then get more detailed in areas that end up being problems.) After I've got a null has my movement the way I want, I'd parent my new mouth (with soft edges, color correction etc.) to the null, then adjust it's position until it's lined up with the old mouth. Now, after you're done with that, you could start to adjust rotation based on her head rotations, if necessary. This probably won't require the same detail as the position.
Post your work when you're done, It'd be interesting to see.
p.s. This is a good opportunity for a cup of coffee and Pandora, as it will get mindless after a while! :)