ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

tracking and replacing a face with mocha

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Victor Camacho Guerrero
tracking and replacing a face with mocha
on Nov 16, 2011 at 9:51:23 am

Hi all,

a month ago I post how to track a face with mocha. thanks to Roland R. Kahlenberg who helpme in the proccess .

So i got a clip with a face who changes his perspective. Look here:



So i tracked the face, and mocha did an amazing work, so i export as mocha shape and place over the face.



So this is how look my composition:
http://www.switchfire.net/1.png

So want I want, was erase the nose, eyes and mouth from the guy. Looking like this:
http://www.switchfire.net/2.png

I applied this mocha shape to this composition, in order to matte what I wanted:
http://www.switchfire.net/3.png

Later , my idea was exporting a corner pin and apply to this late mask ( coming from the last picture) , in order get fixed to the face and keeping the tracking. However I exported in every way from mocha and I applied the corner pin to this matte, and nothing really worked.

Where I am wrong?

Many thanks in advance.


Return to posts index

Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 16, 2011 at 11:03:25 am

Hi Victor, here are a few tips -
1) all you need is a single frame of the 'empty face' when it is fully facing the camera
2) when creating this frame, take note that you will require additional pixels along its edges for you to feather. feathering the edges allows you to blend the the empty face over the original shot. without the feathering the edges will be obvious.

It will be good for you to know how to use mocha's Align Surfaces feature.

Here are the steps -
1) in your 'final comp', drop in the empty face layer on top of the backingplate (original shot).
1A) it looks like the last frame is when the face is fully frontal to the camera and you would have created your 'empty face' on this frame. THIS IS CRUCIAL to mocha's Align Surface feature.
2) In mocha, go to this frame. Duplicate the layer with your tracking data - this is to ensure that you have a backup of the original tracking data.
3) select the duplicate layer and ensure that you are still on the last frame. ensure that your Surface is placed accurately, then click on the Align Surface button (it is at the bottom of the Layer Controls panel. export the corner pin data.
4) in AE, create a mask around the 'empty face' - not the layer's edges but the edges of the actual 'empty face'. draw this mask a bit into the image to allow you to feather the edges interactively later.
5) move the CTI to the first frame - the 'empty face' will look like it's in an awkward position BUT don't worry. It's only awkward because you haven't pasted the tracking data yet). now apply mocha's data to the 'empty face' layer.

All should be fine if your track was good. the only issue you may have is with the edges of the 'empty face'. Hit the "F" key and adjust the mask's feather until the edges line up nicely. You may also adjust the points on your mask if you need more pixels along the edges. The mask will look out of place but that's OK.

Actually, if the mask was drawn in a precomp, you can also adjust the mask properties there. This may be more intuitive as the mask hasn't been affected by the corner-pin data and it hugs the 'empty face' edge as you would expect.

There are other ways of doing this but I find this works well for me. Hopefully others can share their technique(s) for a similar task.

HTH
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index

Victor Camacho Guerrero
Re: tracking and replacing a face with mocha
on Nov 17, 2011 at 12:32:10 pm

Hi Roland,

thanks a lot for your help. It was really useful. you could the result on here:



I saw the result is good but not amazing. I guess that maybe later, in order to composite properly I need to add some retouching to the corner pin.

Which are the variables, which it make the effect looks amazing? the plane, the surface?

I was wondering, why in this case, we need to use the align surface? and when in the future could I recognize similar situation for using this trick?

You could download the whole proyect from here : http://www.switchfire.net/face_liquid.rar

maybe you have a hint how to improve it.

Many thanks for your help!!


Return to posts index


Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 17, 2011 at 1:50:02 pm

Hi Victor, the tracking looks fine once the face is facing the camera but the earlier part of the track is not very good. I'm thinking that you will acquire better tracking data if you started your tracking just as the face begins to be looking straight towards the camera - then, a backward and forward track from this point will work well for you.

Taking a look at it now, it seems like you will benefit from a mocha shape export to ensure that the first many frames have clean edges, with respect to the 'empty face' image as well as to take advantage of mocha's per vertex feather feature. mocha's per vertex feather feature will also help in the latter parts of the composite.

I'll try to take a look at the *.rar file you uploaded but I can't be certain when I will be able to delve into it. One other option is for you to visit the Imagineer Systems user forum on their site. The Imagineer Systems folks are extremely helpful and I have started to post a little over there as well.

[Victor Camacho Guerrero] "I was wondering, why in this case, we need to use the align surface? and when in the future could I recognize similar situation for using this trick?"

I prefer to use the Align Surface feature in mocha when my insert element isn't rectangular and the center of my insert element and its boundaries (edges) will not fit perfectly onto the final composition's edges AFTER a fit to screen procedure.

There are lots of tutorials on how to fit an element onto the screen of a telephone. In most of these tutorials, the insert element is either already at a size that is identical to the final composition's size and if it is not, the insert element is scaled to fit the size of the final comp - this is usually done in a precomp.

This of course works well - most of the time BUT there is seldom the case where the aspect ratio of the insert element is identical to the aspect ratio depicted on the screen of the telephone. IOW, there are issues with such procedures and this will hit you in a bad way if there was a client logo which was part of the insert element because the logo will be distorted with respect to its original and will of course be highly noticeable by your client. This is just an aside, from your question but I thought I'd add that into the mix so that you can have a better idea why I prefer to use Align Surface in most of my inserts done within AE.

Align Surface will do everything that you need to fit an element onto its target area AND yet, ensure that the aspect ratio of the original imagery maintains its original aspect ratio, ie no distortion. Since, Align Surface works so well, as a matter of habit, I use it most of the time when I need to insert an element, regardless of its shape.

HTH
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index

Victor Camacho Guerrero
Re: tracking and replacing a face with mocha
on Nov 17, 2011 at 3:54:36 pm

Hi Roland, thanks for your time


[Roland] "I'm thinking that you will acquire better tracking data if you started your tracking just as the face begins to be looking straight towards the camera - then, a backward and forward track from this point will work well for you."

Well Roland, that is what I did thinking that it could be the best way to go. If you have a chance to see the .rar you will see.

for the rest of the explanation, it was really clear. I will see some tutorials on internet to see how they work .

Like always, many thanks.

If you have a chance of seeing the .rar , please let me know.

Thanks!


Return to posts index

Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 18, 2011 at 1:18:52 am

Hi Victor, I've taken a look at the *.rar file. The track seems good but it can be improved somewhat if you increased the Min % Pixels Used to 90-95%.

Ideally, you should be Cloning in PS instead of AE since it's a static clone. AE's cloning brush can bog down the system especially if you have lots of brush strokes.

You will also need to make a creative decision as to how you want the 'empty face' to look like and how its edges will blend with the backing plate.

I also suggest tracking using B-splines instead of an X-spline. I've attached a JPG to show you the shape I've drawn. The top of the head is iffy at the moment - like I said, you need to make a creative decision on how the edges will blend and the top of the head is quite important in this decision.

Here's a JPG

I'm suggesting B-splines because
1) with the face obscured in the early part of the track, you will have to make lots of shape changes - so might as well use B-splines which you will also need as you will want to export out a mochaShape to AE. Corner Pin data alone won't give you the exact edges you need.
2) you will likely want to use per-vertex feathering that is provided with mochaShape

So, you will need a Corner Pin data to ensure that your insert element matches the perspective changes of the original as well as a mochaShape (used as a track matte) to ensure that your edges match the original.

BTW, mochaPro will provide lots of options for this type of job. Compositing directly in mochaPro will leave out a lot of the guesswork in AE later.

BTW, have you watched the face replacement tutorial by David Torno at provideocoalition.com? It should give you lots of pointers.

HTH
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index


Victor Camacho Guerrero
Re: tracking and replacing a face with mocha
on Nov 18, 2011 at 11:06:00 am

Hi Roland, thanks so much for your reply.

Great answer, everything is quite clear. The only thing i do not understand, is why I need to increase the Min% Pixels used. Should I use this feature with this high value everytime I am doing a track in mocha???

Yeah i saw this tutorial, it helped a lot. The difference is my clip has a different perspective, and the clip to replace is coming from the same source.

thanks a lot!


Return to posts index

Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 18, 2011 at 5:11:02 pm

[Victor Camacho Guerrero] "The only thing i do not understand, is why I need to increase the Min% Pixels used. Should I use this feature with this high value everytime I am doing a track in mocha???"

There's a built-in 'efficiency' algorithm in the mocha tracking engine. The tracking engine will lower the Min% Pixels used if a large area was defined by a spline. Conversely, if a smaller area was defined, a larger Min% Pixels Used value will be applied - all by default.

However, on tricky tracks, it is a good idea to increase this value regardless of the size of your spline. Increasing the Min% Pixels Used will logically increase the accuracy of the tracked data at the expense of a slower track.

FWIW, your clip is a lot trickier due to the large perspective change which is compounded by the tracking region going entirely off screen.
For inserting elements into a picture frame that goes off screen, it's not a really difficult task for mocha as the shape of the picture frame is well-defined. For a face, however, it's of course much more difficult as the change in perspective introduces lots of other issues, especially when the top of the head comes into view first and the chin is out of view for the first few frames.

AND that guy has huge eyebrows! :-D

HTH
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index

Victor Camacho Guerrero
Re: tracking and replacing a face with mocha
on Nov 18, 2011 at 5:24:18 pm

jajjaaj the eyebrows were the most difficult part!

I habe been reading in the manual, that the surface is not possible to be keyframed, which I do not understand why. In the final track I have done, in the first part my surface sometimes " go to hell" because Mocha is not able to understand really well that huge change in the perspective.


Is there any way to achieve a surface aligned the whole time to the real surface of the face???

Many thanks like always


Return to posts index


Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 19, 2011 at 2:54:22 am

[Victor Camacho Guerrero] "I habe been reading in the manual, that the surface is not possible to be keyframed, which I do not understand why. In the final track I have done, in the first part my surface sometimes " go to hell" because Mocha is not able to understand really well that huge change in the perspective. "

If you were to be able to keyframe the Surface then what happens to the spline and the information contained within the spline? You will then end up with two tracking data from the spline and then another from the Surface. You either have one or the other. Even if you were to combine them, how would you interpolate between the two?

Additional, the Surface is required for the spline because the Surface has four points, each corresponding to a point on a corner pin. Splines inevitably will have more than four points so either you select four points within a spline to represent your corner pin points or you have a Surface that does the same thing.

The other advantage of having a Surface in addition to a spline is that the Surface can be offset from the spline - this comes in handy for Offset Tracking. This ensures that the tracking tracking data within the spline is not compromised by moving it to another area when Offset Tracking is required. BTW, Offset Tracking in mocha is similar to Offset Tracking in AE where the insert area lacks trackable information but there is another area which is on the same plane that does have trackable information.

For your issue with the drastic perspective change, well it actually isn't drastic but more the case that the tracking region is totally offscreen and is compounded by some persepctive change as it comes into view, manually changing the KF values in AE is one way. Of course there is also the Adjust Track feature in mocha. But for a minimal number of frames such as this, I prefer to jump into AE and fix it there.

BTW, there will be a mochaPro discount on Nov 25. Checkout the imagineer site for more details.

Cheers
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index

Victor Camacho Guerrero
Re: tracking and replacing a face with mocha
on Nov 19, 2011 at 10:39:37 am

I totally understand your point. My idea was, that if the plane could be keyframed , the plane should be adjusting to the perspective in the best way possible. Anyway, the option to do in AE I agree is the best option.

Roland, I really appreciate your support, it was extremely helpful!!

thanks a lot¡


Return to posts index

Roland R. Kahlenberg
Re: tracking and replacing a face with mocha
on Nov 20, 2011 at 1:56:04 am

[Victor Camacho Guerrero] "My idea was, that if the plane could be keyframed , the plane should be adjusting to the perspective in the best way possible"

mocha's Adjust Track module does (adjusting the perspective after a track) this BUT as I mentioned earlier, due to the nature of the tracking region, even that may not be of much help in this task although it should definitely be something you should look at as a first option.

Cheers
RoRK

Intensive AE & Mocha Training in Singapore and Malaysia
Adobe ACE/ACI (version 7) & Imagineer Systems Inc Approved Mocha Trainer


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]