ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

jagged edges when using alpha matte

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Wouter Dijkstra
jagged edges when using alpha matte
on May 18, 2019 at 4:37:07 pm

Hi Guys,

I'm having a mouth shape and use an alpha mask to mask out the teeth. But I see (even at 100% jagged edges. I tried suggestions from only fora like "Add Alpha" "Increasing from 16 to 32 bit" but it didn't make a difference. Is there a way to get rid of the jagged edges. It would be great to have a clean edge.

Way I did it:

INVISIBLE : MOUTHSHAPE
VISIBLE: TEETH (TM: ALPHA MATTE)
VISIBLE: MOUTHSHAPE



Return to posts index

Filip Vandueren
Re: jagged edges when using alpha matte
on May 19, 2019 at 10:43:57 am

Hi Wouter,

it's a pain, but it actually makes sense: the edges are semitransparent, and in this way of compositing, you're adding semitransparent white on top of semitransparent black at the edges, the result is not semu-transparant perfect white, but a grey that gives you his halo.

two possible solutions:
- precompose everything that's inside of the mouth (BG-color, tongue, teeth) then apply the mouth shape masking. You could do that masking in a number of ways: outside of your precomp with a trackmatte, or inside of your precomp by putting it as the top layer transfer mode "Stencil Alpha", or as the bottom layer, and have everything above it "preserve transparency". (These two last methods will break if your precomp is set to collapse geometries though)

- Only works IF your layers are all shape layers. Keep everything as is while animating, but before you render: hide the Teeth layer, now add this effect to the bottom mouth: "Calculations"; second layer: teeth; second layer opacity: 100; preserve transparency: deselcted. Now add the effect "Set Matte".



Return to posts index

Wouter Dijkstra
Re: jagged edges when using alpha matte
on May 19, 2019 at 5:35:40 pm

Hi Filip!

I was already afraid this was just the insurmountable consequence of Alpha Mattes and using the original layer underneath it. Thank you for the solutions I will try them out! ☺

It's a shame a function like Add Alpha doesn't have any purpose here. (With mask + inverted mask Alpha Add works like a charm.)

It's complex, but I'm happy that there is a way to solve it though. Great you were able to help me, it stuck to me ;-)


Return to posts index


Dave LaRonde
Re: jagged edges when using alpha matte
on May 20, 2019 at 2:45:48 am

Actually, you just have to change the Zoom in your comp window from 400% to 100%. you're looking at it at four times actual size. It will most certainly look bad zoomed in that much.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA


Return to posts index

Oleg Pirogov
Re: jagged edges when using alpha matte
on May 20, 2019 at 6:37:52 am

Filip has summed it up.
My workaround would be to apply Curves effect, setting alpha channel to max:


Return to posts index

Oleg Pirogov
Re: jagged edges when using alpha matte
on May 20, 2019 at 6:39:53 am

Applying to the matte layer


Return to posts index


Filip Vandueren
Re: jagged edges when using alpha matte
on May 20, 2019 at 12:20:14 pm

If you look at that against a non black background, haven't your create actually jagged aliased lines at the black border with this method ?



Return to posts index

Wouter Dijkstra
Re: jagged edges when using alpha matte
on May 21, 2019 at 8:59:01 pm

@Dave, You are right, the more you zoom the uglier it gets. But at 100% the border still shows.

The options Filip gave work quite nice. It just demands a little more work.

Thanks guys!


Return to posts index

Oleg Pirogov
Re: jagged edges when using alpha matte
on May 23, 2019 at 2:42:33 pm

I did. You're right, it's not perfect. Though by stepping back from extreme alpha curve values, some relatively nice in-between result can be obtained:


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2019 CreativeCOW.net All Rights Reserved
[TOP]