ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

How to Render Alpha But Not Apply As Matte In Comp!!!?

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Ross Newton
How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 14, 2010 at 6:29:34 pm

I'm an advanced After Effects user and also use Nuke. I finally had a situation where I needed to render an Alpha channel in an AE comp but I needed it to NOT matte the comp through that Alpha channel to the background. It seems AE does this by default. I tried lots of channel effects and render settings and couldn't get it to do this. Here's an example:

I have a CG building over a sky I created with a solid and gradient effects. How do I render this out of AE with the building and sky as the RGB and the building's alpha channel as the alpha? When I turn on my sky layer it fills in the alpha of the CG building. If I set the sky layer to have the CG building's alpha, then it mattes the sky out of the comp. Even if I subcomp the whole thing and apply the alpha that way, it still mattes to the background. I can never use the building's alpha and have it render the RGB of the sky. It seems to always "matte" the comp. I need to "unpremultiply" this matting effect.

In Nuke, I can just pipe in an alpha channel and it doesn't necessarily matte it.

The only work-around is to duplicate the comp, turn off my solid-sky, and render that as "Alpha-Only" and also render my original comp as "RGB".

What am I missing?


Return to posts index

Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 14, 2010 at 7:12:41 pm

In After Effects, if there's no transparency in the comp, you don't get an Alpha Channel. It sounds like you need to turn some layers on & off, or simply delete them.

To get a matte for the building, duplicate the comp, and get rid of the sky plus other undesirable things by deleting or turning off the appropriate layers. Then add the comp to the Render Queue and render using the Alpha Channel Only setting. I'm pretty sure it's in the Output Module; if I'm wrong, it's in the Render Settings. The result: a white-on-black matte of the building, rendered in the codec you selected in the Output Module.

It's just as easy to get fills: duplicate the original comp, and get rid of the stuff you don't want as fill.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index

Ross Newton
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 12:28:41 am

Hi Dave, thanks for the response.

I think you answered my question, that unless there's transparency in the comp, then there's no alpha. That's what I'm trying to basically do, add an alpha channel but not use it for transparency upon rendering out.

I know I can duplicate the comp and just render an alpha out of the building. I was trying to avoid that just to save time and see if this could be done, like other compositing programs.

It's basically like a shortcut for the compositor, even though we want the sky in the RGB of the shot, I wanted to include the building's alpha channel as a way of isolating the building, and to do this quickly in AE.


Return to posts index


Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 1:56:04 pm

[Ross Newton] " That's what I'm trying to basically do, add an alpha channel but not use it for transparency upon rendering out. "

From this, it sounds like you want to create an AE animation with the transparency built in. In other words, you put the animation above another layer in an editing application, and you see through to the other layer in spots.

Piece o'cake. Just select a codec that supports Alpha cvhannels. I'm a Mac guy, so I favor Quicktimes in Animation or PNG codecs. If I have Final Cut Pro 7 installed on my AE machine, I can also render in the ProRes 4444 codec.

In the Output Module, select RGB+Alpha, and Millions (or Trillions) of Colors+. The big choice: to create either a Straight or Premultiplied alpha channel. If you make both, you know one is certain to work.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index

Ross Newton
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 5:16:24 pm

Thanks again for the response Dave.

Actually, I think you're misunderstanding what I'm asking. I know how to render an alpha channel that represents the transparency of my comp using the appropriate render settings.

What I'm trying to do is render an alpha channel with my comp, that DOESN'T represent any transparency and therefor doesn't get matted in my comp and in the render. I want to "override" the alpha channel with what I choose.

In the above example, I have a CG building and "solid" sky in my comp, so in AE's world I have no alpha channel because it's completley blocked by the full-frame "solid-based" sky. HOWEVER, I want the CG building's alpha to "override" the alpha for my comp when I render it out. The PROBLEM is that, whenever I do a set channels or set matte on my comp, it always MATTES that alpha channel thus killing my sky layer and matting it to the background of the comp.

What I want is a FULL RGB render of my building and sky WITH the alpha channel set as the building's alpha channel... all in ONE render (yes, I know I can do it seperately). The reason I posted this is to see if I can do it in one render. Other compositing packages can do this with no problem, they don't automatically matte your comps based on your alpha channel.

Now, you're probably wondering, why would I want to do this if I'm already in the comping process. In this case, I'm actually building sub-comp passes for a senior compositor who wants to bring in the building, sky, and then use the CG building's alpha as a color-correction matte for the building. It just makes it easier if I can do all that in one pass.

That's the crux of my questions... can AE do this?



Return to posts index

Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 5:43:39 pm

[Ross Newton] "What I want is a FULL RGB render of my building and sky WITH the alpha channel set as the building's alpha channel... all in ONE render"

Oh, NOW I understand! I even understand why I misunderstood, too! Here's why:

You're trying to create a file containing a SELECTABLE alpha channel, and there are no codecs capable of doing this.

It can't be done. Sorry. It's time for Plan B.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index


Ross Newton
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 6:14:33 pm

Yes, I want to be able to "select" an alpha channel.. And there's layer effects to do this ("Set Channels, Set Matte, etc.) But you realize this wouldn't be at the codec or render level... you would set it up in the comp and then just render as "RGB + Alpha" to whatever filetype you want, same as any other render and filetype that can handle the extra alpha channel.

It's just like any other render with an alpha, it's just this alpha wouldn't equal transparency. The alpha channel is just another channel, it doesn't have to equal transparency, but it seems like AE forces this at the comp level.

I even played around with some of the channel effects like "Set Channels" and "Set Matte". No matter what, it always mattes the alpha through to the background color set for the comp. I was hoping there was a setting to turn this off. You see if it didn't do this, then I'd have my building, my sky, and my alpha channel. As it is now, the alpha knocks out my sky and renders through to the AE background color, which is set to black.


Return to posts index

Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Jul 15, 2010 at 9:20:38 pm

Geez, I got it all wrong. sorry.

Okay, if this is at the comp level, and NOT a rendered movie, it's possible: Nest your existing comp into a new comp. Copy the layer that provides the transparency (masked building, etc.) into the new comp. Use it as a track matte. On the lower layer -- the nested comp, that is -- you can then select "No Track Matte" or "Alpha Matte".

Does that do it for you?

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index

Jonathan Kempe
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Aug 26, 2010 at 8:17:49 pm

I am also having this problem!!

I have lots of rendered tiff sequences separated into passes from Maya. It looks like this. First a Beauty pass with the main object and a real alpha channel, then on top of that a Glow pass, Spec pass, SSS pass etc. Those passes doesn't come with a real alpha channel because they do not need a alpha since they are just black with lighter areas. And the lighter areas are all you want. SO, you use the multiply layer effect in the composition, the ADD (or Color dodge, screen etc). Voila! looks perfect against a background!

But it only works against a background, like a solid, a filmclipp or anything. BUT, if i want to export the Maya files with a alpha channel and not the background, its seems very hard. I tried to change to checkerboard background but I cant get rid of the Background. I really need to render just the 3D to make the workflow easier. (I have 186 layers in the comp right now...)


Return to posts index


Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Aug 26, 2010 at 8:25:24 pm

Then make your own alpha channels. It's easy:
  • Put each pass over black its own comp.
  • Add the beauty pass (with alpha channel) to each comp. Make it the top layer.
  • Use the alpha channel layer as a Track matte to create transparency in the lower layer. I think it would be an alpha inverted matte. You can play around with it until it's right.

Now, use each of these comps like it was the original footage in your main comp. Problem solved.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index

Jonathan Kempe
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Aug 26, 2010 at 9:08:59 pm

Thanks, I have tried that but it does not work at all unfortionally...

Since some of the multiplied layers like GLOW creates glow outside of the Beauty-pass with alpha I cant use the Beauty as a mask because the glow will not show since it is masked away. I have also tried to do the Beauty white and lighten the glow, also to make the spec pass lighter and try to mask with the luma-matte, but that is not possible since it destroys the gradients and everything looks really bad.

So, I dont really know how to solve this... I think it should be possible since After Effect for sure creates some kind of alpha-channel information to make it work when there is a background.


Return to posts index

Dave LaRonde
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Aug 26, 2010 at 10:08:34 pm

Then try getting Knoll Unmult. It should do the trick. I hope it's in a 64-bit version now for AE 10.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index


Jonathan Kempe
Re: How to Render Alpha But Not Apply As Matte In Comp!!!?
on Aug 27, 2010 at 10:42:53 am

THANKS A LOT!

This is exactly what I was looking for. I installed the full Knoll Light package, then just added the unmult effect to the separate layers (the multiplied layers with ADD, Screen etc. ) Then changed to checkerboard background and you can see the actual alpha.

Then... render.. then... success!

;)


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]