I'm working on something that I'm sure must be easy, but all my Googling and searching the forums has turned up nothing. Here's my deal.
I'm working on a green screen shot where the talent is doing a controlled fall in a harness in front of a green screen. The screen wasn't wide enough, and for about 40 frames his hands go beyond the edge and need to be roto'ed out. In addition, I need to do a simple wire removal of the harness wires. The green screen looks great and it's easy to pull a solid key off of it, so I'm just rotoscoping around his hands where they pass the screen, not the entire actor. I used the Rotobrush tool and it worked great for creating that. But the harness wires proved to be too thin and the rotobrush tool just couldn't "lock onto" them, it was turning into a mess.
It wasn't a big deal, the wires would be a snap to mask out. So I "froze" the rotobrush and created some masks with the pen tool... but I can't seem to get the masks to show up inside of the rotobrushed footage. The points show up and the colored "frame" is there defining the mask, but it does not mask out the wires. I've tried inverting the mask, changing the mode (add, subtract, difference, etc.) and nothing seems to work. However, when I toggle the "rotobrush" effect off the masks behave as you'd expect them too.
I realize it's a given that there's a way to use the rotobrush in conjunction with masks, but I can't figure out what I'm doing wrong. Any help would be appreciated.
What's probably happening is that some effects disregard the masks on a layer and directly access the source footage.
The simplest solution would be to precompose the rotobrushed layer and apply the masks on top of the precomped layer.
Or duplicate your footage, remove the rotobrush from the top layer, apply your masks there and use it as a track matte for the bottom layer, by setting the track matte to alpha or alpha inverted. This is probably a more convoluted way of working but sometimes it's useful to have everything in a single composition.