You say you've looked at the Keylight Status viewer, and you're convinced the key ought to work? The Status view shows white all over the subject and gray at the edges?
You've got a good key. The only problem: it looks like a certain disgusting, stinky brown substance. Here's what I do:
• Duplicate the keyed footage, and turn off the upper layer.
• Fool with the lower layer's Keylight settings until it looks like the subject looks fine, but is over a grayish, washed-out chroma key background. It'll take some doing, but you will have gotten it to the point where Keylight's spill suppressor is working. That's good, because the Keylight spill suppressor works darned well.
• Open the Modes panel, and use the upper layer as an Alpha Matte for the lower layer. For more info, look up Track Mattes in AE Help.
Poof! You're done! You have a clean key, AND you have spill suppression.
So why does this happen? Your chroma key background's level is probably too high for Keylight. Keylight likes less light on the background than other keyers. The usual admonitions about even lighting and the distance of the subject from the background apply.
Access to a waveform monitor can be a GREAT help. If the brightest highlights on the subject come in at 100 IRE units, the background should be about 45-50 IRE units. The waveform of the green screen should be a thin, level line at 45-50: the thinness of the line indicates even lighting top to bottom, and a level line indicates even lighting side to side.
You can accomplish the same thing by measuring footcandles with an incident light meter. Even though it's much more time-consuming, it can really help when the time comes to key.
Now, if this footage is DV, your fun is just beginning. The color resolution (aka color sampling) of DV video is about as bad as it gets. Oh, it's good enough to fool the human eye, but computers are a LOT more discriminating, and DV's stinky color resolution poses real challenges for chroma keying.
If you've never tried keying DV video before, I recommend watching a podcast by Alex Lindsay, the founder of DV Garage and Pixel Corps, called "The Road to 1080p Part 2". In it you will find an excellent description of the differences between 4-4-4, 4-2-2 and 4-1-1 color resolution, and you will come to understand the challenges of keying DV. It's WELL worth the 15 minutes it takes to watch.
You can get the podcast here:
http://macbreak.libsyn.com/index.php?post_id=70596
As you watch if, keep in mind that Alex Lindsay runs a company that makes DV chroma keying plugins... and he STILL doesn't recommend trying to chroma key DV.
Dave LaRonde
Sr. Promotion Producer
KCRG-TV