Remove orange tracking markers from green screen
I've got a shot with actors in front of a green screen with orange tracking markers applied to it. There's a lot of camera movement in the scene and unfortunately, there's a couple of frames where the tracking markers pass behind the actors due to the camera movement.
How do I remove these tracking markers without destroying edge detail on the actors when these markers come in contact with their outline?
[example image below]
You could use the Lumetri Color/ Secondary Color Corrector to change the orange to green.
Good idea, but it the orange seems pretty close to the skin tone/clothing color of the actor on the right. Probably best to just do it manually with masks. How much time it will take depends on the amount of movement in the shot and with the actors.
thats why they are supposed to be blue stickers isnt it?
Yes I definitely agree with Dave that this is a poorly lit and uneven screen, and that's probably going to cause the greatest problems.
ON THE SUBJECT OF TRACKING MARKERS:
Unfortunately productions all too often shoot without a dedicated VFX supervisor on set, and "wing it" expecting to "fix it in post". And when it comes to tracking markers and green/blue/black screen there is so much misinformation it is shocking.
GREEN SCREEN EXPOSURE
I'm mostly going to talk about tracking markers, but just quickly, green screen is ideally "under exposed" by as much as a stop and a half. But more important is that the screen be very even in lighting, within a quarter of a stop, particularly around the subject/action.
In the example the OP posted, the screen exposure is nearly a stop between the darkest and lightest area. As a matter of reference, the more ideal exposure is the darker area on the far right of the screen. Unwinding the image into linear light we can estimate the actual exposure. In the info panel, we see the values for the darker area have good separation between channels.
The green channel is at 22%, while the red channel is 2.3%, more than three stops difference. The blue channel is a distant 0.8%, more than 4 stops difference. (this will help us in a minute). Meanwhile in the more over exposed areas of the screen, we have values like R 10%, G 35%, B 3%. Here, red is less than two stops away from green. This is a reduction in saturation which happens as the image gets "brighter". Since green makes up 71% of luminance in the Rec709 colorspace, it looses saturation sooner than the other two colors, and the way to prevent this is to keep the exposure down (a spot meter reading at the same level as an 18% grey card should be in the ball park).
The Quick Cheatsheet:
FORMAT: Shoot 444 and avoid 422 whenever possible.
I am not going to defend orange tracking marks on a green screen, but just so everyone is aware, there is a rational behind them. (and again, there are better choices so I mostly disagree with their use in this day and age).
Orange is made up of Red and Green. As a result if you choose the proper orange tape, in the green channel of the image the markers will not be seen at all. But they will provide high contrast in the Red channel for tracking purposes.
However, this means you are relying on the red channel only for tracking information — and red (and blue) are at half the resolution of green at the bayer filter on the sensor. And also, it means that you need to key using blue and green unless you use masks for red (see below for that solution), but a problem here is that blue is the darkest and noisiest channel. For both tracking and keying, it is ideal to have all the channels at your disposal and at full resolution.
Here are the separated image channels:
As you can see the orange marks are barely apparent in the green and blue channels, while being very high contrast in the red.
Now Filip, we could all bash this green screen all day, but I'm assuming you'd like a solution. Here it is:
You're going to be working more with the individual color channels than the RGB image. For tracking, it's not really necessary to make a monochrome video using just the red channel. Use the full RGB image. Because the screen is so uneven there are actually a lot of natural tracking "marks", and you might as well let the tracker use them. If you do have problems with tracks slipping, then you might consider using the red channel to create a monochrome video just for tracking purposes. If noise is a problem, try turning of the BLUE channel first.
HOW TO KEY THIS:
Set the colorspace to LINEARIZE.
For keying, if you don't want to use mocha and make track masks, then just use the green and blue channels, and turn the red channel off, as so:
And also set a Levels Effect and increase the brightness of the BLUE channel by sliding the INPUT WHITE control to the top (highlights) of the blue histogram. It's okay if you clip the foreground subject slightly in the blue channel - the green/blue image is going to be used to create a matte ONLY. Your synthetic image will look like this:
Then sample the green in the keying plugin, and and looking at the MATTE image, adjust the key for a clean solid matte. Then set they keyer to "final result" or "Comp" so that the alpha is live and ready for use.
Immediately UNDER that layer, put your original footage, and then set it to TRKMAT ALPHA. Presto, your footage is keyed. In the keyer, adjust the softness and shape of the edge. In the original footage layer, add a levels and perhaps a spill corrector if needed. Then on the layers beneath, add in your background elements. I did this successfully using your still image from your post (I'll show more below) not sure what issues there will be for full motion though as it looks like it might be noisy.
OTHER KINDS OF TRACKERS
I'd like to cover other tracker types. There is no single type that is best for all situations, they all have various pros and cons. Same color, B&W, BlueonGreen ... But even though this has nothing to do with the shot, since were talking about trackers, let me just say one thing again,
...one more time for all the production people in the back row that didn't get the memo...
............For The Love Of God..........
---------- STOP PUTTING TAPE MARKERS ON THE FRONT OF MONITORS USED IN THE SHOT!!! ------
S T O P P P P P PPPPP!
We like never ever need markers placed on the monitor surface. See those corners on the monitor? THOSE make best marks for us. When markers are placed on the monitor screen, we then have to waste a lot of time painting them all out before we can even comp something into the screen, and often losing the natural reflections that would have helped make the comp more authentic.
And generally we DO NOT need tracking markers all over the place. Modern tracking software can do a lot with the natural geometry of the surroundings. That said, tracking markers are needed in some cases, such as a nice evenly lit blue or green screen. And how to do this and be able to use the key with minimal roto?
One way is to use markers that the foreground subjects never cross in front of. In these cases a black & white marker might be ideal as it uses all color channels. Otherwise you can use markers that are of the same color but a different luminance. For green this means the darkest part of the tracking mark geometry should be about 1 1/2 stops underexposed, and the brighter part should be half a stop to no more than a stop brighter.
But you can also use blue markers on green (or vice versa) and this has an advantage over orange. First, it leaves the red channel clean for use in creating the key, and second you can ADD the blue and green channels together, creating a clean synthetic channel for use in the key. And third, it means you can use both the blue and green channels for tracking, and as green is the highest resolution and lowest noise channel, you want to be tracking with green.
HERE'S HOW BLUE ON GREEN MARKERS WORK (simulated obviously).
Here I covered the orange marks with blue (Note: because the orange ones were crosses so are these, BUT I do NOT approve of using Xs or crosses as track marks. Round dots are better to track and easier to paint out. Some software might prefer other marker geometry though, so ask you friendly VFX supervisor before you shoot regarding best geometry and mark placement).
Now here's how that looks in channels:
The marks are visible in green and blue, but red is clean. REMEMBER: if we only get two channels to key with we prefer RED & GREEN (or RED & BLUE) this is because red is a principle color in humans (our most common subject) and because red makes up three times more luminance than blue. As a result there's more "meat" in red to get a good key with red & green than with blue & green. Of course it is always preferable to have all three RGB channels, and each at full resolution (avoiding 422).
So what's next here with blue on green trackers? The first step is to take the BLUE channel and ADD it to the green channel. then turn the blue channel OFF. When you add it to green, adjust the blue levels and gamma so the that tracking marks BLEND IN with the green channel. You should end up with a synthetic image before keying with only red and green channels, like this:
You may want add a levels effect just before the keyer and adjust the red gamma (optional/as needed).
Then sample the key color, and as above with orange markers, just make a solid matte.
And on the layer immediately beneath, put in your original footage and set to TRKMAT ALPHA. In the keyer, set it to "Final Comp" so that it generates the alpha you are going to use.
On the original footage layer, add a spill suppressor if needed, and levels to make final color adjustments. And beneath that, put the layer of the BG to comp into.
And that about sums it up I think. Good luck!
VFX & Title Supervisor
This may be the most thorough and helpful post I've ever seen on this site. You should submit this as an article to ProVideoCoalition, or something, because it's invaluable information.
Wow, Andrew... THANK YOU! I can't even express my gratitude that you took the time to give me such a thoughtful and detailed answer. Again: THANK YOU! I do really appreciate it!
You're welcome Pilip - Creative Cow comes up high in Google searches, so I figured it was worth it to address a lot of the comments all at once. ;)
VFX & Title Supervisor