Gamut errors: what can cause them, and why are mine so inconsistent?
Hi all. I work at a post-production facility, and we've just finished two projects for our client. One has aired, and the second is being put to tape today, but one of the biggest pains we've had to deal with has been a variety of gamut errors. Luma gamut errors, RGB gamut errors, and even a random composite gamut error here and there. What can cause these to crop up, and why do we seem to be incapable of catching them on the first pass?
After some research I understand what a gamut error is, what it means, and why it's important to stamp them out, but what causes them to show up is what's eluding me.
First off: some background. The projects are edited from ProRes 422 HQ master material that have already been color corrected, with one project edited in Final Cut Pro 7 and the other in Avid Symphony Nitris. Both presented gamut issues when mastering to tape that were not present when playing directly into our Tektronix WVR5000 scope, or doing E-to-E playout through our D5 deck. And then, after recording is done, play-out will reveal a new set of errors. Over the course of several inserts and play-outs, new and heretofore unseen errors will crop up.
We've tried slapping a blanket limiter on our timelines and sequences, but the issue is that in order to get it to choke up high enough to net them all requires us to essentially wipe out our black levels and ruin any dynamic range we had. So far, this has meant that all we can do is several tedious passes back and forth until we have corrected each scene manually. Correcting the scenes isn't the big issue, it's the fact that we're not catching all the errors in a single pass, and it's not consistent between how the video is getting into the scope.
Another issue is cross-converting from 1080p to 720p (our client requires versions to be submitted in both formats). Everything is sourced in 1080p23.98, and edited at the same resolution. We have had problems with letting our Kona 3 or Nitris DX boxes handle the downconverting and pull-down, causing issues like aliasing and motion artifacts, so we're relying on our D5 deck to handle it (the results are superior to either of the boxes, and it's results have never been rejected by our client). However, in bringing the material in from the deck we find an entirely new series of errors come up. For this one, however, I have a theory: it has to do with the fact that we're downscaling 4:2:2 material, and the chroma subsampling is causing a few lines of video to change dramatically enough to throw an alarm. Is there any sane way for me to prevent this from happening?