We are rendering Flash animation composited in AFX out as DNxHD as we have always done. With the latest artwork we are noticing visible at 100% jagged edges particularly on diagonals where colours meet.
The source in AFX looks smooth:
And the output DNxHD at 1080p/25 185 10-bit full RGB looks jagged:
The above is at 100% and the jagged edge is noticeable in both the diagonal red edge and grey line.
Changing bit depths, color spaces, Color Levels, Millions vs Trillions of colors, does not improve the output.
For our pipeline DNxHD is very very convenient so I am trying to avoid switching to DPX sequence output.
After effects 2014.2 Release
Avid Codecs LE 2.6
Project color depth: 16bit / 32bit / 8bit no improvement
DNxHD settings: 8bit / 10bit no improvement
DNxHD settings: RGB / Rec 709 no improvement
DNxHD settings: 185M / 220M no improvement
Output: Millions/Trillions of colors no improvement
The only way I've found to fix it is to tweak the colors.
It seems to be due to yuv color space and compression and it usually happens when two colors with nearly the same luminance but very different chrominance and saturation meet. With red and black I see this quite often with prores 422. To fix it I usually shift the black towards red a bit and desaturate the red until the jaggies go away.
Thanks Kevin and Walter for your responses, chroma-subsampling does make sense but I am not sure how we can work around it.
I have run a few more tests and indeed the issue is not there going to DNxHD 444 or Animation codec.
Using Cineform 422 codec instead gives a blurring of the edge which looks way more acceptable than the jagged results.
Interestingly Media Encoder conversion to DNxHD mxf gives better results than DNxHD quicktime.
Ultimately the project will be delivered as 422 (probably as prores) so we can explore keeping it at 444 until final delivery but its not ideal as we use ffmpeg and ffmbc conversions which do not support DNxHD 444.
To color correct the issue is there any automatic levels that can be applied or must it be done by hand scene by scene?