AE renders EXRs darker?
I'm rendering EXRs out of Cinema 4d and compositing them in AE. Everything is looking as expected in the viewport, but once the comp has been rendered, the resulting Quicktime is noticeably darker.
I'm outputting to Prores, but I get the same dark results from the animation codec.
Changing the color space doesn't seem to have any effect - the renders always turn out darker than the image in the viewport.
Changing bits per channel doesn't solve the problem either.
When importing the render back into the AE project for comparison, the qt is clearly darker than the comp.
HOWEVER! Exporting the comp from AE to Media Encoder works just fine!
I can live with the Media Encoder work around, but this issue has been driving me nuts. Am I missing something simple? I'm pretty sure the issue is related to the EXR format, somehow...
I am using a 2010 Mac Pro, CC 2018.
I have received an unbelievable amount of help from the forums here, but this is my first post.
Thanks a whole bunch!
If you are using 2018, can you really output to a Quicktime? Many of us are lamenting leaving those codecs behind with the latest rev of AE. Whats your secret? Or are you compressing the files out of AE - that could be part (but only part) of the problem.
But, on to your problem:
Check the color profile of the exr's on their way out of C4D and check your project's color profile in AE.
Its funny, I was going to post about this the other day (but never got around to it) after a reinstall of C4D, warning the members about EXR's defaulting to sRGB in C4D's render engine. Given EXR's high color-spacey-goodness it seems silly to limit the files to a decades old and very limiting color space.
When you encode your files with a compressor two things are happening: you are clamping many of the colors with a codec (like H264), so that will often "up" the black levels and bring colors close to white right up to full white etc, and, you are no longer looking at the files in AE. With a color profile set in AE, your screen is being fed the files with the colors converted for that profile and may not be letting them through with their original gamut.
There have been some great posts on this forum about color profiles and file interpretation. It can be tedious reading, controversial, and a highly heated topic, but its worth the work so that you can feel confident that what you are seeing in AE is what you are outputting. Its always better to know that a color meets legal standards or is "right" on your system so that you can say to a client with confidence: "Those are the right levels, the other guys at the broadcast house are wrong"
Thank you Steve!
I think you've solved my problem! And actually, it's the first thing you said.
I'm not sure why I still have access to quicktime in this version of (15.1.1) AE . If it's magic, it's DARK magic, as quicktime seems to be the culprit in this case. After receiving your message, I went to work and did a whole bunch of tests - the thing I should have done to begin with. I was incorrect is suspecting EXRs. The "darkening" was happening with everything - including a gradient ramp generated in AE. As long as I was rendering to quicktime, the results turned out dark. Finally I tried rendering a TIFF sequence and the images were perfect. Rendering anything other than quicktime yielded the expected images. Problem solved. Thank you!
By the way, I see now that I failed to adequately describe what I meant by "darker." Here's a better description. The renders were coming out with the darks slightly crushed. Shifting the gamma by .05 or so comes close to the bad effect. The light values were hardly being effected at all. Mostly, it's that the blacks were being crushed.
I have definitely read most of the talk about color space on the forums here with all of the controversies and debates and would say I'm at about 80 percent or so comprehension of the basics. I'll keep trying for sure! (Don't think I'll ever have the confidence to say the broadcast folks are wrong, but I'll work on it...)
Thanks again for your help!
You might be on the cusp there. I use 15.03 and I still have full access to the QT codecs, but when I went up to >15.1, they vanished. But I don't know what point rev that was. So I backed off and went back to 15.03.
While Qt is only 8 bit it was such a sweet format for mattes etc because the Animation Best version was RLE and as such, tiny tiny tiny. It was also such a great format to throw around, edit, trim and export in the player, multiplatform compatible, scrubbable, loopable and cha cha in the player, had a good info dialog.
And now that I say all that, with QT being only 8 bit and RGB only (I believe) you might be seeing some LUT conversion from your much expanded and possibly log EXR's. You just can't represent all that was in the EXR's in QT so something's got to give. A good LUT can get you close but its' never perfect, especially in the heel and toe of the blacks and whites curves.
Well, somehow I still wasn't sleeping well at night.
Here's a couple critical pieces of info regarding this case:
AE is discontinuing ONLY some of the old QT 7 codecs. The Prores codecs are still being supported. That explains why I am still able to render QT movies with Prores out of AE 15.1.1. But then, the question remains: Why is AE crushing my blacks when I render using QT?
I just relinstalled AE 14 and loaded up the scene that AE 15 was "darkening." AE 14 rendered it perfectly. The rendered quicktime movie MATCHED what I was seeing in the viewport of my project comp.
The obvious conclusion is that whatever voodoo Adobe did to QT that caused them to remove all of the old qt 7 codecs is also somehow how flavoring (crushing the blacks) of all my QT exports. But the fact that I scoured the web and couldn't find anyone else complaining about this still makes me feel that the problem resides on my own system. I suppose it's time to reach out to Adobe and see if they have any idea whats going on...
Most of my stuff is on PC's so the ProRes pipeline gets messy in that world, but yes I assumed that it was still supported.
My understanding of the whole mess is that QT is not being developed anymore and as such can't be made compatible with the higher bit-depth-color environment inside AE. Which I totally understand, but it's the way Adobe implemented it without warning that has us all fuming. What are people to do with legacy projects?
In reality its probably more the Adobe/Apple pssing match.
Its kind of like when we all had to bite the bullet and embrace 64 bit architecture on the hardware side and OSX appeared on the scene - the main reason I jumped ship from the mac platform (sniff, OS9 I miss you!). It was necessary pain to move forward with faster architecture. Rip the band-aid off and keep going!
But given all of that and the odd way FCP developed and then exploded, I'm wondering if there is something that isn't totally 16 or 32 bit clean about the ProRes codec, or, if an older codec is installed on your system which shows up as ProRes but isn't the latest version that others might be using who we've not hear complaining.
Usually these black issues are color space, or monitor gamma, or color management issues. (what are you working in with AE BTW?) It can also be a YUV vs RGB issue.
Just for a test try rendering in PNG in C4D and then work in 8 bit with ProRes in AE and see if the problem shows up. If not then it's the higher bit depths that are not translating well.
Does it have to be ProRes? What about BlackMagic?
I completely understand why the community is upset. I definitely cut my teeth on some of those codecs so they hold a lot of nostalgia for me. As for "jumping ship," I am with you on that too. I very much want to switch to PC, but thus far I have been too much of a coward.
But on to the important news. I talked with the folks over at Adobe and they are logging in my problem as a bug! I was suddenly filled with joy - as if I had discovered an uncharted island, or a new plant variety. In all of my shortsightedness, though, I forgot to ask the tech guy if they would please name this bug after me.
To answer your question, I'm an animator and compositing artist. I'm currently I'm working on a short personal film, which is how I ran into my "darkening" issue.
Thank you again for your help, Steve. I appreciate your time and attention to detail. Your suggestions definitely got my problem solving juices flowing.
It always feel great when Adobe confirms it's not just you.
Here's another reason to consider the "jump". We just took delivery of a new monster machine: 256 GB of Ram, 32 cores each @ 4ghz and an unspeakable expensive graphics card. The PC was cheaper than the graphics card, but a Mac (as close as we could get to that config but still falling short) was going to be 3 times the price. Maybe if they cut back on the titanium in the case? While this isn't a usual example, the cost difference does hold to 2-3 times higher for the Mac equiv across the product line. My $3000 tricked out PC laptop compared spec-wise to a $6000 Macbook Pro.
The only things I miss from the Mac days is the way the mouse moved, and the file structure control (the way the file structure appears to you on a PC changes depending on whether you are inside an app, outside an app, across the network, looking at it from a windows tool, from a shell, saving, or opening a file. I guess I shouldn't expect too much from a company that can't even keep the right click menu consistent throughout the day.) In some ways Mac wanting control over the entire user experience was beneficial, but in other ways, very limiting.
I will most certainly be coming back to the cow for advice when it's (finally) time to upgrade my rig.
I remember over a decade ago this being a problem with Quicktime and Final Cut Pro, as Apple was using different gammas for output and display.
The problem here sounds like a gamma curve problem - as in not being applied, or being applied incorrectly.
EXRs are linear (gamma 1.0), but a prores video needs to be somewhere around 1.9/2.0 if it's scene referred Rec709.
1) Can you screenshot your project color settings?
I've grown a bit tired of Adobe's Color Mangelment so for color critical situations I use Preserve RGB in the output module, and then use an adjustment layer on top of the output comp to do the conversion from linear 32bit to whatever colorspace I need to deliver. This affords total control of the output conversion, as well as allowing checking/adjusting through a view LUT, set as a guide layer above the conversion layer.
VFX & Title Supervisor
Thanks for your interest in my problem. You guys are super helpful!
As it stands, I'm calling this "darkening" issue resolved.
I mentioned this in one of my long winded posts, but it turns out that this was not an EXR problem at all. I suspected EXRs at first. But actually, the "darkening" was happening regardless of the source. The "darkening" occurred on prores footage, photos, and even a gradient ramp generated from inside AE.
Anyways, there are plenty of work-arounds here including exporting to media encoder, rendering an image sequence, or rendering from AE 14. All of those methods yield clean results. The "darkening" I experienced only occurred for me when rendering a QT (any codec) from AE 15.1.1 and 15.1.2. And the issue has been logged as a bug with adobe.
15.11 is where the legacy QT codecs met their maker. I wonder if, with the remaining but still long-in-the-tooth QT codecs, AE is forced to emulate ProRes in 10 or 12 bits, but from a 32 bit universe. And as such, can't quite get the conversion LUT just right.
I don't remember if ProRes was 10 and 12 bit linear or Log. The latter would complexify that conversion exponentially.
AE is forced to emulate ProRes in 10 or 12 bits, but from a 32 bit universe. And as such, can't quite get the conversion LUT just right.Hi Steve,
Most forms of ProRes are either 8 or 10 bits, typically with a video gamma, or sometimes LOG*, but never linear**. ProRes 4444 is 12 bit.
For instance ProRes as it comes out of an Arri Alexa is typically LOG-C, which is a slightly different curve than Cineon LOG, but still "more log than gamma." ProRes *can* be encoded with any curve you like***, some flavor of log or some flavor of gamma encoding. The transfer curve is a function of the color profile or LUT used when outputting, not the codec itself.
I almost always work in 32 bit linear (gamma 1.0) space inside after effects. Some of the issues that can come up working in linear in AE are partly due to faulty color profiles at various points in the pipeline, including the monitor. If you are working in linear, it is critical to have your monitor both properly calibrated AND PROFILED, so that AE can use your calibrated monitor profile to correctly display images using "View > Use Display Color Management".
I'm not sure what you meant by "emulate" ProRes — you mean convert from a linear (no gamma) to a gamma encoded form? AE does not use "LUTs" for color management, it is based around ICC profiles****. For color management to work, everything has to be profiled. The issue has nothing at all to do about bit depths — if we were only converting between 10 or 12 bit integer and 32 bit float, the math is trivial. The issue is converting from a flat transfer curve in linear space, to a non-linear space such as video which has a gamma curve, or a film intermediate which typically has a log transfer curve. And here we rely on the profiles.
If you don't have an accurate monitor profile, what you see inside After Effects working in linearized mode may end up looking very different on output. When outputting from the linear (no gamma curve) environment to video, which has a gamma curve, the image needs to be transformed. In AE that transformation usually happens with an ICC profile.
But I've found that those transformations in the output module may come with issues, some are due to how AE handles the linear environment, and some are due to problems with the ICC profiles themselves. For instance in Adobe's version of the Rec709 profile, the gamma curve is wrong, and no different than the sRGB profile (a simple 0.45 for 2.2 display curve). See this article on Rec709 profiles.
As I mentioned in the previous post, I now avoid using the color management in the output module, and instead set the output to "Preserve RGB" and use an adjustment layer for color conversions. This is of course the required workflow when using 3D LUTs on output, but I find it useful for all outputs instead of relying on CMM. I discuss the reasons in depth in this article.
* Footnote 1: the reason "LOG" is around is because a film negative density happens to be logarithmic in nature. When Kodak developed Cineon, they were scanning film negative straight into a digital form, and so that digital representation is log due to the fact that it is a scan of negative film densities, which are log. Gamma curves date back to CRT type television systems, and survive today because perceptually we see greater detail in darker areas of an image, and gamma encoding provides greater luminance resolution (more data) in dark areas.
** Footnote 2: "video" such as Rec709 is gamma encoded, so it is not linear. Linear means a gamma of 1.0 which is a straight line (linear is also the nature of light in the real world). Most video is encoded with a gamma curve close to 0.45, which unwinds into linear "light" when displayed with a monitor at a gamma around 2.2 to 2.4. So essentially, there is LOG, LIN, and VID. I discuss the point in more detail here:
And Stu Maschwitz goes into even greater detail here:
*** Footnote 3: Linear (gamma 1.0) will not fit well into an integer storage format, and needs to be stored in a floating point format such as EXR. Integer formats such as ProRes, JPEG, etc. need the image to be encoded with a curve in order to fit a useful dynamic range into the container.
**** Footnote 4: While you can use 3D LUTs in After Effects with a LUT plugin such as OpencolorIO, that is not part of the "color management" system. The color management uses ICC profiles which are a little different in form and purpose. Some ICC profiles may contain LUTs, such as those intended for output. But the idea behind ICC Color Management is that everything gets converted to a neutral space such as CIE L*a*b*, and then converted to some other working or destination space. 3D LUTs as used in the film industry map directly from one color space to another, frequently to emulate a particular device or output type, e.g. to make a digital projector look like a film release print.
VFX & Title Supervisor