Color space problems again
After much duress, I solved my color space problems by not specifying any color space. My mp4s look good enough on most browsers and operating systems. My understanding is they just use a generic sRGB profile to display the mp4. I even bought a generic monitor and used the default setting while creating the videos which uses just a generic sRGB profile I think. Anyway, I was very happy. But my streaming encoder had different plans. I'm not entirely sure but I think it is assuming that my unspecified color space is rec601. If I specified it as sRGB, it would probably encode fine.
So that's my problem. How do I take my lossless animation .movs (which I use to create my mp4s) and which needless to say do not specify a color space (in Adobe Bridge Metadata, it says the color space is untagged), how do I take my .movs and just gently tag the color space to sRGB? (Rec709 might be a good choice also that my streaming encoder might accept.) I'm happy to use ffmpeg if somebody knows how. I use it for other things, so I'm perfectly comfortable with that. Hell, at this point, I'd buy some expensive software. HELP!!
Thanks in advance
JERRY SAID: My understanding is they just use a generic sRGB profile to display the mp4
They don't use any profile AT ALL.
A "profile" implies color management, and video is normally NOT color managed. Video is normally "rendered with a specific display target".
What is color management: Color management is a process where ICC color profiles are used to convert image date from one color space to another color space, using either a "connection color space" or a "device link".
What is rendering for a specific target: This means the image data is rendered in such a way that when played back through a specific target device without further manipulation, it looks as intended. A "specific target device" means an sRGB monitor, or a Rec886 monitor, etc.
When you work with color management turned off, then you are de-facto working to render to a specific target — this means that your display monitor must be of the same target that is your render intent. You chose an sRGB monitor, so that *particular* sRGB monitor becomes your render target. Ideally you have used a hardware profiling device to create a system profile specific for that monitor so that it is as close to the baseline sRGB target as possible.
After Effects default for import of non-tagged images is sRGB. This can be changed in the "interpretations rules.txt" file.
Some other things I want to clear up: Just because your animation movs are untagged does NOT mean they have no color space — they certainly do!! They have a gamma curve and look the way they do based on the color primaries of your target.
Also: sRGB and Rec709 have identical whitepoint and color primaries. The ONLY difference is the transfer curve (aka gamma). Rec601 and Rec709 have identical transfer curves and whitepoint, but they have very slightly different color primaries (slight enough that it is common for Rec601 to simply be interpreted as Rec709).
As for your streaming encoder assuming any particular profile: is it CHANGING the look on output? What encoder are you using?
VFX & Title Supervisor
The encoder is the Azure Standard Encoder. *What's really annoying is that even if I skip encoding and just add a manifest to an mp4, it changes the color. Support will contact me today.*
I gave up on profiling and just did a lot of device testing. We're only interested in playback in browsers. I looked at the results in many different browsers. I went to the store and looked for the "median" monitor. The one that was most similar to all the others.
It probably is a gamma issue.
My question is about exporting. My hope was that if I put mp4s with encoded color profiles into Azure then all would work. But I cannot find any practicable way to encode color profiles in outputs except for tiff sequences that have no audio and require God knows what workaround.
In Bridge, when I look at the Metadata, there does not seem to be anyway to get a color profile to appear. And research on the internets indicates that no Adobe product (AE, Pr, AME) actually delivers outputs with embedded profiles for sRGB or Rec709 or for animation. Maybe audioless sequences are the exception.
Is there a way to export with a color profile?
If I could export with media tagged with sRGB, then that might make Azure happy.
If I could export with media converted to Rec709 and tagged with 709, then that might make Azure happy.
(I have a related question about Adobe, but I don't want to over complicate the discussion.)
In general, their are two tendencies in these color discussion. The rationalists push for deep understanding of one of the most arcane subjects in media. The empiricists push for lots of testing. I'm firmly in the latter camp. I just need to get things working. Even if I understood it all, I'd forget it in a week.
How can I get color space tags on my videos? That is the only question that will allow me to figure out what Azure wants. Adobe really seems useless. Am I wrong? If not, then can I do it with FFMPEG?
Thanks Andrew for taking the time.
Andrew, I just noticed your response to the older post.
I'm not interested in different browsers/computers. I can limit myself to just one of my computers. Take your pick. If I play the video locally through my browser, it looks one way. If I instead stream or progressively download it from Azure, then it looks another way. And I as I said above, this happens even if I bypass Azure's encoding.
There is a tiny chance that my online player, Bitmovin, is tweaking gamma, but they don't seem to discuss color spaces at all, so I tend to doubt they are responsible. Most likely, Azure is assuming my video is Rec709 when it is really sRGB. Somehow I have to embed a profile. But Adobe seems to give me now way to embed a profile.
This all seems quite absurd. Why can't I just add the metadata?? Oh, by the way, this is sRGB!! Or convert it and then say, oh, by the way this is Rec709? Adobe has a zillion color space settings for import and working, but very little for outputs.
If I could tag my videos, then I could do some experiments (empiricist that I am) with Azure and probably not even have to waste anybody's time.
To repeat: How on earth can I embed the color space? Surely, somebody, somewhere can do it. I think my question is very very clear.
I used ffprobe on the mp4 that I get from Adobe and it says something about rec709. But I changed the matrix from 709 to 601 and voila, the stream looks good.
So, it seems it some sort of gamma issue with Azure streaming.
But oddly if I convert to HLS locally and upload that to dumb storage, then the videos are again too dark.
So it seems there are gamma problems in other places also.
At least I have a workaround (#985). ;)
Although sRGB and rec709 are close enough, gamma 2.2 and gamma 2.4 is not
At some point AE shifted to gamma 2.4 from 2.2 to match Premiere but now has gone back to an additional menu in Project settings for Gamma. Now when working in Unmanaged Color Workspace you have the option for Assume Gamma 2.2 or Gamma 2.4.
Your computer monitor is likely sRGB gamma 2.2 (therefore your browser and rest of what is on your computer) and your TV is likely rec709 gamma 2.4.
Jerry's issue has to do with his workflow and encoders, and if I recall some interpretation settings, and I believe he is using color management.
Pierre said: Although sRGB and rec709 are close enough,
Rec709 and sRGB use identical color primaries and white point. Other than the transfer curve, they are not just "close enough", the primaries are interchangeably identical (if working in a linearized space, there is no difference in choosing sRGB or Rec709 as far as the working space is concerned — the only difference is in the output module when rendering to a gamma-encoded file format).
Pierre said: gamma 2.2 and gamma 2.4 is not
Just to be pedantic: sRGB (file encoding) is not gamma 2.2, but a parametric curve. Some (badly made) sRGB ICC profiles may use a simple 2.2 gamma curve, but the correct transfer curve is actually a gamma curve of 2.4, offset by a linear section near black, that results in approximating a 2.2 curve. The linearized section is to allow the curve to be inverted using integer math (computationally less expensive than float).
Rec709 is never gamma 2.4,( though there is much confusion on this point.) Rec709 defines the source/file encoding and Rec1886 defines the DISPLAY, and Rec2035 defines the VIEWING environment.
Rec709 is *also* a parametric curve that approximates a 1.9-2.0 gamma curve. Here the linearized section was intended to help with noise in the black areas of camera signals of the day (remember the spec was developed circa 1990, well before digital HDTV emerged).
But these show the curve as encoded in the signal or file — NOT in the DISPLAY itself. The Rec709 curve is "source referred" and even though the sRGB curve is called "display referred",NEITHER curve defines the actual display gamma of the monitor in any particular instance.
The sRGB curve for the file encoding is parametric, but the actual monitor spec is a simple gamma 2.2 curve.
Using log intensity in this graph to make it more apparent, note the difference between the sRGB curve (file encoding) and a simple 2.2 gamma curve (Idealized display).
But in addition, displays are very user adjustable. Some even have a control labeled gamma, but most or all have BRIGHTNESS (gain) and CONTRAST (lightness/luminance) – and unless a hardware calibration and profiling is done, they could have been set to an "equivalent" gamma from 2.1 to 2.5 or more.
(Also, remember that legacy CRT displays were typically a gamma of 2.4 — see "Digital Video and HD" by Poynton)
In the case of Rec709, the 1.9~2.0 signal/file gamma is fed into a monitor with an EOTF display gamma of 2.4, per Rec1886. Rec709 defines ONLY the SIGNAL or FILE gamma. Rec1886 defines the MONITOR gamma.
DON'T FORGET THE GAMMA GAIN
Both sRGB and Rec709 have a "system gamma" that increases the gamma by the time it is displayed. For sRGB it's a power of about 1.1, and it's a power of about 1.2 for Rec709.
Side Side Note:
DON'T FORGET THE ICC PROFILE
If you are working in a color managed system, then the actual physical gamma of the display is less relevant or simply irrelevant. Assuming the display is profiled with a hardware colorimeter, the effective system gamma will be whatever the CMM adjusts it to, based on various settings.
Pierre said: Your computer monitor is likely sRGB gamma 2.2 and your TV is likely rec709 gamma 2.4.
The monitor and the TV are likely all over the place. Both have easy to access user controls that can shift the effective or actual gamma all over as a typical user adjusts for "what they like" for any given environment. Unless they are adjusted with hardware (a colorimeter), there is nothing to enforce a 2.2 or 2.4 gamma.
And in fact, unless the monitor is in a calibrated viewing environment (such as Rec2035), then the specified gamma of sRGB (or Rec1886) may in fact be wrong in terms of creating the same perceptual uniformity. In fact the reason for the increased gamma in the system is because the expected viewing environment is dimmer than the environment while the image is being captured with a camera. The brightness of the viewing environment critically changes the perception of the displayed image.
Rec 709 related display and room specs:
Rec 709 FILE/SIGNAL:
sRGB related display and room specs:
Notice the substantial difference in ambient light for the Rec2035 view environment (modeled after a dark living room) vs the sRGB view environment (modeled after a typical office environment).
The point to this long winded post is simply to indicate the importance that the FILE encoding is NOT intended to be the same as the DISPLAY'S GAMMA, as the overall system gamma is designed to increase the gamma from end to end.
Thus a Rec709 file should never be encoded as gamma 1/2.4, but should be the parametric curve closer to 1/2.0, thoiugh it should be noted that the viewing of the working space SHOULD be setup (such as with a view LUT) to display that system gamma gain.
VFX & Title Supervisor
No problem with what you say - just saying Adobe normalize inputs to a working color space, Premiere uses always rec709 gamma 2.4, historically default in AE was gamma 2.2 - now they added in AE a Working gamma with option gamma 2.2 and 2.4
So all I am saying is it's best in AE to work in sRGB gamma 2.2 so your video in browsers or most applications look as intended (otherwise you will likely get a gamma shift on playback).
Oh I see what you're saying Pierre — I rarely use Premiere. But I thought Premiere was not color managed? If it is not, then it would be correct to send the image into a 2.4 gamma calibrated monitor, or to emulate that with a LUT.
I don't use the newer versions of AE — when you say they added a "working gamma of 2.4", I assume you mean when colorspace is off/unmanaged? I've looked at the new features list, and I don't see that as an option? What/where is it? Can you post a screenshot, I'm curious what Adobe means here.
VFX & Title Supervisor
Screenshot - if someone loads a linear EXR file from a 3D render, AE will by default delinearize it to the working space...
Thank you for the screenshot Pierre,
Well that's interesting. Looking into this and based on how Adobe has traditionally handled non-color-managed operations, that preference only does the following:
1) Use 2.4 gamma for certain blend operations.
But I think I can also safely assume that switching the preference between 2.2 and 2.4 will have NO EFFECT on the appearance or rendering of outputs (except perhaps a render to EXR).
Prior to that feature, with color management off, AE would use a gamma encoding for blends (unless "use gamma 1.0 for blends" was checked), and the display data would just be sent to the monitor, which typically would be an sRGB monitor with a ~2.2 gamma.
While I can't test this I have to assume that if you switch this to "assume 2.4", the displayed image will not change, and thus if using that same sRGB monitor, you will still be seeing images with a 2.2 display gamma, not 2.4
With color management OFF, you need to physically adjust your monitor to a 2.4 gamma to see 2.4 gamma, regardless of how that preference is set. Also, output modules work the same as if "preserve RGB" was checked with CM ON.
With color management ON, you can simulate different output devices, such as you can emulate a 2.4 gamma display on your standard sRGB 2.2 gamma display. Also, the output module can convert to a different space/gamma unless "preserve RGB" is checked.
If you use AE with color management off AND "blend using gamma 1.0", then the only thing I expect that preference to do is convert linear material to the selected gamma, which you would previously do by using the Profile Converter effect or Levels with a gamma adjustment.
VFX & Title Supervisor