FORUMS: list search recent posts

The Color Wheels Mystery ... An obviously wrong answer

COW Forums : Apple Final Cut Pro X Debates

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Simon Ubsdell
The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 10:54:54 am
Last Edited By Simon Ubsdell on Jan 3, 2018 at 11:52:29 am

Oliver has pointed out to me that someone on Bill's Facebook discussion is suggesting that I am confused and that I am wrong to think that we should be expecting a Lift/Gamma/Gain model here.

Instead the poster is claiming that what Apple have done is implement the SMH method and not Lift/Gamma/Gain.

Now I happen to know something about the SMH method since Rob and I used it for the color corrrection section of Hawaiki AutoGrade , so will try to explain as clearly as I can why this theory is wrong.

I can only assume that the poster looked at Apple's naming of the controls (Shadows, Midtones, Highlights) and jumped to the incorrect conclusion that this was the model being used. But of course Apple are notorious for this: witness sequences that are called "projects" and so on. So nothing should be automatically assumed from the naming.

SMH was originally developed by Baselight to handle Log footage (and subsequently adopted by other color systems) and very simply it works on the basis of dividing the full range of the image into three distinct but overlapping bands - it's conceptually very similar to a 3-band audio equalizer.

The value of SMH lies in being able to target the shadows, midtones and highlights much more precisely than you can with Lift/Gamma/Gain. I repeat: much more precisely. (Not less precisely!)

Thus a "Shadows" correction will only ever extend up into the mid greys and never even remotely reach the high mids or whites.

A "Highlights" correction similarly will only ever reach into the mid greys and never ever touch the low greys or blacks.

And the "Midtones" correction will only ever reach into the low mids and high mids and never reach into the blacks or whites.

The maths of this are a little more complicated than you'll want me to explain here but they are not rocket science.

Now the whole point of the video that I made was to show that the 10.4 Color Wheels were behaving less precisely than the Lift/Gamma/Gain model. I repeat: less precisely. Not more precisely. The polar opposite of more precisely, in fact.

SMH has a very characteristic footprint that is easy to recognise. You can test it out for yourself if you grab copy of Da Vinci Resolve and take a look at the Log color wheels which use this model. (As far as I know, and could be wrong, SMH is only ever found as an additional tool to Lift/Gamma/Gain, never as a complete replacement for it.)

Here is a set of screenshots that show SMH corrections, using the same greyscale ramp that I tested with in the video:



You can see how the line for the Shadows deflects only at the low midpoint. The line for the Highlights deflects only at the high midpoint. And the Midtones correction is a curved deflection overlapping gently with the deflection points of the other two.

This is what we expect to see from SMH.

And this is the exact opposite of what we are seeing from the 10.4 color wheels (in Rec.709):



In this case, we can see clearly that a "Shadows" correction pulls down the whites; a "Highlights" correction pulls up the blacks; and a "Midtones" correction pulls up both blacks and whites.

Edit: (Incidentally the straight lines that we see in both the "Shadows" and "Highlights" corrections here, albeit anomalous in one way, are a clear indication that the process being applied is a straight multiplication of the input pixel value by the control value (or much less likely an addition, or a mixture of the two). If there were curvature of the line or a deflection at any point between zero and one, we would be looking at a different process, but clearly we are not.)

So let me stress one more time: the observable and measurable behavior of the Color Wheels in 10.4 is the diametric opposite of SMH. Instead of more precise ranges, we are seeing much less precise ranges.

It simply could not be clearer.

So what is the answer to the mystery? I won't claim to know anything for sure at this point but Oliver's theory seems to hold up very well indeed with no evidence that I have yet seen that it is wrong. More on that later if I get some time to do more testing.



Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Brett Sherman
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 2:25:33 pm

Let me just say, I appreciate your thoughtful and knowledgable investigation of this. I can't understand why anyone would think differently.

--------------------------
Brett Sherman
One Man Band (If it's video related I'll do it!)
I work for an institution that probably does not want to be associated with my babblings here.


Return to posts index

Jeremy Garchow
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 4:23:07 pm

Yes, thanks so much, Simon.

This all great conversation.


Return to posts index


Robin S. Kurz
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 5:25:27 pm

Interesting.

- RK

____________________________________________________
Deutsch? Hier gibt es ein umfassendes FCP X Training für dich!


Return to posts index

Bill Davis
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 5:28:30 pm
Last Edited By Bill Davis on Jan 3, 2018 at 5:47:24 pm

Oddly, I just spent a small chunk of time this AM chatting with Phil Pan on-line regarding this general topic.
(Hint: there are plenty of solid reasons to eschew Facebook participation, but as a modern means of connecting with expertise if used judiciously, it has few equals, IMO.

Just prior to our chat, he had posted a general overview of color science terminology that I found HUGELY useful as I start to learn more about this topic.

With his permission, I’ll post it below.

But the one thing it has utterly convinced me of is that color science is complex, it’s implementations in modern software are evolving, and I’m WAY dumber about these things than I ever imagined.

This was his response to some questions about basic color science as it relates to the new stuff in X...

Phil Pan:

Let's start by defining the vocabulary.

The term 'Brightness' is defined by the Commission Internationale de l'Éclairage as "the attribute of a visual sensation according to which an area appears to exhibit more or less light". Brightness is therefore, by the CIE's definition, a subjective quantity that cannot be measured. Brightness doesn't have a formula or units, contrary to CIE Luminance and CIE Lightness, which do. Brightness should be understood as the subjective effect (the adjective of its appearance) of Luminance.

The CIE defines Luminance as the radiance weighted by the human spectral sensitivity function; Luminance, measured in candelas-per-square-meter, is the human vision (perceptual) equivalent of Radiance (measured in Watts per steradian per square meter). In film/video reproduction, only Relative Luminance is used (Luminance relative to a defined white level) — this quantity is denoted "Y" and has linear characteristics in relation to physical radiance.

Because of the characteristics of the human visual system, a linear/uniform increase in Y intensity yields a non-uniform increase in brightness perception. To compensate for this, the CIE has defined a uniform brightness scale, called Lightness (denoted L*, pronounced 'L-star'), which ranges from 0 to 1, and which has a characteristic power-function curve. Each increment of 1% in Lightness is not only perceived as a uniform increment, but constitutes also the threshold (the 'just noticeable difference, or JND) of brightness perception (denoted "delta E").

In video systems, for matters of computational efficiency, an approximation of CIE Lightness is used, which is called Luma; it is Luminance raised to a gamma factor (say, 2.2) and is denoted Y' ('y-prime'). Exposure and Contrast are different beasts altogether.

Exposure is a dimensionless quantity that is related to the Irradiance of a photonic device (Watts per square meter), or how many photons hit its surface per unit of time. Exposure Value (EV), which is colloquially called 'f-stop' is set in an exponential scale (doubling or halving photon count). "Changing the exposure" in a numeric simulation that is properly engineered in linear (relative Luminance) units, is the same thing as applying an additive ('offset') to the input values.

Contrast is a subjective value (similar in semantic function to Brightness) that is related to the ratio of two Luminance or Reflectance stimuli. When applied to an entire image, contrast refers in essence to both the spread of its histogram and the barycentre of this spread (the contrast 'pivot').

Gain is not a unit of colour science, but a term in electrical engineering which is a simple multiplier. A gain of “2” is a multiplication by 2. A gain of “0.5” is a multiplication by 0.5.

Setup (also called ‘pedestal’ or ‘black level’) is an archaic term from the days of composite NTSC. It represents the (paradoxically ‘visible’) black level above horizontal blanking (the level at which the electron gun’s spot had to invisibly cross the frame to draw new scan lines). Since there were no less than three NTSC standards for black level, old analogue systems had a “setup” parameter so one could conform their black level to whatever NTSC standard they were using. The most common Setup (black level) was pegged at 7.5% of picture excursion. The term ‘picture excursion’ comes from the defunct american Institute of Radio Engineers (IRE), which defined electron gun blanking at 0 IRE units and white level at 100 IRE.

Avid is the oldest digital video software company on the block and its tools are laden with legacy items that will not be removed until the last user from the old analogue broadcast days retires. It makes zero sense in 2018 to use analogue-days nomenclature and units; nor does it for Final Cut to still show a waveform monitor that expresses picture excursion in IRE units, with antiquated concepts such as ‘superblack’ and ‘superwhite’. But the FCP team didn’t have much of a choice, since it needed to compete with Avid in what had become a buttoned-up, conservative market.

In 2008, the American Society of Cinematographers’ (ASC) Technology Committee, defined a bare bones (minimum common denominator) colour decision exchange standard, to enable dissimilar digital imaging systems to achieve and conform the most basic colour grading styles among each other. The Color Decision List (CDL) format is an XML structure that defines four control dimensions: Slope, Offset, Power and Saturation.

Translated in video systems, Slope is the resultant linear angle of the video transfer function achieved through a combination of Lift and Gain. Slope affects contrast (the steeper the slope, the more contrast). Offset is the same thing as either ‘Brightness’, ’Setup’, ‘Offset’, ‘Exposure’ or ‘Picture Level’. Power is the same thing as ‘Gamma’. And Saturation is the same thing as HSL saturation (amount of chroma).

Kodak’s Glenn Kennel and ILM’s Josh Pines chose these units because they encompass a pretty wide array of possibilities and can easily be converted to and from legacy colour grading systems in the video world. It’s good to know that Slope (Contrast), Offset ( Exposure/Brightness/Setup), Power (Gamma) and Saturation, taken together are the most ‘’orthogonal’ control dimensions in a colour grade; this can help you choose what tools are best suited to perform your grades.

Now, if there are problematic naming conventions in FCPs various colour tools, one shouldn’t panic. As is the case for Avid, there are legacy reasons for the naming conventions to be messy. What matters is that you understand which parameter does what to your image. In time, as the industry evolves, Apple will be in a position to ditch the old names and abide by a more coherent standard.

(Ref. Poynton, Charles. “Digital Video and HD”. 2012 Elsevier; Tooms, Michael. “Colour Reproduction in Electronic Imaging Systems”. 2016 Wiley; Giorgianni, Edward. “Digital Color Management - Encoding Solutions”. 2008 Wiley)

Creator of XinTwo - http://www.xintwo.com
The shortest path to FCP X mastery.


Return to posts index

Simon Ubsdell
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 8:09:47 pm
Last Edited By Simon Ubsdell on Jan 3, 2018 at 8:12:08 pm

Dear Bill

Thank you for sharing your chat with Philippe.

To my mind there are two ways in which one can go about imparting this kind of information:

a) you can try and make it as easy as possible for your readers/audience/viewers/listeners to follow along, even if they don't get to grips with the full complexity of the subject;

or b) you can dazzle everyone with your superior knowledge by copying and pasting impressive-sounding passages from the standard texts.

Both approaches are valid, but my preference is for the former.

Although the first option is recommended when you have been caught out in an egregious error.

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index


Charlie Austin
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 8:51:05 pm

👍

-------------------------------------------------------------

~ My FCPX Babbling blog ~
~"It is a poor craftsman who blames his tools."~
~I still need to play Track Tetris sometimes. An old game that you can never win~
~"The function you just attempted is not yet implemented"~


Return to posts index

Bill Davis
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 10:46:06 pm

[Simon Ubsdell] "or b) you can dazzle everyone with your superior knowledge by copying and pasting impressive-sounding passages from the standard texts."

Uh, ... all I thought I was doing was posting material from a guy I had met online who knows BOATLOADS more than I do - so what I was starting to learn about might be useful to others.

If it came across as ME trying to be ANY type of expert - holy heck I’m sorry.

I thought I’d been SUPER clear about that.

Swear to God I just thought it was both interesting and informative and might be useful since there are people who hang here that are not on Facebook.

Period.

Creator of XinTwo - http://www.xintwo.com
The shortest path to FCP X mastery.


Return to posts index

Shane Ross
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 9:05:25 pm

Why was Avid brought into this? Even colorist or online people who use it (and it's mostly not by choice, but because of production chain) complain about Avid and rail against how antiquated it is. How useless it is anymore. Avid is not the standard for colorist tools to be held up to...its tools are really old and very limited, compared to Resolve or Baselight.

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index


Bill Davis
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 11:05:18 pm

[Shane Ross] "Why was Avid brought into this? Even colorist or online people who use it (and it's mostly not by choice, but because of production chain) complain about Avid and rail against how antiquated it is. How useless it is anymore. Avid is not the standard for colorist tools to be held up to...its tools are really old and very limited, compared to Resolve or Baselight.

Shane
Little Frog Post
Read my blog, Little Frog in High Def"


Look, I had much rising confusion about this issue.

Like many, I questioned whether there was an ACTUAL flaw in how X handled color in the new 10.3 tools.

It was being discussed on another forum.

That discussion attracted participants with high level color science expertise.

I don't know much about this topic, but some folks like Phil Pan (his on-line handle) are legitimate color scientists - so when I learned a lot from reading this post there - I felt it appropriate to try to spread that expertise.

So I asked for and gained specific permission for the re-post I made.

IT's what HE wrote on his own - copied by me VERBATIM and ENTIRELY - so there would be no possible claim of cherry picking.

If AVID was mentioned, it's because he elected to mention them. Nothing more or less than that.

If you have issues with the suitability of the color approach in ANY software program, X, Premier Pro, Resolve, AVID, heck - MacPaint for all I care - look elsewhere for answers.

I just thought more expertise was better than less when discussing such a complex topic.

Period.

Creator of XinTwo - http://www.xintwo.com
The shortest path to FCP X mastery.


Return to posts index

greg janza
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 3, 2018 at 11:28:23 pm

This ain't "color" rocket science. Either the color tools work in a manner that fellow video professionals have defined as the standardized norm or they have a bug flaw that needs to be addressed. Or is Apple now re-inventing color too?

We're all quite familiar with software bugs in this industry so adding one more bug to the pile is acceptable.

I Hate Television. I Hate It As Much As Peanuts. But I Can’t Stop Eating Peanuts.
- Orson Welles


Return to posts index

Scott Witthaus
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 9:46:29 pm

[greg janza] "This ain't "color" rocket science."

Is this really that big of a deal? I mean if Wheels don't work "manner that fellow video professionals have defined as the standardized norm", can't you just use curves or color board? Hell, just go over to Resolve. Lots of gnashing of teeth over something that most probably view as a minor "issue", if it is one?

Scott Witthaus
Senior Editor/Visual Storyteller
Managing Partner, Low Country Creative LLC
Professor, VCU Brandcenter


Return to posts index


Michael Gissing
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 9:55:54 pm

[Scott Witthaus] "Is this really that big of a deal?"

It depends if you had spent days grading in X. If Apple fix it in an update then will it make previously graded material using the wheels track incorrectly? No-one likes to have to redo work. I thinks it's a bug and it is only fair to bring it to the attention of editors who may waste precious time. After all, why use the fastest NLE in history if you spend days redoing work?

If this was a third party developer then the same level of debate would be warranted.


Return to posts index

Scott Witthaus
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 10:11:45 pm

[Michael Gissing] "It depends if you had spent days grading in X. If Apple fix it in an update then will it make previously graded material using the wheels track incorrectly?"

Days grading already in 10.4, a dot-zero version of a major upgrade? Standard thinking among a lot of video/post professionals is not to jump on a major upgrade until some of the kinks/bugs are worked out that are inevitable in a major release.

Scott Witthaus
Senior Editor/Visual Storyteller
Managing Partner, Low Country Creative LLC
Professor, VCU Brandcenter


Return to posts index

Michael Gissing
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 10:19:00 pm

[Scott Witthaus] "Standard thinking among a lot of video/post professionals is not to jump on a major upgrade until some of the kinks/bugs are worked out that are inevitable in a major release."

Which is why it is important for the community to share these findings and warn others. So it is a big enough deal to debate. I also scan forums before jumping in on updates. My days of beta and alpha testing software are over, but the heroes that do it should be acknowledged for finding and sharing bugs so the rest of us can benefit from their efforts.

Given that you imply this is a major release and the color wheels are a major feature, I don't understand why you question this debate.


Return to posts index


Simon Ubsdell
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 10:00:14 pm
Last Edited By Simon Ubsdell on Jan 4, 2018 at 10:06:17 pm

[Scott Witthaus] "Is this really that big of a deal?"

I would say that the current set-up is entirely workable (if curious) and nothing to get too distressed about.

But because I am interested in this kind of stuff, I wanted to share my findings in the hope that it would interest others and perhaps lead to a better understanding of some basic color science principles that it does pay to be aware of since they can help with your work.

That's all really.

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Shane Ross
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 4, 2018 at 12:45:26 am

Bill, I wasn't accusing you of this. I know you just copied and pasted. Sorry to give that impression. I was just wondering why Avid came up at all when color science was being talked about. That's all

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index

Axel Schreiter
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 5, 2018 at 8:04:13 am

@Bill:
I think I know what you are trying to say. Unless we are not sure what this is all about, we shouldn't panic. When first confronted with FCP X in 2011, I thought: this doesn't work for me. As we all know now, this was not just a panic reaction. The first versions were very buggy. They weren't stable. I saw the beachball too often, and performance was worse than on FCP7.

The new timeline paradigm (along with the new naming conventions) was the part where Apple was ahead of everyone. But this particular point wasn't the reason for the famous PR disaster.

I came on board late. I think it was 10.0.6. Watched someone edit on a set on a MBP, and I was impressed. Wouldn't have been if the app had crashed then ...

Same with the Color Board. Not too long before X, I had learned Apple Color. Now instead of implementing a grading suite, they adopted the iMovie tool. I thought, they must be kidding.

So I learned Resolve. Almost without noticing, I found myself using the Color Board. It is so fast and intuitive. Is Resolve superior? You bet, but it's not necessary for 90% of what I do.

While not being a colorist, I do follow professional advice on how to perform CC/grading, with primary and secondary iterations and using scopes for accuracy. For my way of approaching secondaries, HSL curves are my tool of choice, and now that they are in FCP, I am very happy.

Before having seen Simons video, I was already irritated that the new Color Wheels seemed to be less powerful than the pathetic Color Board. It's like you say, I thought I must be missing some new ingenious paradigm. Is there one? I wish your color scientists could help us with a plain explanation. In my understanding, the existence of a, er, "3-way-color-corrector" just implies that each of he ranges is a selection, and of what use is any selection if it's almost unlimited? This is the simple question your experts should answer.


Return to posts index


Oliver Peters
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 5, 2018 at 2:13:10 pm

What isn't getting mentioned a lot in a "wait and see" approach is the real world ramifications of the issue. Mainly this:

1) You end up doing twice as much mousing around to use the controls than need be.
2) If it is something that gets corrected or changed in a future update, your current corrections will likely all change after the update with exitsing projects.
3) The nature of the control is that it can induce needless clipping.

- Oliver

Oliver Peters - oliverpeters.com


Return to posts index

Simon Ubsdell
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 5, 2018 at 3:05:04 pm
Last Edited By Simon Ubsdell on Jan 5, 2018 at 3:42:30 pm

[Oliver Peters] "What isn't getting mentioned a lot in a "wait and see" approach is the real world ramifications of the issue."

To expand on that a bit in relation to the RGB performance of the wheels, here is the Color Board result when pushing the midtones of the greyscale test ramp to yellow:



Entirely as predicted by the conventional model.

And here is what the Color Wheels do with the same correction:



You can see red "whites" being pushed above one (100%) and the green and blue "blacks" being pushed below zero, very heavily so in the case of the blue. Similarly the green and blue "whites" are reduced, the blue dramatically so, rather than being pegged at 100% at the top of the curve.

I think it's fair to say that this is not really desirable.

You can start to get back to the desired result by pulling the "Shadows" and "Highlights" away from yellow, but that's two extra corrections to get to the same place, and even then it's not exactly the same place.

Incidentally, this is not a separate issue to the one we have been highlighting up to now - it's merely the repercussions of the exact same issue as they affect the RGB.

That is to say, there is one answer to this single mystery, not several separate mysteries. I think ...

Edit: There is an added complication when dealing with RGB in that we need to factor in the luma coefficients but it's best not to add more difficulty to the explanation at this point. Suffice it to say that because green, red and blue are not equally "bright" (to use a non-technical term for it) RGB corrections need to factor in those relative "brightness" values and if the input pixel values are not being correctly interpreted (as is the case with the color wheels in Rec.709) that can make life quite tricky.

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Simon Ubsdell
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 5, 2018 at 3:45:46 pm

[Axel Schreiter] "Before having seen Simons video, I was already irritated that the new Color Wheels seemed to be less powerful than the pathetic Color Board. It's like you say, I thought I must be missing some new ingenious paradigm. Is there one? I wish your color scientists could help us with a plain explanation."

I hope this new video will help you understand the answer:







Simon Ubsdell
tokyo productions
hawaiki


Return to posts index


Simon Ubsdell
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 5, 2018 at 8:32:17 pm

Apologies to all. Senility appears to be setting in.

I wrongly claimed in the above post that we implemented SMH in Hawaiki AutoGrade, when in fact we only implemented it for the RGB and not the Y.

It still makes for a very interesting and useful set of controls.

But SMH can only ever sensibly be thought of as an addition to Lift/Gamma/Gain, never a complete replacement for it.

Targeted corrections are great, but they are inherently problematic.

Lift/Gamma/Gain gives you the smoothest possible transition between adjacent pixel values and that is fundamentally a very good thing. But when you start to target more defined ranges then it becomes a lot easier to mangle your image quite badly.

For this reason, SMH needs to be handled with care - think of it as local surgery, not as a universal panacea.

And one of the reasons I bring this up in this context is to do with the new curves features in 10.4.

They're absolutely great and it's really nice to see this functionality in an NLE and not just in the big grading systems. But at the same time, inexperienced colorists risk their images falling apart in all sorts of ways that they might not be alert to.

It's important to think of these tools as problem solvers, not primary grading functions. Otherwise problems potentially lie ahead.

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Bill Davis
Re: The Color Wheels Mystery ... An obviously wrong answer
on Jan 6, 2018 at 2:16:58 am
Last Edited By Bill Davis on Jan 6, 2018 at 2:19:24 am

Simons above post seems to me very useful advice.

Many like me will be exploring this new implementation of well understood colorists tools and anything that helps people learn to “go slowly” and use restraint with a tool that can get you into trouble - is probably very good advice.

My 2 cents.

Creator of XinTwo - http://www.xintwo.com
The shortest path to FCP X mastery.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2018 CreativeCOW.net All Rights Reserved
[TOP]