FORUMS: list search recent posts

H264 gives better results than ProRes 422(HQ) or Uncompressed?!

COW Forums : Apple Final Cut Pro Legacy

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Adrian Makai
H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 10:20:02 pm

H264 gives better results than ProRes 422(HQ) or Uncompressed?!

I have a peculiar problem and was wondering if anyone could chime in. I have some footage that I need to send to a colorist and want to export it to 10-bit uncompressed. But the export always ends up with some weird, pixelated color-smearing in one particular scene when I export to either uncompressed or ProRes 422HQ. Curiously, the footage looks fine if I export via H.264. Why doesn't uncompressed or PR give superior results?

Here are the particulars:

  • Footage originated on Panny AF100 8-bit, 4:2:0 but we added some VFX layers and exported new footage as 10 bit QT files
  • I converted the above 10-bit files to ProRes 422 HQ for editing
  • My FCP 7.03 sequence set to Render ALL YUV in high precision YUV
  • Before exporting I set sequence to uncompressed 10-bit
  • I have tried exporting as self-contained QT movie or sending to Compressor and get identical results either way
  • Hardware is Intel MacPro, 8-processors, 12 GB RAM


The problematic scene is in a desert and has some blown-out highlights, especially on the chrome trim of a car and some snow-capped mountains in the background. That scene looks fine in my FCP timeline - the aforementioned highlights are certainly hot, but no weird color smears.

But when I output to either uncompressed or ProRes 422HQ, the highlights get smeared with pixellated rainbow colors, like some 8-bit Atari video game. When I output to H.264, no smearing, the footage looks as good as it does in my FCP timeline.

What could be causing this and how could I get around it? I don't want to send H.264 compressed footage to the colorist.

I would upload some frame-grabs, but I'm not authorized as clip contains some name actors and producers would not be happy.

Any input much appreciated!


Return to posts index

Rafael Amador
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 10:29:32 pm

Are you rendering ALL YUV stuff in High Precision?
You shoud do that wen exporting to any 10b codec.
rafael

http://www.nagavideo.com


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 10:31:16 pm

Yes, as mentioned in my bullet points:

- My FCP 7.03 sequence set to Render ALL YUV in high precision YUV

I'm really scratching my head on this issue!


Return to posts index


Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 11:21:47 pm

Ok. This is a pretty crazy workflow.

First, AVCHD to ProRes. (1)

Then VFX and export to 10b UC. (2) maybe (3)

Those then converted to HQ for editing. (4)

Then exported back to 10b UC (5)

That's perhaps 5 generations not including any I missed. Considering your already highly compressed 24ish megabit/sec source, this is unnecessary.

Please post a screen grab of what you are seeing so we can speak objectively about what you might be happening.


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 11:36:52 pm

Okay, here is a screen grab with cropping to highlight the problem areas. Note the purple splotches on frame left (car windshield) and on actresse's right arm and chest. The two actors in back seat also have the color smear, as does a thin band on hood of car.



In my timeline, these color splotches are not there. Those areas simply look a bit hot but otherwise normal. And if I send to H.264, those areas also look normal. They only look like in this pic if I send to uncompressed or ProRes 422HQ.

In terms of editing, what workflow do you recommend?


Return to posts index

Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Aug 31, 2011 at 11:52:50 pm

These look like transcode errors of some sort.

Let's start from the beginning.

Did you log and transfer the AVCHD footage? To what?


Return to posts index


Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:00:07 am

I believe the assistant editor logged and transferred the AVCHD right to ProRes 422HQ


Return to posts index

Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:05:30 am

Excellent.

Then when you do the VFX, what's the workflow there? Where do you do the VFX?


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:09:35 am

After editing the final cut, the entire movie was exported to TIFF sequences (I know another step!) so that we could move it to Windows for VFX work in Combustion.

Work was done in 16-bit in Combustion and then exported to 10-bit MOV animation for re-import back into FCP on Mac.

After all the above steps, the footage looks great playing in FCP. It's only when I try to export again that I get the color smears on that one scene.

Oh - and just to test, I went back to the very earliest cut of the movie, before any work had been done (other than the original transcode to ProRes422 HQ), and tried exporting that particular scene to a self-contained movie. Same problem, whether I do self-contained or send to Compressor. The H.264 looks fine. The unocomp and PR looks like the above pic.


Return to posts index


Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:18:54 am

And as another test, I opened the original media clip (the first transcode from AVCHD to ProRes422 HQ) and it plays fine in Quicktime Player. Screen grab below:



Return to posts index

Matt Lyon
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:34:09 am

If the original clip truly has no problems, then you should find the clip in the earliest cut of the movie you mention, select it, then right click>"reveal in finder." Open that clip in quicktime player 7 and trim out just the section you need for VFX. Use "save as..." to give the movie a new name, and send that short clip to the VFX artist. Now he/she is working with the closest to original generation footage possible and hopefully with none of the problems you are seeing.

Just out of curiousity, are there any filters or motion effects being applied to the clip before you did the problematic exports?

Hope this helps,

Matt Lyon
Editor
Toronto


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:38:04 am

Thanks for the input but the problem has nothing to do with the VFX step. As mentioned a couple of posts up, I tried going back to the earliest cut of the movie (before anything had been done other than dropping the clips into FCP) and exporting that sequence to QT. Same proble - looks fine playing in the timeline or when exported to H.264. Looks smeared when exporting to uncompressed or PR422HQ.

Still a mystery...


Return to posts index

Matt Lyon
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 1:09:01 am

Hi Adrian, I meant to say "colorist" instead of VFX artist. I was trying to show you a way to send "clean" footage, without introducing the artifacts. If your footage looks fine in the FCP timeline, then you need to examine how you do your exporting, because a "self contained quicktime" should theoretically be a 1:1 copy of your clip, with no re-compression (assuming you haven't applied any filters/motion tab effects, etc). Based on your original post, the problem might be that your changed your timeline setting to "ucompressed" before you did your export ... there is no reason to do that. Leave your timeline settings so they match your clip setting and simply export a self contained quicktime.

Prores HQ is generally considered overkill unless you are editing 4K footage, to answer your other question.

Your workflow involves tons of codec and color space conversion so it is not surprising that problems are being introduced. In general, you should strive for as few transcodes as possible. Changing your prores footage to 10 bit uncompressed does not make it look better. I would only do it your colorist specifically can only work with that format. Also, when you exported your whole movie as TIFFs, the image files would have been limited to 8 bit color depth.

Matt Lyon
Editor
Toronto


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 1:12:41 am

Thanks for the feedback, Matt.

Even if I don't change my timeline to 10-bit, but keep it at ProRes422 HQ (identical to the base clips) - same problem! I was originally doing that before I tried 10-bit.

Thanks for the reminder on TIFFs 8-bit limitation. The original footage was 8-bit, so hopefully we didn't lose any quality when sending to VFX. It didn't look like any quality was lost.

It's only after VFX have been applied with higher bit-rate layers, that we are now trying to stick to 10-bit.


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 1:32:04 am

Actually, one reason we used TIFFs is because it was supposed to be lossless, or so we thought?

Does FCP limit TIFF output to 8-bit regardless of the bit-rate of the footage? If yes, that would be nuts...


Return to posts index

Matt Lyon
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 2:09:33 am

TIFF images themselves are lossless, but when you export from a video codec to tiffs, you are converting from YUV color space to RGB color space. That introduces issues, like the the fact they have different color gamuts.

There is no such thing as 10 bit TIFF files, just 8 bit per channel or 16 bit per channel (and maybe floating point? I can't remember...). In any case, FCP doesn't do 16 bit per channel anything, so I don't think it is "nuts" that it is limited to 8 bit per channel TIFFs. Last I checked, quicktime player won't open 16 bit TIFFs either.

Matt Lyon
Editor
Toronto


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:19:54 am

Just posted the original screen grab in the thread for your ref.


Return to posts index

David Roth Weiss
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:33:48 am

I'd be willing to bet that those pink aberrations would not be there if you used ProRes 422 instead of HQ.

And, I'd be will to bet that in AE the render was either premultiplied or not premultiplied, and whichever it is, that's the issue.


David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
http://www.drwfilms.com

Don't miss my new tutorial: Prepare for a seamless transition to FCP X and OS X Lion
http://library.creativecow.net/weiss_roth_david/FCP-10-MAC-Lion/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 12:39:38 am

Interesting. Why would 422 be better than 422HQ? I always thought the latter is supposed to be better.

And what might account for the fact that if I just open the original 422HQ clip in QT, it plays fine? I posted a clean still from that clip a few posts above - no pink smear.


Return to posts index

David Roth Weiss
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 1:06:33 am

[Adrian Makai] "Interesting. Why would 422 be better than 422HQ? I always thought the latter is supposed to be better.
"


From time to time HQ simply kicks off some problems that ProRes 422 just doesn't. I'm not 100% certain why, it's just the way it is.

[Adrian Makai] "And what might account for the fact that if I just open the original 422HQ clip in QT, it plays fine? I posted a clean still from that clip a few posts above - no pink smear."

It's most likely some type of generational thing with ProRes HQ. I've never seen anything like that with ProRes 422


David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
http://www.drwfilms.com

Don't miss my new tutorial: Prepare for a seamless transition to FCP X and OS X Lion
http://library.creativecow.net/weiss_roth_david/FCP-10-MAC-Lion/1

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


Creative COW contributing editor and a forum host of the Business & Marketing and Apple Final Cut Pro forums.


Return to posts index

Michael Gissing
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 1:54:30 am

Interesting that the artifacting is on the 10 bit exports and not the H264 which is 8 bit.

Try an 8 bit uncompressed and see if that helps. It is clearly something to do with peak whites. Another left field idea is to drop the 3 way cc and bring the peak whites to 100 or just under and see if that helps with the 10 bit exports


Return to posts index

Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 2:51:35 am

[Michael Gissing] "Interesting that the artifacting is on the 10 bit exports and not the H264 which is 8 bit.

Try an 8 bit uncompressed and see if that helps. It is clearly something to do with peak whites. Another left field idea is to drop the 3 way cc and bring the peak whites to 100 or just under and see if that helps with the 10 bit exports"


This is exactly where I was going to go with this. It used to happen long ago with scaled 10bit video. The trick at the time was to put am "empty" filter on it like saturation turned to zero and rerender/export.

In this case though, my thoughts are some sort of clipping issue when coming from RGB.

Adrian-

So the original ProRes HQ export straight from timeline, pre VFX still shows the artifacts? So how did you get the clean screen grab? Or are you saying your orig footage/export doesn't show this?

To get around the 8bit QT limitation for image sequences, use motion, AE, even Compressor. In Compressor, make sure frame controls are on.


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 2:56:59 am

[So the original ProRes HQ export straight from timeline, pre VFX still shows the artifacts? So how did you get the clean screen grab? Or are you saying your orig footage/export doesn't show this?]

Yes, pre VFX still shows artifacts.

Yes, clean grab was from orig footage transcode from AVCHD to ProRes 422 HQ. Without opening FCP, I just opened that particular clip in Quicktime Player and grabbed the clean frame.

And again, that very same clean clip plays fine within FCP as well. It's only when exporting or sending to Compressor in either uncompressed or ProRes 422HQ that I get color smear.

Will try the 8-bit suggestions and see if that works.


Return to posts index

Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 3:31:11 am

[Adrian Makai] "Yes, clean grab was from orig footage transcode from AVCHD to ProRes 422 HQ. Without opening FCP, I just opened that particular clip in Quicktime Player and grabbed the clean frame."

Do you have a capture card and a broadcast monitor?

Have you looked at this on another computer/monitor?

And all you are doing is File > Export > QuickTime Movie @ current settings and self-contained checked (leaving recompress off)? What happens if you export a ref movie?


Return to posts index

Rafael Amador
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 1, 2011 at 10:35:38 am

Crazy workflow.
For me those colors are typical when trying rendering 1080/10b stuff on a short GPU computer.
IMO, nothing related with Prores vs ProresHQ.
rafael

http://www.nagavideo.com


Return to posts index

Adrian Makai
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 5, 2011 at 4:11:53 am

Thanks Guys - problem solved!

Added a basic color correction and broadcast safe filter, adjusted down the highlights and chose a very conservative broadcast-safe setting and presto! Tested it every which way and it now exports properly.

I still find it interesting that H264 somehow auto-corrected for all that while the uncompressed and ProRes 422HQ exports did not. Oh well, not gonna sweat the why's anymore. Back to editing.


Return to posts index

Michael Gissing
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 5, 2011 at 7:15:02 am

[Adrian Makai] "I still find it interesting that H264 somehow auto-corrected"

I am not surprised as I suspected it was a 10 bit versus 8 bit issue and H264 @8 bit effectively clamps peak whites to 100.


Return to posts index

Jeremy Garchow
Re: H264 gives better results than ProRes 422(HQ) or Uncompressed?!
on Sep 6, 2011 at 4:32:37 pm

[Michael Gissing] "I am not surprised as I suspected it was a 10 bit versus 8 bit issue and H264 @8 bit effectively clamps peak whites to 100."

Mmm. I might have to disagree here. It doesn't have to do with 10 vs 8 as 8 bit video can carry "illegal" signals.

I would think it's RGB vs YUV. I do think that most likely, the h264 was clamping the values to RGB and therefore the problem was solved.

Still love you though Mr Gissing!


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2018 CreativeCOW.net All Rights Reserved
[TOP]