WHY can I not get a truly raw render?
I want to pin this one down once and for all. With less than a day left to me, it's terribly important.
Premiere Pro CS4 has never given me a truly raw output. By that, I mean a video file which I can re-import and compare minutely with the original video in the project from which I generated the raw video. No matter what mode (V210, UYVY) or bit-depth ("Render At Maximum Depth" on/off) I pick, the resulting "raw" video has inexplicably lost roughly half of its horizontal resolution; if I look close, what used to be fine 720x480 pixels has for whatever reason been modified into a sort of 360x480 bastardization (not a precise loss of resolution, but more like some kind of degradation). Hazarding a truly wild guess, it may be that Premiere Pro is performing a horizontal downscale and then upscale before generating output, based on my pixel aspect ratio setting (DV NTSC widescreen).
What really gets me is that if I tell Premiere Pro CS4 to encode directly to Media Encoder's MainConcept MPEG2 codec (NOT the solution I am after, but at this point I was willing to deal with it), the resolution is STILL tampered with before the Media Encoder ever gets anything to encode, and so the resolution in the MPEG2 output is just as poor as the raw video output.
(Incidentally, telling PPro at the time of render that my project has "square pixels" rather than "DV NTSC widescreen 1.2121") results in PPro shrinking the vertical resolution during output, permanently.)
If it matters, the project I'm working with is (as is probably obvious by now) YUV, 8 bits per channel, and that is of course the target output so I can avoid two unnecessary colorspace conversions. I may toy with AE's raw output before long (importing the PPro project), as a last resort.
You know I have to ask to be sure, any de-interlacing going on?
How are you comparing those 2 outputs? I don't mind doing some testing on my end.
CS4 should actually give you a cleaner output since it actually uses more precise PAR than in previous versions.
San Francisco - Bay Area
Thanks for replying.
No deinterlacing per se. There are two source materials in the same timeline and the degradation affects both:
1: A mpeg2 video (from DVD) at 23.976 progressive fps.
2: A raw 23.976 progressive fps render from AE which was itself originally a 29.97fps mpeg2 video from DVD which I painstakingly reconstructed in AE with reverse pulldown.
To compare footage (before and after Premiere Pro sends it through Media Encoder, whether for "raw" or mpeg2 encoding), I consult Premiere Pro's preview window, zoomed in to 400% so I can discern finer detail. The kind of degradation which occurs certainly doesn't require close inspection to be recognized.
I had considered for a moment that perhaps what I was seeing was a loss of chroma resolution, due, at least in the case of mpeg2, to the nature of the video format. But this wouldn't make sense, because all of the source video WAS originally mpeg2, so any loss of chroma resolution would already have been inherent. And besides, that would do nothing to explain why the supposedly raw renders undergo the same degradation.
one of the most commonly Mis-understood aspects of compression has to do with recompresion.
if a video starts from DVD mepg2 and is then edited, it must be recompressed upon export as the group of pictures properties (GOP) has been altered in the process of editing. The chance that the GOP will line up with the export GOP settings is near impossible.
This kind of compressing something compressed, (generation) will result in poorer quality exports.
Converting the mpeg2 to uncompressed before exporting to mpeg2 again won't make it any better at the export stage.
I hope this clarifies your results.
Jon Barrie ;-)
Thanks for the information. In this case, the considerations you point out were not completely misunderstood by me.
I point out that the degradation I am seeing is happening here:
Input: raw video (exported from AE as raw)
Output: raw video (exported from Premiere Pro through Media Encoder as raw)
No compression is inherent in the raw source. Hypothetically, no compression should be taking place in the raw export. The degradation I am seeing is NOT visible on the input file (raw from AE, remember) prior to export. Also I need to be specific in saying that the input file is not an AE project file but an actual raw render.
Hi Eric, can you verify the term RAW. It's not used in video. Uncompressed is commonly used but I'm not sure that what you mean. RAW is a format used in photography terms for uncompressed photos.
A raw 23.976 progressive fps render from AE which was itself originally a 29.97fps mpeg2 video from DVD which I painstakingly reconstructed in AE with reverse pulldown.
Did you rebuild an AE project based on an mpeg video as your reference? Or did you convert the mpeg video itself from 29.97 to 23.976?
What resolutions are your AE comps, clips and ppro sequences?
Jon Barrie :)
> can you verify the term RAW
My bad. "Raw" is my shorthand for "uncompressed."
> Did you rebuild an AE project based on an mpeg video as your reference? Or did you convert the mpeg video itself from 29.97 to 23.976?
Within AE, I imported the original 29.97fps mpeg2 video, did my edits (this involved performing reverse pulldown on different segments of the video, as the phase kept changing from scene to scene), and allowed AE to output the result as an uncompressed video some 130GB in size. 8 bpc, I do believe.
> What resolutions are your AE comps, clips and ppro sequences?
AE comp is 704x480 NTSC non-widescreen 23.976fps, as is its raw output which I use in PPro. The other source - an mpeg2 video - is 720x480 NTSC widescreen 23.976fps. The PPro sequence is its preset for DV NTSC widescreen 24fps.
In spite of the variability between the two source videos (widescreen vs. not, and differing resolutions), I again stress that no degradation of either video is apparent in PPro's video preview prior to the uncompressed render, yet BOTH sources get degraded in the same manner upon rendering.
Ok, so then the recompression situation is actually happening here.
the AE rendered file is not actually uncompressed data, it is an uncompressed version of the original mpeg2 compression.
In order for you to have clean export with progressive material you need to be in a progressive sequence in PPro.
If the DV preset used is interlaced then you export the sequence as progressive it will deinterlace it = loss of half the resolution. Then there is the render with preview files. If this is ticked then you will be using the DV compressed renders to then compress to whatever your export settings are going to.
If you are rendering out to Quicktime with the codec set to Animation 100% you will end up with a large file but not real gain in video quality. MPEG2 export from this will incur a quality hit, that is just mathematics of compression, and recompression. MPEG compressed to MPEG (generation) will always incur in loss of quality.
Can you upload a couple of still frame screen grabs for out comparison. If it is as simple as making a new seq in desktop mode, which i feel it will need to be, then copying and pasting the edit into it you should be home free.
Let us see the results it will help us to pin point the actual problem. :)
- Jon Barrie
Thanks for your help. I should have clarified that the PPro preset I used was 24fps (progressive).
I'll be happy to (attempt to) post a couple of screen grabs once I return home and have the project in front of me. I hope this won't prove too late in the day.
I rendered off four .BMP frames from my project:
1: From the mpeg2 source, before rendering as uncompressed
2: From the mpeg2 source, after rendering as uncompressed (reimported to Premiere Pro)
3: From the (mpeg2 29.97fps -> AE 23.976 uncompressed) source, before rendering as uncompressed
4: From the (mpeg2 29.97fps -> AE 23.976 uncompressed) source, after rendering as uncompressed (reimported to Premiere Pro)
On a wild guess, I decided to take these four frames and see how they looked in the rather more predictable (and preferred) After Effects. To my mild surprise, the phenomenon of horizontal resolution degradation was not apparent. However, there were still some subtle chroma & luma differences, underscoring the remaining fact that I still haven't gotten a truly uncompressed (and I mean identical) WYSIWYG render from PPro.
This begs the question: What is PPro's inherent flaw which causes it to misrepresent a frame of video in such catastrophic fashion? You give it an uncompressed or mpeg2 video from another source such as AE, and its Program Monitor window displays the frame correctly, but you give it video rendered with Media Encoder (mpeg2 OR uncompressed) and it somehow crushes the horizontal resolution _in the Program Monitor window_ but not in fact?
Please post your results by uploading to the cow reply post interface.
As requested, here are a couple of shots. Premiere Pro CS4's Program Monitor window, showing uncompressed video, before and after it gets re-rendered as uncompressed video with Media Encoder. It should be obvious which one exhibits the horizontal degradation I've repeatedly referenced.
I feel compelled to note once more that it turns out the actual files themselves are not degraded in this fashion. It is instead a perplexing flaw in CS4's Program Monitor rendering. The same flaw is not in AE, for example.
I also discovered something interesting which may give a clue as to why CS4's Program Monitor is failing in this manner: If I take the uncompressed video (the one spat out by Media Encoder), line up the image to reveal the horizontal degradation, and then move the entire image upwards one pixel (ie change the vertical Position to 239), the horizontal degradation in the Program Monitor disappears.
It looks like pixel aspect ratio is affecting the overall shape of pixels. The slight gamma shift in the two clips is probably with the gamma settings in after effects preferences. QuickTime movies have problems with gamma shift.
It is hardly noticeable and it's not somethingni would worry about. Butnif you can post the actual rendered out frames then we can use them for our own analysis. Export same frames to bmp at 32bit.
400% is highly critical. What kind of production are you working on? In my experience clients are much less critical than us tech heads and I doubt your client/s will notice the slight gamma shift.
DV resolution foe widescreen is stretched and looks worse on the program than on a tv so don't get too worked up over the program monitor ifnit looks right when
exported. This is not an error, this is the same across all digital video editors. Calibated tvs with clean realtime output cards is the way to assess grades, never trust the computer screen.
- Jon Barrie
> It looks like pixel aspect ratio is affecting the overall shape of pixels.
Certainly. But I no longer suspect the aspect ratio has anything to do with this strange display flaw. It seems to be specifically related to how PPro - the Display Monitor, at least - is interpreting video which requires reverse pulldown to be displayed.
> The slight gamma shift in the two clips is probably with the gamma settings in after effects preferences.[...]It is hardly noticeable and it's not somethingni would worry about.
I agree it's hardly noticeable, and certainly less so after MPEG2 has done its inevitable damage. But it still would be nice to be able to nail down an uncompressed export which is not only exactly what I started with but can also be used in some kind of MPEG2 encoder. I mean, before yesterday, I was taking it for granted!
> Butnif you can post the actual rendered out frames then we can use them for our own analysis. Export same frames to bmp at 32bit.
Done. Actually, Media Encoder is busy encoding my movie with the oh so lovely MainConcept codec. (Side note: It's taking over seven hours to do a two-pass encode - the maximum permitted by any Adobe solution, and I now strongly suspect why. Meanwhile, my other codec gave me its result after 3 hours and six passes. Pity its output turned out to be unusable.) And Media Encoder would not give me the option of pausing one render to make room for another. And of course CS4 lost the ability to simply give me one darn frame. Part of their famously hair-brained overhaul. So I had to render these off with After Effects. Hopefully this did not compromise things.
> 400% is highly critical. What kind of production are you working on? In my experience clients are much less critical than us tech heads and I doubt your client/s will notice the slight gamma shift.
This is a restoration of a movie for my mother. It cannot be purchased, despite being quite old and fairly well known, so I have had to use various sub-par sources to generate a hybrid, including a DVD transfer of a VHS tape. In spite of how it all sounds, the result is rather good. It is very true that my mother is unlikely to notice or even care about subtle inadequacies of quality. But two things. First, this will be playing on an 80 inch projection screen, so anything I can do to maximize the quality (outside of rendering the thing as something besides a standard DVD) is something to aim for. Second, I've already invested a lot of time in this project. Theoretically, minor tweaks should add inconsequential work to what I've already put into it.
You say you are working with a DVD source made from a VHS tape - do you have access to the VHS tape? As DVDs are so highly compressed, you'd be better off recapturing the VHS tape direct to the NLE system using an analog capture card, where you could capture as uncompressed, or at least a high data rate I-frame codec with 4:2:2 color. MPEG-2 for DVD and DV codecs are losing a lot of image info.
If that'd been an option, yes, I'd have done it. But the DVD was all I had.
As it happens... CCE SP can't deal with 24p video (it shifts the entire frame upwards one pixel and duplicates the second-from-bottom scanline at the bottom, and no tweaking can get around this). I had to go with Adobe's MainConcept mpeg2 codec, which only allowed up to a two-pass encode. The result was, predictably, rather horrendous. (But at least MainConcept didn't mess with my scanlines.) So... now I'm rendering the whole thing off as a 1080p Bluray.
Using the tried and true method of stacking the two clips/stills on top of each other and setting the top to opacity method difference with a Waveform monitor I see there is no difference in luma or chroma between the two supplied shots. So it's a trick of the eye and technically there is no difference.
Try it yourself. Photoshop & AE can confirm this method too. But PPro has the waveform monitor to further prove the technical end is a near perfect match. The slight value showing could be in the BMP algorithm or the conversion stage of the before and after. This can happen when blowing up low quality VHS 240/250 scan lines to a higher value as the pixels need to overlap the higher res grid which creates a variable subpixel value depending on which codec is used at export and then the subpixel can be reinterpreted differently as it never completely lived in a pure pixel from the beginnig. But this appears to have 0 RGB value through most of it, the occaisonal value pops up per pixel. Well within legal contrast and would never be noticed on a moderately calibrated TV/projector/plasma/LCD/LED.
Like I mentioned in an earlier post we can sometimes get too caught up in a minor thing we think is important and end up spending so much time on it only for the end result to look fine or not be any different at all.
Blowing it up to Bluray is not my recommended pathway as the pixels are futher being blown up and most BluRay players can up-res a DVD with internal hardware to a very high quality.
- Jon Barrie :)
> Using the tried and true method of stacking the two clips/stills on top of each other and setting the top to opacity method difference with a Waveform monitor I see there is no difference in luma or chroma between the two supplied shots. So it's a trick of the eye and technically there is no difference.
There is, actually. In that particular case, the difference is easier to see up close than it is by studying the waveform. But let's take a look at a different frame:
The waveform discrepancies are more apparent. And in the image itself, the nature of the borders of objects has been modified visibly. Could be argued that these are still subtle, but the point is that this was supposed to be an uncompressed render - a file format costly in size with the advantage of being the effective substitute for proper frameserving / Dynamic Linking / what have you. And as such, this is a failure.
> This can happen when blowing up low quality VHS 240/250 scan lines to a higher value as the pixels need to overlap the higher res grid which creates a variable subpixel value depending on which codec is used at export and then the subpixel can be reinterpreted differently as it never completely lived in a pure pixel from the beginnig.
This one is tough to buy, because the effect is if anything even more pronounced in frames that have already been rendered as uncompressed.
> Like I mentioned in an earlier post we can sometimes get too caught up in a minor thing we think is important and end up spending so much time on it only for the end result to look fine or not be any different at all.
And it's worth mentioning again that this kind of problem is indeed minor, particularly in this case where I'm just making some DVD, yet it would be desirable, if at all possible, to get the result one might justifiably expect: the minimization of unnecessary image degradation.
> Blowing it up to Bluray is not my recommended pathway as the pixels are futher being blown up and most BluRay players can up-res a DVD with internal hardware to a very high quality.
90% of my purpose in recreating the project as a Bluray is to defeat the considerable compression artifacting. In fact, that's the main benefit of Bluray as a whole: Fewer and smaller compression artifacts. (In my opinion.) I've already made my fair share of BDMVs out of SD sources, proving what was easy enough to intuit.
In this specific case, the need to take this approach was severely underscored. First, I had a good deal of difficulty with CCE SP, as it ended up being incapable of rendering 23.976 MPEG2 without modifying the frames. This left me with MainConcept 2-pass. MainConcept is trash. There's no kind way to put it. The resulting encode was essentially unusable. I'll give it points for not exhibiting the perplexing issues CCE SP had. Anyway, bypassing the whole MPEG2 headache is another 5% of the purpose in going Bluray.
The remaining 5% is that it gives me the opportunity to make the two separate sources (one being full frame, the other 2.35:1) better fit the frame edges. I had been keeping them pixel-perfect since it was just 720x480, but this meant keeping ugly edges, or blacking them out.
I'm not sure we've all defined our terms completely.
Regarding "uncompressed" -- there is a lot of leeway in the video realm for what this really means. 4:2:2 YUV video is termed "uncompressed", but the chroma samples are 1/2 that of luma samples -- that's compressed.
If you wish to see if Premiere (and NOT output codecs) is changing frame pixels, I would suggest using a truly uncompressed format, such as a TGA frame sequence in and out, and compare that. Premiere, as far as I know, works in RGB colorspace; most codecs for presentation will work in a variant of a YUV space. Something is bound to change a bit in the process. MPEG2 is a very lossy codec (it was designed that way), so it's not fair or accurate, in my opinion, to judge that output vs. "original" footage.
I had the same issue with loosing resolution in output files, and here is my story:
Upgrade both Premiere and Media Encoder to latest versions, and render with MAXIMUM RENDER QUALITY....
When exporting progressive footage from Premiere to Media Encoder Yu MUST make custom SEQUENCE which is PROGRESSIVE (NO FIELDS), because if you export progressive footage from default interlaced sequence, this thing with deinterlacing is happening....
Try, and you will see a big difference.....
Not sure how to do that, exactly. The sequence from which I export claims to be 23.976fps, and the video used in said sequence is all interpreted as progressive. If there are other (well-hidden?) variables I need to be sure to tick, I missed 'em.