ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

unexpected zip compression of DNxHD file

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Graham Macfarlaneunexpected zip compression of DNxHD file
by on Aug 9, 2012 at 4:03:59 pm

AE CS5 on Win 7 x64

I have been provided two QT files, A1 and B1 each 1080p ProRes (not sure if they are 422 or 4444).
Each is about 1.5 minutes long and around 2GB size.
AE CS 5 recognises both A1 and B1 as: apch, trillions of colours.
If I zip these ProRes files I only get a very tiny reduction in file size (less than 1%) as expected.

As a test I rendered out both files from AE to DNxHD without doing anything to the videos.

Settings used:

AE project set to 16bit

QT container
Avid DNxHD Codec
trillions of colours
Q = 100%

Codec settings:
709 colour levels
No Alpha
1080p/25 DNxHD 185 10-bit

The resulting files A2 and B2 both maintain a 2GB file size and visually are identical to the originals, however if I zip each video file separately (with legacy settings) A2 manages 50% compression and B2 manages less than 1% as with B1.

* Both videos have a similar ratio of action to non-action (not that zip should be able to take advantage of this anyway).
* The amount of grain is comparable in both too, although A1 / A2 have a slightly softer look to them as if a very slight blur has been applied to the source video.

Could this softness be enough to account for the better compressibility of A2 and if so why would A2 compress to 50% when A1 did not?

Any and all ideas welcome!

Many thanks

Graham Macfarlane
3D animator and VFX specialist
London UK


Return to posts index

keith mcgregorRe: unexpected zip compression of DNxHD file
by on Aug 9, 2012 at 7:50:45 pm

Your compression scheme is not listed, which software are you using? In mac, the standard right click and choose compress just throws out redundant and repeated code, providing in a smaller file. In video that is high quality and not GOP there really is no redundancy or repeated code so hence no real reduction. When I use stuffit deluxe I might get a 25% reduction when I really try to squish it. Again, with video we need that information and there is no repeating code so hence no reduction in file size.
If you change the depth to 8, millions of colors, then you'll see a reduction in both size and quality, but if you do a test can you visibly see the difference? Most peeps won't notice, and if it still looks good to the eye then mebbie the producers won't know either.

Here's an interesting read I found:
http://www.maximumcompression.com/data/jpg.php

Hope this helps.
-Keith

Reality? What did you make it?


Return to posts index

Graham MacfarlaneRe: unexpected zip compression of DNxHD file
by on Aug 10, 2012 at 10:09:53 am

Hi Keith,

Thank you for your reply!

I’m using After Effects CS5 on Win 7 x64.
I’ve attached a screen grab of the settings explained in my first post:


I had a closer look at all my DNxHD files and noticed that compared to the ProRes originals there is a very slight noise softening that is just appreciable at around 800% zoom. This fact though, should affect both A2 and B2 equally.

I also realised that video A contains a slightly higher proportion of areas of near solid colour (eg a clean painted wall) compared to video B (eg containing a bush). Could this really attribute to near 50% compressibility?


Below is a list of all the other compression tests I’ve run to try to make sense of this puzzling situation.

ProRes originals (I suspect they were originally 8bit but supplied to me saved as 10bit, unsure how to check):
* A1 = 1800MB (zipped: 1791MB) (approx. 0.5% compression)
* B1 = 2010MB (zipped: 1985MB) (approx. 1% compression)


DNxHD: (only 100 frames compressed for speed)
* A2: 8bit prj – output depth-millions – 1080p/25 DNxHD 185 8bit = 90.5MB (zipped: 49MB)
* A3: 16bit prj – output depth-trillions – 1080p/25 DNxHD 185 10bit = 90.5MB (zipped: 60MB)
* A4: settings as with A3 but 100f are from different point in video = 90.5MB (zipped: 41MB)

Strangely A4 zips better than A3. Both have locked off cameras but A3 has two moving actors in shot compared to A4 only having one actor moving in shot.


* B2: 8bit prj – output depth-millions – 1080p/25 DNxHD 120 8bit = 59MB (zipped: 58.5MB)
* B3: 8bit prj – output depth-millions – 1080p/25 DNxHD 185 8bit = 89.5MB (zipped: 88.5MB)
* B4: 16bit prj – output depth-trillions – 1080p/25 DNxHD 185 10bit = 89.5MB (zipped: 88.5MB)

Graham Macfarlane
3D animator and VFX specialist
London UK


Return to posts index


Walter SoykaRe: unexpected zip compression of DNxHD file
by on Aug 10, 2012 at 2:26:27 pm

The efficiency of lossless compression like ZIP will be very dependent on the source material. This is a stark contrast to lossy visual compression, which can be constrained, sacrificing quality as necessary to maintain bitrate.

Roughly speaking, DCT-based image compression (like DNxHD or JPEG) works as follows:
  • Convert the image to multiple grayscale channels (often spliting an image into luma and two channels of chroma, similar to YUV video)
  • Break each of these monochromatic planes into blocks
  • Transform the spatial pixel data into a frequency space for each block through a process called discrete cosine transform
  • Quantize (round off) the coefficients that represent the frequency data (this is the first opportunity for data reduction through eliminating visual data)
  • Re-order the DCT coefficients in a zig-zag pattern to increase the likelihood of getting a run of several zeroes in a row, and performs run-length encoding (basically saying 10x0, instead of 0000000000).
  • Huffman encoding, which means that the data is optimized so that frequently-occurring symbols are abbreviated to take up less space, and rarely-occurring symbols are expanded as necessary.


DNxHD is a highly-constrained VBR intraframe scheme. It is not intended to yield maximum compression of the signal, but rather a predictable bit rate. It's therefore possible that there would be further opportunities for compression after the fact, especially where there's repetitive data (perhaps that cleanly painted wall that appears largely unchanged in frame after frame after frame).

So basically, yes, it's certainly plausible that in a few cases, but certainly not all, you'll get huge efficiency out of ZIP compression for video.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Graham MacfarlaneRe: unexpected zip compression of DNxHD file
by on Aug 10, 2012 at 3:59:57 pm

Thank you Walter for the confirmation and the interesting info on compression!

Graham Macfarlane
3D animator and VFX specialist
London UK


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]