Codecs and drive speeds
Dear Creative Cow members,
I am curious to know if there is a way to calculate how fast of a drive, and how fast of a connection you will need for specific codecs. For example, how can I calculate when FireWire, or a single drive is not fast enough? Is there a way to factor in color corrections and effects?
Thank you so much in advance for your posts.
Codecs vary wildly. They can be VBR (Variable Bit Rate) or CBR (Constant Bit Rate). There can also be many flavors of a codec (ProRes, DNxHD, for example) This means that any any given point in time, the exact data rate changes.
Most codecs have their data rates published, in terms of Mb/s (MegaBITS/second) This can be confusing, because drives are usually measured in MB/s (MegaBYTES/second). So, before beginning to compare - ensure you have the right numbers. (Quick tip - MegaBits/8 = MegaBytes/s)
HDDs can vary wildly. External firewire 400 drives, for example, may only hit 40 MB/s, whereas a newer SATA drive may hit 70MB/s +.
Also, what is the transport of the data? Firewire 400, for example, may not deliver enough bandwidth to allow all of the throughput your HDD can deliver. USB and ethernet are also guilty of this, bandwidth is additionally poor for sustained throughout. So, determine how your array is connected to your host.
When you start using various RAID sets, it gets more tricky. For example, 10 drives DO NOT equal 10x1 drive speed. RAID formatting also alters your results. RAID0, for example, gives you the most speed available. RAID5 cuts down on that performance by ROUGHLY 12%, but you gain redundancy in case a drive dries. This is why you always do some real world tests. Lots of research!
I usually take 75-80% of the max throughput of a single drive. I then "add up" the streams of my codec to determine how many streams I can get (as a ball park) within that 75-80%.
Example: If my speed test on the drive is 70MB/s, I have ROUGHLY 56MB/s available. If I were to use an Avid Codec, lets say, DNX145 (which is 145 Mb/s; so 18.125 MB/s) I should be able to get 2-3 streams from that one drive.
Many of us use the AJA System Test for test drive performance.
Again, this is a rough estimate, and YMMV.
Color Correction and Effects: Usually these are "baked in" (rendered) so it is essentially 1 stream of that video codec. If NOT rendered, then the effects/ color are being generated in real time by the CPU and application, so drive speed does not become as much of a factor (we then get into how fast the OS drive is and RAM you have...that's another dissertation.)
You've inquired about a topic that has many, many different angles and alone keeps some people employed.
.: michael kammes mpse
.: senior applications editor . post workflow consultant
.: audio specialist . act fcp . acsr
Hear me pontificate: Speaking Schedule .
Thank you. That is great advice.
AJA System Test is free, and it's what everyone uses.
You can also download AJA Data Calc, either on your MAC, or even on your iPhone, to figure out how fast your drives need to be, and how much storage you will need for that compression codec.
Pretty much every drive manufactured today can do >30MB/sec, which means that you can do DVCProHD, and every flavor of Apple ProRes except 4444, which is 41MB/sec, and unless you have an antique FW400 drive, almost every drive today works over 41MB/sec. A 55 dollar 1TB SATA drive in a MAC Pro will do over 120MB/sec today, so even dirt cheap drives can do pretty much everything you want. Not unless you get into multiclip, uncompressed HD, multiple layers, or 2K work.
I use AJA Data Calc on my iPhone almost every day - and it's free.