Benefits of shooting in DNxHD or DNxHR codec for post?
I am wondering what workflow people are using when bringing footage into Avid that was shot in DNxHD, and what are the benefits of shooting in that codec for post in Avid?
For the past few years we have been shooting our content with Atomos recorders that record in Pro Res 422, because this allowed us to take our footage direct to edit in Final Cut Pro. And although the size of the files are very large, it allowed us to only have to backup one version of all the footage, instead of backing up camera masters and transcoded footage.
We recently started using Avid and I figured I would get those same benefits when recording in DNxHD on our Atomos recorders. Can I just Link the DNxHD codec footage into the avid project and edit with it natively? I am going to be multigrouping 2-3 camera shoots, and I have been told it is unwise to make multigroups with linked footage because it will result in poor performance.
I figured the best workflow for DNxHD footage would be to Consolidate the footage into MXF files in the Avid MediaFiles folder, but then I have the consolidated footage on my server and I have to backup the camera masters. So in other words I have to have space for 2 versions of all my footage on my server.
If this is the case then I guess there really is no direct to edit workflow with Avid? If I can't work with the Linked footage then the only benefit I get from shooting in DNxHD compared to ProRes is that I can consolidate instead of transcoding which is a quicker process?
So why are people spending the money to upgrade their cameras to shoot in DNxHD?
[Ryan Dundas] "Can I just Link the DNxHD codec footage into the avid project and edit with it natively? "
Yes. DNxHD is the Avid native codec. You can AMA link and edit, or link and then consolidate (zero quality loss) to your media drive (this wraps it in the MXF wrapper, opposed to QT wrapper...zero quality loss).
[Ryan Dundas] "I am going to be multigrouping 2-3 camera shoots, and I have been told it is unwise to make multigroups with linked footage because it will result in poor performance.
Well, you'll get poor performance when trying to play back 3 streams full res unless you have a very fast RAID. That is one thing you'll need if you intend on full res multiclip. But yes, consolidating and making MXF might help a bit with performance...but the people who told you that mainly refer to other shooting codecs not meant for editing, like H.264, AVCHD, RED...and so on.
[Ryan Dundas] "I figured the best workflow for DNxHD footage would be to Consolidate the footage into MXF files in the Avid MediaFiles folder"
[Ryan Dundas] " but then I have the consolidated footage on my server and I have to backup the camera masters. So in other words I have to have space for 2 versions of all my footage on my server."
In the wise words of my post super..."Footage doesn't exist unless it is in at LEAST two locations." We archive the camera masters themselves in to locations...and THEN have the media we work with in another. And it would be VERY wise not to have the camera masters and the work media on the same server. Lose that server, lose everything. Archive the masters on another server, or get bare SATA drives and a drive doc and archive them to that (pretty cheap)...and MIRROR them so you have two drives with the archives. Or back them up to LTO. When you shoot tapeless, it's more risky than having tape...hard drives die.
[Ryan Dundas] "If this is the case then I guess there really is no direct to edit workflow with Avid? If I can't work with the Linked footage then the only benefit I get from shooting in DNxHD compared to ProRes is that I can consolidate instead of transcoding which is a quicker process?"
Well...you can. You can AMA link to the QTs and edit those...the codec is fine. It's mainly that you'll need fast drives if you intend on doing that. BUT...DO NO EDIT THE CAMERA MASTERS! Meaning...don't edit on the ONLY COPIES you have. Back those up...two locations, remember? This is why it's wise to Consolidate...then you have two copies. NEVER edit from your only copy of the camera masters. That is never advisable.
Heck, you can even keep shooting ProRes and consoldate in Avid. Avid supports ProRes...can even convert to ProRes MXF (pardon, re-wrap).
Little Frog Post
Read my blog, Little Frog in High Def
Hey Shane thanks for the knowledge.
When it comes to archiving, we always back up the camera masters on G-Drive external drives, and another copy on SATA drives using a dock like you mentioned. So we have it in 2 locations as well as on our server for edit.
For this project it sounds like I could link the footage straight from the camera media then consolidate it to my server instead of copying to the server then consolidating. That way my server doesn't need space for both versions and I will have my backups.
Shane my other question I have for you is; if I decide to edit with low res proxies for better multigroup playback performance, what metadata do I need to be sure is there so I will easily be able to relink back to my high res footage that I linked to transcode the proxies? I figured I could just relink using Source File name, but I read in another post that this metadata will be removed after making the multigroup so you have to use Tape Name?
I did a test where I multigrouped 3 cameras together using proxy files. Then I cut a sequence from the multigroup and chose Commit Multicam Edits then tried to relink back to my high res footage and I was having trouble getting it to relink. I was told I need to add a unique Tape Name to every single clip that I make proxies from? Is this correct?
[Shane Ross] "Heck, you can even keep shooting ProRes and consoldate in Avid. Avid supports ProRes...can even convert to ProRes MXF (pardon, re-wrap).
Just one note about the ProRes.mxf Codec in Avid. It's only available on Macs.
[Ryan Dundas] "I was told I need to add a unique Tape Name to every single clip that I make proxies from? Is this correct?"
Yes this is correct. The Tape name is either derived from embedded metadata or if there is none then it should be the same as each clips filename. My suggestion would be to use the free version of DaVinci Resolve (12.5.6, 14 is in beta at the moment) to create your OP-Atom Proxies. I'm guessing your going to do the offline edit in 1920x1080 so use DNxHD 36 for your Proxies. If your going to do it at 2K or 4K then use DNxHR LB instead. The most important thing is setting up the Conform Options in the Master Project Settings. Heres a screen shot of how it should be setup so that the Reel/Tape Name Column will be properly populated to assure accurate relinking to your original media. This setting assumes that the original media does not have an embedded Reel/Tape Name. Give "Embedding in source clip file" a go first and if the reel column stays blank then set as per the screen shot.
"MY MEDIA/PROJECT MOTO: If you think three copies of your media or project are enough.
Take a moment to place a value on them and then maybe add two more.
Hard Drives are now stupidly cheap. A RE-SHOOT AND YOUR TIME AREN'T."
[Ryan Dundas] " if I decide to edit with low res proxies for better multigroup playback performance, what metadata do I need to be sure is there so I will easily be able to relink back to my high res footage that I linked to transcode the proxies?"
Use Avid to link to the footage, and transcode to DNx36 or 14:1. Edit...and then relink to the masters when you lock.
[Ryan Dundas] "I did a test where I multigrouped 3 cameras together using proxy files. Then I cut a sequence from the multigroup and chose Commit Multicam Edits then tried to relink back to my high res footage and I was having trouble getting it to relink. I was told I need to add a unique Tape Name to every single clip that I make proxies from? Is this correct?"
Yes, you need to add a TAPE ID to the footage after you AMA link it, so it knows what is what. Many cameras restart names and timecode and that's VERY confusing.
Little Frog Post
Read my blog, Little Frog in High Def
Ok so it looks like there is no way to navigate to each file individually when relinking, like in Premiere or Final Cut?
All I can do is select what drive I want to search and what metadata to use to search?
[Ryan Dundas] "Ok so it looks like there is no way to navigate to each file individually when relinking, like in Premiere or Final Cut?"
Sure there is. Who said there wasn't? OH... you mean like LINK and then look for it in a window. No, sorry, Avid doesn't do that. Media is handled differently than PPro and FCP does. The way Avid does it helps it track it better....less change of wrong links
[Ryan Dundas] "All I can do is select what drive I want to search and what metadata to use to search?"
When you relink, you open the bin with the sequence, take the low res media offline...open the AMA linked footage bins (make sure the drives with the linked media is also connected) and relink to all open bins. And yes, Avid will do the rest itself.
Little Frog Post
Read my blog, Little Frog in High Def
For my test project I transcoded all of my linked 1080 DNxHD 145 footage to 720 DNxHD 50 proxies by changing the project format from 1080p/29.97 to 720p/29.97. I did this because when I left the project format 1080p/29.97 and tried to transcode to either Source 1/4 or Source 1/16, the only option it would give me for resolution was DNxHR LB MXF. I didn't want to work in that codec.
I think moving forward I will try to leave the project 1080p/29.97 and transcode to DNxHD 45 and see how my system handles multigroups with 6 or more cameras in 1080 instead of 720
And it sounds like I set the source info wrong for relinking. If I have to add a unique tape name to every single clip that is pretty annoying, I have easily over 100 clips from 10 different cameras. Not sure why it can't just remember the name and location of the linked file it was transcoded from.
Am I screwed if I don't add the Tape Name to the linked file before transcoding? Or in other words if I forget to add it and transcode a file, can I add the same Tape Name to the linked and transcoded files and expect a reliable relink?
I'm sure it is preferred to do it before transcoding but sometimes editors bring in footage when I am not around and I can see them forgetting to do this. If so do I have to re-transcode?
I would add a few things to widen the potential options you can pursue, though the answers you have are probably best practice.
One thing I'd investigate though is, if you do end up going with a workflow where you transcode anyway, then do you need to shoot at a DNX codec? It would depend on the camera, because there might be an argument in favour based on quality regardless of whether you save any time in transcoding, but that's something worth checking. If your specific setup confers no advantage in quality over a different acquisition format, and if you find yourself having to transcode anyway for better performance with groups then you may as well investigate what other acquisition formats the camera has and see if there are options of equivalent quality for less storage consumption, so only your edit server has to store the comparatively storage heavy DNX codecs, and you don't need as many G-drives.
You may also want to test a workflow where you shoot DNX, consolidate DNX and group the consolidated clips and see what kind of performance you can expect with your machines and server and get the people who'll also be cutting to test as well. Often the performance sacrifice, when the media is consolidated and re-wrapped is in my personal experience usually pretty minor and if it's subjectively acceptable to the cutters then you can potentially skip and entire offline/online workflow step and don't have to worry about potential conform issues later down the track.
Relinking in Avid, to anyone used to the methods used with other systems, is a pain but if all the planets are aligned then relinking to AMA clips in a bin does work and is the closest approximation to the approach of specifically designating a filepath as would be done on other software. It's very picky about this, sometimes it's like you're in court and Avid is the prosecution looking for technicalities and loopholes in your approach in order to deny you your request to relink. It's like it will find ANY excuse to come back with the old 'no clips were relinked' message. But I've found though that when using a workflow which involves offline to online conform, having DaVinci Resolve take care of the transcoding and using the steps Brent posted works very well. I may also just have been lucky that in the circumstances where I've had to do conforms, the crew were diligent with best practice and the camera was one that worked well with professional workflows so each filename was absolutely unique, reel numbers had been set to increment between shoots and timecode was free-run so in assembling my 'case' against the 'prosecution' there was lots of evidence. Therefore if you have sway over choices in production and if there is flexibility in camera settings and options, try to ensure that it isn't possible to get duplicate timecodes on a given shoot or duplicate names over the whole project.
Another thing to consider is what formats can your hardware work with at an acceptable level of performance, including with groups? I mention this because, sometimes it's possible to transcode to an Avid editing codec that is of acceptable quality for the deliverable specifications, and also at a level of quality as to be essentially no quality loss from the originals. In such a case your transcodes can be your online media and there's no need to conform. This is good in situations where the camera only records compressed formats like h264 and where it's not particularly high spec anyway since in such cases you have no choice but to absorb the time and storage costs of transcoding anyway, but you can at least avoid the conforming stage. Hopefully in your case, your testing would find that the proposed flavour of DNX for acquisition is one the your hardware can work well with for your circumstances because then a simple consolidate workflow is your best choice, and there'll be no need of a conform.