Copy all media from a project to an external drive
I am an AE and my company recently switched from Final Cut to Avid. So far there have been many things that have frustrated me about how Avid works. It is really blowing me away that Avid is considered the industry standard when everything I do involves a ton more time and steps then with Final Cut or Premiere. Maybe I just don't know what I am doing?
For example, I am working on a multi camera reality show. I create 720p proxies at DNxHD 50 of all my media to ensure smooth playback with my multigroups. But it is a nightmare trying to relink back to my online media after edit. The only solution I have found for a reliable relink is to add a unique Tape Name to every clip in my project before transcoding the proxies. I have hundreds of files in my project so this takes hours. I am blown away that I can't manually choose the files to relink to, and Avid is not smart enough to remember what linked source file was used to transcode the proxies to make it easy to relink back to them. And that is not even the problem I am posting this about!
Just when I thought I couldn't hate Avid any more, I come across this issue. All I want to do is copy all of the media from a project to an external drive to give to an editor so they can work from home. Not the linked media, just my proxy media which is all on my raid in the Avid Media Files folder.
I can use the Media Tool to locate all of the files I need on my raid, but when I select them all it won't let me consolidate them from the media tool window. I can only do that from the bin that the master clips are in. And since I have my footage organized into a ton of different bins, I have to go bin by bin to consolidate all the media to my external drive. In other words I can't just copy everything at once and leave the transfer going overnight. I have to wait for each bin to finish consolidating before I start the next, and I have well over 20 bins of footage so I will have to sit here all day and watch it so I can start another bin when one finishes.
I read that the old way people did this was click reveal file on each master clip and move those .MXF files to the Avid Media Files folder on the external drive. I have so much footage in my project that this would take me days!
I know there has to be an easier way to do this but I am not finding much when I search. There is the automatic duck media copy application for $99 but I refuse to believe I need to spend more money just to be able to move my media to another drive. With Premiere or Final Cut I would do the media management myself so making a drive for an editor was as simple as copying one folder from my raid and leaving it transferring overnight.
Somebody please make my day and tell me there is an easy way to do this that I have yet to discover!
[Ryan Dundas] "For example, I am working on a multi camera reality show. I create 720p proxies at DNxHD 50 of all my media to ensure smooth playback with my multigroups."
That's a fairly high data rate format...why that over DV or 14:1? Guess you don't have that much footage yet, or that much storage issue? Just curious.
[Ryan Dundas] "But it is a nightmare trying to relink back to my online media after edit. The only solution I have found for a reliable relink is to add a unique Tape Name to every clip in my project before transcoding the proxies. I have hundreds of files in my project so this takes hours."
Well, it shouldn't. Because you AMA link to the footage, highlight all the clips, add a TAPE ID, then transcode. Then repeat for the next import. This doesn't take that much time at all. Adding maybe 15-20 seconds to the process for each AMA group. FCP Legacy was slick because it added this right away when you Log and Transferred the footage. I do really wish Avid would do that too. But it, nor PPro, nor Resolve nor FCX nor ANY OTHER NLE does this. You need to manually add a source after you link.
There is no easy way to do this if you didn't set this up from the start. If you didn't plan for this. Because all of the media is in one folder...full res, proxies...footage across projects. So either you get into the habit of changing the numbered folders name every time you transcode to offline, so that they all go into one folder, and then change it again...there's no way to track it. I say get the Media Copy app. $99 is less than the time it'll take you to do this once. VERY helpful app
Little Frog Post
Read my blog, Little Frog in High Def
If I can give a word of advise..., I came from Decreet Logic Edit* background in the late 90's. We hired a new lead faculty in the early 2000's and she was trained in Avid. Needless to say you can guess where the college program went.
I found Avid utterly confounding. Worst yet, I was teaching this program (thankfully an Intro class) and the college offered absolutely no training. I was on my own to figure it out. And back then there wasn't much if anything on the internet. So, a couple of things you are going to have to accept:
1. Avid came from the 1980's. A lot of the processes, terminology etc. remain the same so as to not annoy the user base.
2. It's Avid's way or the highway. You have to (often begrudgingly accept) that you have to do steps Avid's way. When you input a footage it is assigned a number that resembles a licence plate number regardless of what file name you use. It uses a reference pointer to find that "number." Thus it can be nearly impossible to find that footage if you are looking on the hard drive itself.
This can be confounding to a user from other programs but in a large facility that may have multiple editors using the same footage off the server..., at the same time, it allows them to name the footage as they chose and it will still point back to the same captured clip.
If you don't accept these you will always fight Avid. When most people left FCPX they seemed to migrate to Premiere. Was the move by your company to Avid a well thought out plan? A lot of people hear, "It's the industry standard" and assume it is the best. Unfortunately making the the" best" for you (and the company) requires a lot of experience that isn't gained over night nor transferred from a previous edit program.
My apology for not having a direct answer. I never used Avid in the manner you do. We had a Unity server and that was challenging enough even using DV in/out. My best advise is to study up when you can and learn on the fly. In a away file management is a whole subset of working with programs like Avid.
Hey Shane, we went with that format for the proxies because that is what the post sup asked me to try and it seems to be working alright with our system. No storage issues yet, they are shooting much less footage than we expected.
I looked more into automatic duck media copy. With that I have to export an XML of a sequence, then it is able to find all of the media from that sequence and move it to a target drive. Still more steps than I want to do because I don't have all of my footage in one sequence.
But I was thinking if I do layout all of my footage into sequences and have those all in one bin, can't I just select all of those sequences and Consolidate them to my external drive? That will copy over all of the media used in those sequences correct?
I am just nervous because I am going to have to reconnect the editors sequences on our raid after they are done working. I fear that if I consolidate, the new media files it creates on the external drive will have different names or some other crap that makes them difficult to reconnect to the original media on my raid. I am testing this with some sequences now
So here is an update, but I still have some things I need help with.
After all else failed I payed the $99 for automatic duck media copy. The manual for the product says that I can select an Avid .avp project file, or .avb bin files, and the media will automatically be located and copied to the location of my choice. I tried selecting the .avp for my project and the transfer gave me errors on a bunch of files. Also it tried to copy over the high res footage that I have linked into the project. I only want to transfer the low res proxies that I transcoded from those linked clips in Avid.
So I tried going through and selecting all of the .avb bin files for all of the bins that I have my proxy media sorted into. I also added all the bins that contain my music and graphics. This time the copy completed without errors. So I moved my project folder from my server to the drive, disconnected from the server and tried to open the project from the drive. I was expecting everything to relink automatically since it copied everything over with the exact same folder structure, but nope! Apparently nothing is that easy in Avid. Some of the media was relinked, but the majority of it showed up as offline. I tried to select all the offline files and relink, but Avid will not relink to the files.
I checked the log file that Media Copy creates and it shows all the files from all my bins were copied over without error. So I went back to my original project on my server and located a clip that is not relinking. I click reveal file and then search for those .mxf files on the drive and I am able to find them under the same folders as the server. Avid is just not able to relink to some of the clips even though the .mxf media is on the drive. Why is this?
I tried deleting the media database files (.pmr & .mdb) in every folder on the drive then relaunching Avid but I still can not get the files to relink. And that takes me to my next question...
When I delete those database files and reopen Avid, I expect it to show the indexing files box with a progress bar, but it doesn't come up. And the database files are not being remade immediately when I open Avid. Then I left my computer sitting for about a half hour and they appeared. Is there some way I can force Avid to re-index my Avid MediaFiles folder? Or maybe there is a setting I have to select so that it is done automatically when I launch Avid? I looked under media creation settings but didn't see anything?
Let me start by saying I feel your pain after many years now I've finally reached something kind of like peace with Avid, and begrudging acceptance that since it still hasn't been dethroned and had not in all the many years previous to my starting, I will have to assume it's going to stick around. Honestly though, I still hold hope someone will offer something better that gets as much adoption, because the 'industry standard' moniker has allowed this software to get away with a LOT which is less easily forgiven of competitors. That said, part of the reason it's the industry standard is that it is the industry standard and standardisation in whatever form, even byzantine and unforgiving can be very beneficial. Everyone uses it, which has allowed the software to include abilities and features that would only have been heard of or conceived of by people doing things industrially and in response to user confusion and a lack of any real care for user experience Avid is allowed to simply demand greater skill in the user. That second point is both a strength and weakness, if you don't know how to use a feature in an iphone because it's confusing you don't get people saying it's because you're not 'professional' enough to know how to use an iphone properly and you should go read the manual. By that same token though, in an industrial setting it's more reasonable to ask that users be required to master often obtuse and impenetrable procedures and terminology because to an extent such mastery is required to actually be operating in that industry and certain things at that level are complex enough that they can't be made intuitive or easy and the software engineers are just better assuming the clients will have a certain level of training. They also use that as an excuse for just about everything.
Anyway diatribe over, while I'm not sure how to help much with where you're at since your update, if I read right your original problem was the inability to efficiently move media from one specific project on to a drive for an editor to work on externally. I assume that you have multiple projects using the same storage and that the media then is all jumbled together in to the Avid MediaFiles folder as avid likes to do. If that's not the case and you just have one project you could just copy and paste the Avid MediaFiles folder on to the external drive (just putting that out there just in case it's a nice easy fix like that I'm sure that's not the case for you though).
Moving the media from one project, to another drive using Avid
You seemed to be on the right track initially, I think you gave up too soon though. You mentioned you used the media tool, and presumably loaded all the media from your project and hoped you'd be able to select it all and consolidate to the desired drive. A reasonable assumption. In fact you actually can do it like that with only one more step. The media tool doesn't let you consolidate the clips you're right, but you don't have to go bin by bin either.
Don't worry, this doesn't move the clips out of their existing bins, they'll all stay there these are duplicates but still linkable to the original source when it's time for online and still referencing the exact same proxy media on disk. If you give the editor a copy of the Avid project, you don't even need to give them this bin, the clips in the existing bins will be online and linking to the consolidated media on the editor's drive. The only thing I'm not sure about is if the editor opens another editor's timeline, and attempts to find the bin a clip is stored in, whether it will find it, or whether it will attempt to look for the duplicate instead. That could be frustrating, you might want to test that. This at least avoids the need for additional software and is quite straightforward to do as it works pretty much like how you thought it would.
Avoid playing Avid's way if it gets your way
Interestingly, I didn't know this not able to consolidate quirk about the media tool until answering this, I'd always assumed I could just consolidate from there if I needed to but I haven't yet needed to, so that's something I'll file away for later. I point that out, and the fact that I'm unsure about what happens when you want to match frame a clip back to it's original bin if you use this method because I work in a way that circumvents the need to do that anyway.
I Use a method Shane was referring to earlier which considering how commonly it's used makes me think it's an example of something Avid should really just allow to happen as a user choice in their interface rather than a commonly known 'trick' that everyone uses. Anyway, what I do is to - at the ingest stage -
This method essentially 'freezes' an MXF folder once it's renamed. It means that the database file Avid created on ingest is readable by Avid, but no longer writeable, which means you can segregate the media however you like, as any further media created by rendering or ingesting, will go in to it's own '1' folder by default and leave your existing footage in its renamed folders alone. I tend to do my import batches by shoot date, but you can use any distinction you like, you might also want to add the project name to the folder name. A handy side-effect of this is that it also by default separates out media created during edit from media created at ingest because any editor you hand the media to will have an MXF folder full of folders whose names aren't numbers and so when their Avid has to create media (like when it renders), it'll create a '1' folder and so that folder will always be editor media, that can be handy too because if you want to, you can rename it something like 'EditorName_Renders' and give it to other editors to pick up where the other editor left off.
Relinking stuff in Avid (try not to do it)
Also, regarding your troubles with relinking back to online. I also empathise, Avid seems to enjoy making this process as difficult as possible and when it doesn't work it's always your fault for something you should have done. While again it's reasonable to expect professionalism and best practice of professional operators, this just doesn't seem to be nearly such a pain in competing software so frankly, it could be better, the problems seem as far as I can tell to in some way or another all stem back to the core nature of how to software works that is a hangover of the 1980s. They're unwilling to re-engineer the entire product and also while it does cause pain for many of us at this stage, some of how it operates still makes sense for a lot of tape or film based situations and Avid won't want to break how it's working for them.
However I typically don't have to worry about this too much (though recently I've been given a nice reminder of how annoying conforming is in Avid) for several reasons.