Avid Media Composer and sharing media on a SAN
Can someone help me out here: I am coming from Final Cut Pro, and I don't really understand how Avid MC deals with media files. In researching a SAN solution I see references to things like "Virtualization for Avid” (metaSAN) and "Project sharing" (EditShare). I had been tending towards a metaSAN solution, but they only support the "Virtualization for Avid” on windows, and I would like to leave open the option to stay on Macs.
So I am picking up that Avid won't just link directly to the media on a volume, they require the media to be moved into a project file? And of course multiple stations can't share the project file directly, so these solutions above are a hacky translation layer that fakes out MC to thinking they are the only one touching the project?
I don't need full "project sharing", but I do need multiple edit stations linking to the same media, to avoid duplicating the media for each edit station (the whole point of a SAN vs. DAS yes?).
The world of Avid is not that easy unfortunately. It will not work on a SAN sharing media unless you use Unity or one of the products you note above, Avid's naming conventions are basically the same and sequential so two systems connected to the SAN would overwrite the other, not knowing which system is correct. and while you can read one to many, only one had permission to write to avoid corrupt media. There are better people than I on the who can explain the minutiae and nuts and bolts to SAN management.
I would add two other systems to the above, Facilis terrablock a SAN hardware/software solution and commandsoft fibrejet, a SAN management software, works on almost any SAN
Erik Freid | MediaSilo, Inc
207 South Street | Third Floor | Boston, MA 02111
t. 617.423.6200, m. 617.306.8632, f. 617.507.8577
Hiya David and Erik,
Here's an article from Walter Biscardi. - It's a nice post he did on the 7th that discusses (in part) some of this
Not sure if you are having the need to work on the same project at the same time as someone else. A lot of folks can break one project into smaller projects and each work on their own piece.
You may find some insight there.
You can use Avid Media Access to point at a given NAS volume and it "just works". There's no special action required. (and you get the added bonus of not transcoding and re-ingesting everything into Avid)
CTO, Small Tree Communications
The problem with AMA linking is that Avid strongly suggests transcoding. The AMA workflow is fine for basic assembly, but it's likely you won't be happy long-term with the performance of an AMA timeline, regardless of the storage system.
Avid uses a database file in a media directory that is owned by a single workstation. When multiple workstations write to the same drive, additional media directories have to be created so that the databases are unique for each workstation. This can be achieved to some level by tricks and virtualizations on the server side, but it's best to let the Media Composer do the creation, which it will do when the media directory lives on a Unity/Isis. This is the same for the bin locking/project sharing feature.
Facilis offers the native media and project sharing that normally only exists when using Avid SAN solutions. We do this through an emulation mode that presents our Shared File System volumes as if they are native Avid storage. It works, it's cross-platform Mac and Windows, and it's been shipping since 2009.
I think you'll find the Creative Cow forums are an excellent source of information on SAN and NAS solutions for the Media & Entertainment market. There are a number of options out there.
The TerraBlock is a turn-key shared storage solution which is cross platform (Mac, Windows, Linux), Ethernet and/or Fibre Channel, Virtual Volumes, Interoperable with third party NLE’s, Asset Management, and workflow tools, Pro Tools file-level support, no per-seat license fees, and Avid Media Composer, Apple Final Cut Pro, and Adobe Premier Pro Project sharing capability.
Working with Avid Media Composer, the TerraBlock not only allows you to share your media files easily, but also supports native project sharing. There's no separate abstraction layer. In an Avid environment, many artists utilize Project and bin sharing to speed up the workflow and add efficiency. The TerraBlock allows you to share bins and projects across multiple clients. When you place your bins and projects on TerraBlock Volumes, several users can work on the same project at the same time. Avid editors have a locking mechanism to help you keep track of who is currently working in a bin. This allows one user to write to a bin and multiple users can read the files in that bin. This ensures that changes aren't inadvertently overwritten to the bin.
Some of the key benefits of Facilis TerraBlock include Performance, Interoperability, Collaboration, Stability, and overall Value. Feel free to contact us if you need more information.
577 Main Street, Suite 120
Hudson, MA 01749
978-562-7022 x 111
Hi David -
everyone is being so nice and professional here. Well, of course, I will not be so nice and professional.
I know exactly what is going on here David - you want a cheap solution. You say "it's just a network - what's the big deal" - well, it's not just a network, and unless you buy AVID ISIS 5000, EditShare or Facilis Terrablock, you will either have to deal with workarounds, or you CANT HAVE SHARED STORAGE ! YOU AIN'T BUILDING THIS YOURSELF DAVID ! Do you understand me ?
Alternate solutions that work are using VOLUME BASED solutions - this means that you setup seperate partitions (or drive volumes) and only one guy can write to a volume at one time. This process works, and is successfully sold by Studio Network Solutions (SANmp) and Command Soft (Fibre Jet). Facilis until relatively recently, also used to only do Volume Based solutions for AVID, but relatively recently (a few years ago) they got file based working as well as EditShare. Other companies, like Small Tree and Maxx Digital are close to a working solution as well.
SO, why does this happen ? Because AVID writes a .msm and a .mdb file whenever you launch Media Composer, and if you are on one common volume or partition with two or more systems, the second (or third) Media Composer will overwrite the first guys .msm and .mdb file, and you will see MEDIA OFFLINE on your AVID on your edit 1 system. AVID (and others) get around this by using a METADATA SERVER to manage all of these files for each client. And companies that do VOLUME based solutions only allow ONE EDITOR to write to one volume or partition, so no one erases the .msm and .mdb file, because only one guy can WRITE to that volume. That's why it's called VOLUME BASED.
SO, why does AVID do this, and why doesn't AVID change this ? Because NO ONE WOULD BUY AVID ISIS OR UNITY if they made it easy, and would just buy cheap (or cheaper solutions). And brilliant companies like EditShare and Facilis have already figured out the successful solution for a file based workflow, and others (like me) are still struggling (but close) to a cheapo solution that works. And of course you can get a Volume based system right now from Studio Network Solutions and others.
but one thing for sure David - you ain't building this yourself to show your boss what a big hero you are.
Want shared storage for AVID Media Composer ? SPEND MONEY.
Hi Mr. Zelin, I was hoping you'd drop by and give your very valuable input. (This is a new account, but I have been lurking around the cow for quite a while, and plenty long enough to appreciate the help you have to offer around here, and your unique style :) )
Yeah, like I said in my original post, I am not familiar with the Avid MC workflow, and like anyone I want to hold on to my tried and true way of working, which with FCP is each editor has their own project, but all link to the same media. Steve mentions that with AMA that is posable, but as James brings up, that seems to be a not exactly recommended path.
You bring up an interesting point about volume level locking, and I would like to to explore that further: I have a been a little nervous about file level locking, and the need for a meta data controller as is, even if we stay on FCP (or switch to Premiere). It seems to me from the reading I have done that file level locking tends to be a bit more unstable then volume level.
The way we work now is generally with one station for ingest and a few edit stations, so I think volume level locking would be fine for us? The ingest can have write, and everyone else reads.
So in a Avid MC workflow on volume level locking, the ingest creates "bins" of new material, and the edits can all read those same bins? Can the ingest update those bins and the edit stations do some sort of refresh to see the new stuff? How can the edit stations pass back and forth the projects or timelines (in fcp terminology)? Like if we are working on a long form doc, and have two edits working on separate parts of the film, how do they eventually come together?
What would you recommend? Volume level or file level? If money was no object, it would probably be an editshare? I have no problem with spending real money on getting a solid solution, but like anyone, I am looking for the best value I can, meaning the most storage space for my money. But of course lots of storage that goes down all the time, or corrupts, or forces my editors to waste time in workflow is no good to me.
IF money was no object you would simply buy an AVID ISIS 5000 and stop this way of thinking. AVID is a file based system (and Unity from AVID - now called ISIS 7000) has always been this way. AVID ISIS 5000 and 7000 is the basis for what everyone else wants to be.
Facilis Terrablock outperforms the AVID ISIS 7000 system (and 5000), and is a teriffic product for AVID systems.
There is NO answer unless you say "I have X amount of money to spend for a shared storage enviornment". There is a young lady on the Arrays and RAIDs' forum who wants to build shared storage system for $2000 - $3000. It's not going to happen. You need to say "I HAVE THIS MUCH MONEY FOR THIS PROJECT" and then you can determine what you can, and cannot afford to buy. Stop thinking about the features that you need, and think about what you can afford to buy. Your ultimate decision will be completely based on this, because if cost was no object, you would buy an AVID ISIS system and be done with it.
An entry level AVID ISIS 5000 for 16 TB is $33,000.
Ahh, well I am sure giving you a number first is what you prefer as an integrator, but for me actually what I am looking at is features first, price second.
The problem is, like everyone else in post, I are not sure what happens after FCP7, but I am not ready to commit to Avid. However I do need a SAN now, (on FCP7) and so I am looking for the solution that works now, and will work with either Premier or Avid later. The reason I am looking at features first, is, I don't need full blown project sharing, I just need all stations accessing the same media. I am looking for the simplest way to do that (under the KISS principle, as well as for value). Yes I could get an ISIS 5000, and I am perfectly willing to do that, but I suspect it is overkill for what I want, and therefore fails on the value metric.
I am just looking for a little help figuring out how Avid works on the SAN, and particularly, what short of full blown project sharing exists. It seems there is a step in there that is volume level sharing, I just need a bit more info on the workflow there . . .
[David Parker] "The problem is, like everyone else in post, I are not sure what happens after FCP7, but I am not ready to commit to Avid. However I do need a SAN now, (on FCP7) and so I am looking for the solution that works now, and will work with either Premier or Avid later. The reason I am looking at features first, is, I don't need full blown project sharing, I just need all stations accessing the same media. I am looking for the simplest way to do that (under the KISS principle, as well as for value). Yes I could get an ISIS 5000, and I am perfectly willing to do that, but I suspect it is overkill for what I want, and therefore fails on the value metric."
I'm in the exact same position – FCP7 user that doesn't need project sharing, just access to media files – but I'm looking into transition to Avid so I could potentially need a system that Avid can work with too.
[James McKenna] "The problem with AMA linking is that Avid strongly suggests transcoding. The AMA workflow is fine for basic assembly, but it's likely you won't be happy long-term with the performance of an AMA timeline, regardless of the storage system."
What if I only worked with DVCPRO HD and Apple ProRes files after they had been transcoded/ingested from P2 cards? Or are you saying AMA just doesn't work well at all despite the codec? I feel like if AMA worked well with DVCPRO HD files then I'd be fine setting up any kind of SAN, but I'm probably wrong about this... right?
Hi Creative Cow management team,
these gentlemen are asking for advice on a shared storage system, that is pointing to an alternate solution to Facilis Terrablock.
You can see how uncomftorable of a situation this is. I have praised the wonderful Facilis Terrablock system in this thread, but these gentlemen are looking for an alternate solution - even though I have mentioned several brands in casual conversation - I am very uncomftorable saying "oh, you should look at this great solution" - when every post is being routed to the Facilis forum.
If this was on the SAN Networks forum, or even the AVID forum, I would be able to answer their question, but because it is routed to "everything Facilis", I cannot answer their question, or offer a solution, as I will NOT preach another product, other than Facilis on something that shows up on their forum.
It is my opinion, that this should be addressed.
What we are trying to tell you is that there is no way in Avid to just point to media on a SAN and just "use it" unless you have only one Avid on your SAN. It is not about project sharing, it is about SAN file access management (and Avid system database management)so you do not corrupt your media, or delete it all together. If you are using FCP on a SAN you are using X-SAN which is doing this for you. You need SOMETHING to do this that works with Avid (X-SAN does not, nor does stornext which is the same thing) If you want a cheap solution, Fibre Jet or SANmp both are between 1000-1500 per seat, and you need it on all systems accessing the media.
If money is not a concern then just get an Avid ISIS, Terrablock, Editshare, GraniteSTOR, SAN solutions, or another per-configured system that is known to work with Avid (and most will also work with FCP systems).
Regardless, a SAN is not easy to build, hire an integrator like Bob, Matt, or Steve, they will all do you right and save you money in the log run by avoiding downtime or dead end solutions.
Erik Freid | MediaSilo, Inc
207 South Street | Third Floor | Boston, MA 02111
t. 617.423.6200, m. 617.306.8632, f. 617.507.8577
Hello, I'd like to clear up a few mis-conceptions about our software and hardware solutions please.
SANmp is $499 per seat and it works with iSCSI or FC. There's also iSANmp for $199 per seat (iSCSI only). This software works with most 3rd party storage solutions.
EVO is our shared storage server solution. EVO comes with unlimited free licenses of SANmp. There are no per-seat license fees if you are an EVO customer. EVO also does SMB and AFP file sharing utilizing our "Share Browser" feature for managing cross platform permissions. Mix any combination of 1/10Gb & 4/8Gb fiberchannel to meet all your connectivity needs. Supports Mac, Windows and Linux. EVO also works with XSAN, Storenext, MetaSAN.
We have many referenceable Avid installs throughout the world. Give us a shout if you would like to learn more: Ph# 877-537-2094 or visit: http://www.studionetworksolutions.com/video-san.php
Studio Network Solutions
Throwing another hat into the ring - you also may want to look at GB Labs Space (http://www.gblabs.com/products/Space/). It is a high performance network appliance that works with Final Cut and Avid media (or Premiere, etc.), is very low maintenance, and costs less per TB than anything in its class.
In the US it is a fairly new solution (they are based in the UK, but have just opened US offices) and we only recently picked it up (as a reseller/integrator), but every place we have put a demo unit has resulted in the client keeping it.
A lot of our clients are in the same position as you, and there is no clear direction for current FCP houses to follow. So definitely make a decision that won't diminish your flexibility.
New Media Hollywood
Just giving my 2 cents... I work for Warm Springs Productions in Missoula Montana, managing 4 Facilis servers with a combined total of 240TB of shared storage. We just completed the transition from FCP 7 to Avid, and while there were definitely some growing pains, Facilis was a good choice for us, as it works great for both Final Cut and Avid. They even have some project sharing tools for Final Cut and Premier.
If you need help with workflows or migration from one system to another, feel free to drop me a line. It's not something you want to tackle on your own. kendallATwarmspringsDOTtv
Talk with Shane R. at Facilis, he may be able to direct you to a used unit at a very reasonable price.
Hi David! We are working on a software that will let you do what you are asking about. I can send you a BETA if you are interested?
I hope I am not violating any COW-rules by offering this...
All the best
Hi David Norden,
I am interested in your beta. Please contact me off list at
Hi Bob, I Will get in touch with you very soon.
All the best
I'd be interested in this as well - where can I contact you?
[Bob Zelin] "AVID writes a .msm and a .mdb file whenever you launch Media Composer, and if you are on one common volume or partition with two or more systems, the second (or third) Media Composer will overwrite the first guys .msm and .mdb file, and you will see MEDIA OFFLINE on your AVID on your edit 1 system."
False. Not gonna happen.
Avid updates .pmr file when you quit, and both .pmr and .mdb if you add or delete media files. (There's no .msm file at all) But that doesn't matter. Updating and overwriting doesn't result Media Offline. If .mdb file changes, another system starts scanning (which I must say may last forever depending on how much media you have in Avid MediaFiles folder). But after the scan is complete, that newly ingested or imporetd media becomes online for the second editing station. But .mdb changes also if you render(which means creating media) to the volume it resides on. And that makes collaborative work an endless scanning.
That's what gonna happen.
Or you need to render to the separate volume and avoid ingesting (or importing) during the editing session.
I am reading all of your recent posts with great interest, including your replies to Walter Biscardi on his testing with MC6. I understand that you are not seeing a lot of this with Win systems.
You state -
"If .mdb file changes, another system starts scanning (which I must say may last forever depending on how much media you have in Avid MediaFiles folder). But after the scan is complete, that newly ingested or imporetd media becomes online for the second editing station. But .mdb changes also if you render(which means creating media) to the volume it resides on. And that makes collaborative work an endless scanning.
Or you need to render to the separate volume and avoid ingesting (or importing) during the editing session."
Now, I do not have access to multiple MC6 systems, so I am reading all of this with great interest. Are you saying that if you let the second and third MC client system scan the shared volume, once the scan is complete, you will be able to read all the media without giving anyone "Media Offline " ?
And are you saying that if I create separate volumes for rendering for each editor (easy to accomplish), while multiple editors use this media from the common shared volume to edit with, I will be able to work without issue ?
As you can imagine, when multiple editors work, they need to ingest (or import) because they are on different jobs. But if they are on the SAME shared volume, this is unavoidable. So how would you get around this (without creating a volume based system - one volume per editor) ?
I look forward to your reply.
[Bob Zelin] Are you saying that if you let the second and third MC client system scan the shared volume, once the scan is complete, you will be able to read all the media without giving anyone "Media Offline " ?
That is exactly what I am saying. No offline media.
[Bob Zelin] And are you saying that if I create separate volumes for rendering for each editor (easy to accomplish), while multiple editors use this media from the common shared volume to edit with, I will be able to work without issue ?
You will be able to work without scanning. Unless you add something to the shared volume.
[Bob Zelin] As you can imagine, when multiple editors work, they need to ingest (or import) because they are on different jobs. But if they are on the SAME shared volume, this is unavoidable. So how would you get around this (without creating a volume based system - one volume per editor) ?
That's the point to argue. All of that depends on a workflow. I used to work at the company (companies) with a special "ingest team" who would ingest all the material prior to the editing (usually at night). Editors had no right to do that. They even had no possibility to do that as the editor's place was a keyboard, a mouse and a couple of monitors. And no access to the ingest room. Need to capture something - call "specialist", give him a tape or whatever you have and wait for your time in his "ingest table". Some kind of distribution of duties. A bit silly. But that's not a solitary instance, so I think we may call it a workflow. As a side note, all that took place in an ISIS environment. Thus, there are certain situations, when it is possible to avoid ingesting during the editing session.
But in case the editor ingests to the shared volume while he works, that's not gonna work. You gonna be watching Avid scanning the drive all day long. And that's not what an editor expected to do, I think))
Let's get back to [David Parker]:
The way we work now is generally with one station for ingest and a few edit stations...
...the ingest creates "bins" of new material, and the edits can all read those same bins?
Yes, they can read those same bins or if you don't want to open the same bin from different stations for safety reasons the ingest distributes that bin to editors' project folders (on a Finder level).
Can the ingest update those bins and the edit stations do some sort of refresh to see the new stuff?
Yes, you can either update those bins, but, at first, editors should close them if they have them opened (not so comfortable), or you can distribute additional bins to editors as mentioned above, or use the MediaTool in Avid.
But beware of scanning))
How can the edit stations pass back and forth the projects or timelines (in fcp terminology)? Like if we are working on a long form doc, and have two edits working on separate parts of the film, how do they eventually come together?
Simply by copying bins, containing the sequences and combinig those separate sequences into one.
Again, everything depends on a workflow. The possible solution here I think could be:
- Ingesting most of the media in advance;
- Each editor has a local (or mapped) drive for rendering, titles (they also generate files in Avid Mediafiles folder);
- In case the editor needs to ingest or import something - use that local (or mapped) drive as a temporary storage and consolidate to the shared volume at the end of a day (when there's no one "on the other side of a network" with the Avid launched) and "plan" scanning for the night time.
Of course, that's a work-around, but finally it works. For free.
One more thing:
To make it all even easier, there's Media Creation preference (orange box) in the settings list in Avid. Different tabs where you can specify the target drive (red box) for different types of possible media (green box). This way, you can automate the process at some extent and also exclude the chance of creating media on a shared volume. By the way, if it is a network drive that is shared (I mean, it is not a local drive of one machine, shared to another), then you don't even need to partition it. Here's how I see it:
SAN is the place "The Ingest" captures to. Editors have their personal folders (Editor_01, i.e.). Make them shared folders, then "The Editor 01" connects to his pesonal folder: Command+K, enter SAN IP Address/Editor_01 and he has that folder mapped as a network drive (green boxes). Therefore, Avid can use that "drive" to capture and etc (orange box). That is briefly.
And Bob, anticipating what you may say, I certainly understand that this is still a volume-based system, just put another way.
I appreciate your response, and your experience more than you know.
Thank you very much for your detailed answers.
I'm digging in a bit late in this thread as i've recently been discovering "the hardway" exactly all the issues you mention.
I've been struggling with 3 Avid MC systems hooked to a shared access point and endless scanning waits and the un-ability to see bins locked are evaluating our patience.
Whilst i await for response from the manufacturer of the brand who claims the system should work fine, your active response has proved priceless, so firstly, thanks for this. I find your turnaround really smart and creative.
Secondly i'd appreciate if you could go further for me and clarify, once at the end of the day, into which folder should we drop the rendered files of each of the networked participants in order to share the new media from the sharepoint again.
Thanks in advance,
In the suggested working environment you should drop newly created files (files, not folders) into "the shared" Avid Mediafiles location: let's say, for instance, "SAN -> Avid Mediafiles -> MXF -> 1".
Or you can create a numbered folder inside of MXF folder, i.e. "2" and copy the files there - thus making them easy to locate in case something goes wrong.
Or you can rename your local "1" folder into "2" (in case you don't already have a folder named "2" in shared place) and copy this folder into that same shared MXF folder.
Or you can use Consolidate feature inside Avid to gather new media to the shared storage.
Hope that makes sence))
But don't forget: It doesn't prevent scanning, but only makes it controllable.
Feel free to ask,
Hi again Eugeny;
Been testing your suggested workflow and does it's job fine.
Many thanks again.
I'm really glad it works for you.
Just chiming in to say "ditto" to everything David Parker has said as I'm looking for a similar solution for exactly the same reasons.
I'm in an educational facility and we do a mix of quick turnaround news pieces as well as longer promotional type pieces. Most the news/quick turn around stuff is shot on XDCAM and edited in Final Cut, but for the "higher end" stuff we can tap into the resources of our parent organization and shoot on formats like HDCAM/HDCAM SR, RED, etc. We'll edit proxies in house on Composer and take our projects to their editing facility for finishing on Symphony.
So a handful of Avid projects, with Final Cut being our current bread and butter.
So we might be moving more towards Avid in the future, but we're still very much a Final Cut shop...and this makes it hard to invest big bucks in a system like Isis or even Edit Share when our Avid usage is so modest.
Like David mentioned, I've read a lot on the Cow on this subject, including many informative postings by Mr.Zelin.
I recognize that a $5,000 solution will have different limitations from a $15,000+ solution. What I'm still unclear on, even after all this time, is the advantages, disadvantages, benefits, and shortcomings of a Volume-based sharing system. I don't feel like I fully understand what it makes possible compared to, say, moving a firewire drive around and what is still impossible compared to Isis. I don't really understand the full workflow from start to finish.
In Final Cut we have all three of our workstations working off a single network volume. Editors rarely work in the same exact project at the same time, but the network makes it possible for editors to use common media across multiple projects and brings some mobility to our editors as well (So Editor A might prefer Workstation A, but they can work at Workstation B or C should the need arise). Also, FOR US it's proven to be a lot easier and more reliable to keep a networked RAID array online and backed-up than managing an army of portable drives.
Since I've been following this thread, I just wanted to point out that OWC has made some exciting announcements in the area of low cost SAN this past week. I saw their setup at CES and talked to one of the people developing it. They seem to think that it's going to be a system with a low enough cost to get us cheapskates in the door (Starts around $5,000) while scaling up to meet larger needs too. Here's their product info page:
Hello David, actually im lookin for an alternative for ISIS. Dont need to work with avid projects simmultaneusly. Just Media Sharing. What solution did you use? any advice?
A good replacement is a solution from Elements. They have huge success in Europe and you can get those appliances in the US now as well.
It's a full replacement for ISIS without any emulation. So even if you don't work simultaneously, you could.
Official page: http://elements.tv/
oh no !
Another one - another competitor.
responding to a 2011 post.
In Europe, DDP, Tiger Technology, GB Labs all have wonderful solutions for you. And now, Element ?
Rescue 1, Inc.
Pretty certain GB don't have an option for native AVID sharing.
Open Source Shared Storage