Ready to expand from 1 machine to 3
I have Mac Pro and an 8 bay raid connected to an Atto R680. I'm preparing an expansion to 3 machines with one being the server. The set up will be for an assistant to help prep incoming projects.
I'll be running Media Composer 7, and the majority of my footage will be DNxHD145 as well as ProRes 422 HQ.
Does each machine need a 10gbe card? Does the server need some 3rd party software to manage media flow, or do I simply need 10gbe connections and adjusted settings within File Sharing in the system preferences?
Appreciate any advice.
It depends how many streams you want to deliver at each client.
Based on the formats quoted, you'll be OK with 10gig from the server and gigbit to each client.
Be aware Avid on Mac is a tricky customer when it comes to AFP/SMB. It also depends if you want project sharing, or not. So you might consider purchasing software from somebody like SNS... I bet they'll be along soon enough.
If you opt to use SMB then you'll probably need Thursby Dave on the client machines.
You can email me if you have any Avid deployment questions :)
What kind of problems does Avid have sharing over AFP?
I do not want project sharing. When my assistant is done prepping, I'll import the bins into my project.
In my experience it depends on the MC and OS version. Some issues are worse than others.
I've witnessed a variety strange behaviours, from warnings when attempting to save projects (although these appear to be cosmetic), to weird playback/stability issues.
So I think its fair to say that enabling all drives seems to come with a few caveats... at least on Mac.
Disclosure: Its worth stating that I mostly work with netatalk (an open-source afp implementation). However, I've also seen these things crop up with bona fide Apple AFP over the years.
Overall I'm saying that you mileage may vary, so testing will be key, unless you buy software the takes care of everything for you :)
indiestor.com - "Avid project sharing, shared!"
Thanks again Alex,
Would you mind providing a small list of software options?
Hi Jack -
you are not going to have success with a simple shared network using AVID Media Composer. Period. AVID requires a metadata manager, or a volume locking software, like Studio Network Solutions SANmp, Tiger Technology MetaSAN, or Command Soft Fibre Jet, or Flavoursys Mint and/or Strawberry.
You can certainly buy a turnkey solution from countless companies that make AVID compatible shared storage systems, if you don't want to buy an AVID ISIS 5000. Facilis, EditShare, Studio Network Solutions, Small Tree, Tiger Technology, etc, etc. all sell TURNKEY solutions.
SO - what happens if you ignore this advice. You will go into media database hell. You will plug in all of your computers (with 1G or 10G) into a switch, and you will play back, and everything will work, and you will say "Bob is an idiot - this all works just fine". And then the first person will WRITE something to the shared drive volume (your 8 bay on the ATTO card), and ALL the computers will start to beach ball, and scan the media database, for 20 - 30 minutes or longer. And this will keep happening every time someone creates a render, or saves their project data, or digitizes new media onto the shared volume.
The way to avoid this is to go into your Settings menu, and have EVEYRTHING saved to your local computer (renders, project data, etc.) except for media, which everyone can read, but cannot write to.
"WOW - what a pain" - you say. Well, you can buy a working system from one of the companies I just mentioned that resolves this issue for you. The less expensive ones are volume locking (only one guy can write at a time), but the more expensive ones (and of course including AVID ISIS) can do it all.
But let me make this clear - if you think that you are going to simply plug in your 1G or 10G NIC cards into a simple switch, and instantly you will have AVID shared storage, without any issues, then you are in dreamland, and anyone that tells you otherwise simply has no idea of what they are talking about.
You can do it with GREAT care as to not write to the shared volume, or you can do it with special software from companies like SNS and Tiger Technology. Or you can buy a complete working system. But you are going to SPEND MONEY to make this happen. End of story.
Rescue 1, Inc.
I definitely need to be able to share storage that two computers can write to, but I don't need to share projects.
Any thoughts on using SNS XTarget combined with Xsan?
XSAN will not help you one bit.
SNS XTarget will load onto a DEDICATED server (like a Mac Pro, that has your ATTO and disk drive hooked up to it). You then load SANmp
admin onto this server computer. You then load SANmp client onto your client computers, and the SNS iSCSI initiator onto your client computer, and NOW you have a shared storage system. Of course, you have to have a fibre or 10gig card in your server, which connects to a QLogic fibre, or 10gig switch, and now you can use this switch to network to your client computers.
SO - what is your purchase ?
Mac Pro server (just a Mac Pro 3,1 or greater with OS X 10.6 or greater).
your ATTO card
your drive array
a 10 gig card in the server or fibre card in the server.
SNS SANmp admin
a switch (fibre or 10G)
for each client -
fibre card (if you are using fibre)
10G card (if you are using 10G)
1G native Ethernet port on your Mac
SNS iSCSI initiator
SNS SANmp client
see, that's all. WOW - it adds up quickly, and it's just not "plug and play". That's why buying a turnkey system (like a SNS EVO) is much easier.
I talk to people every damn day, and see posts like this every damn day. I just saw a post about a guy that wanted to spend 700 pounds on a Synology NAS so he could have shared storage. The bottom line, is that you are asking "what is the cheapest way I can do this" - what you will soon learn is that there is no cheap way to do this, and the aggravation factor is great. It is SO MUCH EASIER to buy a turnkey solution that will work for your AVID systems (or any other NLE that you have) than to try to piece this together.
If you use MetaSAN, you are going to use Fibre, with a QLogic fibre switch, and fibre array, not your ATTO R680 and 8 bay. MetaLan will be the same aggravation as SANmp. And Command Soft Fibre Jet is Fibre only. So you will need the fibre cards, fibre cables, fibre SFP's and fibre switch.
WHY CANT IT JUST BE EASY ? Because, that's why. This is why you see all these posts with Drobo, QNAP, and Synology, that dont' work.
Now, you can get QNAP and Synology to work (not with AVID, but with FCP and Adobe CC) - but they have to be in the RIGHT CONFIGURATION with the right 10G options from QNAP or Synology.
Anyway, people hate when I write stuff like this, just like our friend Greg got REALLY offended when I stated that he can't do shared storage with a 40 dollar Thunderbolt cable (and you still can't) - but EVERYONE (lots of people including you) are saying to yourself "there has GOT to be a way that I can do a cheap shared storage system that I can just plug in and have it work, so it will be easy, and I don't have to hire anyone, and I don't have to buy any of these expensive damn turnkey systems". Well, there ain't.
There are plenty of solutions out there - either you pay the money to have it done for you, or you sit there and suffer thru setting it up yourself.
Let me tell you a little story, Dr. Johnson. While video streaming is very popular, and widely used by "everyone", I personally don't know how to do it. So just this afternoon, I tried to do is (with Livestream, which is free). And after 2 hours, and creating my free account, I still can't get it to work. I am frustrated, which is why I am writing right now. But I intend to be able to CHARGE PEOPLE MONEY for doing this simple service, and I am sure it's simple, and I am sure that I am doing something very stupid, that someone knowledgeable could show me in two seconds, and I would say "wow, that's easy". But I am not willing to pay anyone to do this, so I will sit here and suffer, and figure it out (eventually - and hopefully). And once I accomplish this (I hope) - I will be able to "sell my service" to do this for other people. This is what it takes to setup this stuff (like XTarget) - you certainly can do it - you are a smart guy. You just have to put the effort out - this is not a plug and play process - with either XTarget, MetaSan, or Command Soft (all three will give you what you want).
Rescue 1, Inc.
I'm confused about one thing. I thought NSA handled all the metadata...
Thanks for laying out my needs...Now, do you think I can save some money by using my telephone cables instead of ethernet ;)
Well I didn't hear any suggestion of Drobo, so I think we can safely assume that won't be happening...
As I said right back at the top, if you want project and media sharing, then you will need specialist software for the job.
However, since this forum basically run by vendors/sales people, I'm not going to provide a list - they can set out their own stalls. (hint - I like the SNS stuff, but thats between you and them).
All this aside, if you simply want to provide a separate share for each editor, which sounds like your goal, then you can definitely use regular shares (with care).
My point was that there can be caveats using Mac MC with AFP/SMB.
Look at a company like EditShare. For the longest time they have been primarily SMB based. However, in order to make Mac clients behave they sell Dave licenses for each seat...
In my view your questions are very relevant to this forum, unfortunately its quite a ranty place to come and ask them! haha
indiestor.com - "Avid project sharing, shared!"
Hi Jack -
what is NSA ? Unlike all the other editing software on the market, AVID requires some sort of metadata management for each user. This is what AVID shared storage is all about. That is why you can get an AVID ISIS to work instantly, and this is why EditShare, and Facilis are excellent solutions, and other companies like Studio Network Solutions and Small Tree (with Flavoursys Mint) can do AVID shared storage.
As I said, you can do all of this and spend ZERO money, if you are willing to "follow some rules" and just keep your project and render data local to your edit station, and just share your media.
There are lots of people that I have run into over the years that say "can I use my telephone cables" - or "can I go to Home Depot to buy these cables" - when in fact Home Depot charges MORE MONEY than a professional company like Markertek to get professional parts. The moral here is that "everyone" wants to do this for free, and posts to find out HOW they can do it for free, without spending any money. well you can't.
Alex doesn't like the "ranty" attitude, because he (and countless others) want to go to a public forum to also find out "how to do it for free". And when they hear that they can't do it for free - or if you can't do it for free, there are serious limitations or work arounds - then they get angry and say "screw that ranty moron clown". This is no different than when you see people on other forums can't understand why they can't do uncompressed 4K dpx files on their MacBook Air - and if someone suggests that they need an expensive drive array to accomplish this, they get VERY upset.
What can I say. I could be silent, or I can tell you the truth.
Rescue 1, Inc.
Here you go Jack - tell me if you feel comfortable doing this ...
Installing indiestor is much easier than you might think! Regardless of experience, the following page will guide you through the process of setting up your own Avid specific storage server. Depending on the number of workstations you want to attach, suitable hardware choices range from a laptop or desktop, to a fully blown server chassis stacked with drives and hardware RAID.
•Suitable storage provision (hardware RAID recommended)
•Gigabit network connectivity (minimum)
Supported operating systems:
•Ubuntu Precise (12.04)
•Ubuntu Raring (13.04)
* We use Wheezy here at the indiestor lab as it seems to yield the best performance across the board. That said Ubuntu Desktop is a great starting point if you are new to Linux.
The following steps can be simply copied and run in a terminal window. You will need to have administrative privileges on the system to execute these commands.
Download and install the GPG key for packages.indiestor.com:
wget -q -O - http://packages.indiestor.com/indiestor.gpg.key | sudo apt-key add -
Add the appropriate indiestor repository for your operating system:
echo "deb http://packages.indiestor.com/apt/ubuntu precise main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb-src http://packages.indiestor.com/apt/ubuntu precise main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb http://packages.indiestor.com/apt/ubuntu raring main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb-src http://packages.indiestor.com/apt/ubuntu raring main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb http://packages.indiestor.com/apt/debian squeeze main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb-src http://packages.indiestor.com/apt/debian squeeze main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb http://packages.indiestor.com/apt/debian wheezy main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb-src http://packages.indiestor.com/apt/debian wheezy main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb http://packages.indiestor.com/apt/mint nadia main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
echo "deb-src http://packages.indiestor.com/apt/mint nadia main" | sudo tee -a /etc/apt/sources.list.d/indiestor.com.list
Refresh the cache, entering the password as required:
sudo apt-get update
Install indiestor directly from our repository:
sudo apt-get install indiestor
This command will automatically fetch and configure all required dependencies. A dependency is a software or library that another program requires to run.
In the case that a samba configuration already exists you will be asked if indiestor should overwrite the smb.conf file. You can ignore this note if samba is being installed automatically as an indiestor dependency. Be aware that if you choose not to overwrite smb.conf, the indiestor global and share directives must be manually added (as documented in the manual)
For both security and convenience indiestor allows only certain elected users to use the command line administration tool. You must add at least one user to the indiewheel group as follows:
sudo usermod -a -G indiewheel USERNAME-TO-ADD
That's it! Grab yourself a cup of coffee and familiarise yourself with our manual. There are a few post install tasks you will need to perform at each workstation you wish to connect such as enabling Avid 'alldrives' in the console.
If you get stuck please don't hesitate to get in contact, we'd be more than happy to help!
Rescue 1, Inc.
Well Bob, I must admit, I'm a little disappointed you didn't know that NSA handles all the metadata. I thought everyone knew that.
Alex, indiestor appears to be right up my alley.
[Jack Johnson] "Alex, indiestor appears to be right up my alley."
Well sadly it doesn't run on Mac, but Vidsharx does and its a great piece of kit! ;)
[Bob Zelin] "Here you go Jack - tell me if you feel comfortable doing this ..."
Well I don't disagree with the sentiment at all... and it is very true you have to be familiar with Linux to use indiestor.
Then again, since Jack doesn't want bin/media sharing, or run Linux.. I didn't bother to mention it.
Overall, best of luck with the project Jack. I'm sure you will land on your feet with this. Any more than a couple of streams to a couple of clients and I would bring the cavalry in...
indiestor.com - "Avid project sharing, shared!"
Alex is correct - Vidsharx is a great product and is being successfully implemented for small AVID installations of your size.
ps - I am so "fried" that I did not get the NSA and metadata comment until I googled it. I actually said "what the hell is he talking about - what is NSA". Well, what can I say, I am an idiot.
Rescue 1, Inc.