Buying shared storage ( a personal take for those who are diving in)
You have found this post because you need shared video storage, and like us, you are a Cow type of operation. In other words, not ILM.
You have a tight budget to consider.
We first invested in shared storage about 4 years ago.
We do a bunch of post-production work as part of our design business.
In terms of workflow, it was the best move we've made.
Our business has grown because we have been able to take on bigger projects and break them up so that many editors and graphic designers can work on them at the same time.
We started with our 'ghetto san' (search for this on this forum), which still works, because at that point , it was all we needed. Now we have invested in a grown-up San environment, which can deal with about 12 people. We do a lot of 3d and After Effects work and some HD editing, so the I/O demands aren't too full on. As I say, we are not a post facility, but our business depends on doing post work.
I have some words of advice in terms of buying shared storage.
1) If everything is on your shared storage device, you'd better have a great backup plan. Because if you lose your shared storage device, then everything is gone. So when you budget for your shared storage, you need to budget for this as well.
Until this happens, no-one will want to think about this backup plan, but you WILL have an issue at some point, so you need to buy this at the same time as your shared storage.
You are going to need a secondary disk-based backup that backs up your primary server nightly. And you are going to need a tape-based back-up too.
2) If you are doing deadline-critical work from your shared storage device, then your disk-based server is going to have to be almost the same spec as your primary server.
Otherwise you will be sending your editors home until you have found a new server for them all to work from and copied all that data from tape back to this server. That could take a couple of days.
Alternatively, you could copy it from tape to their local machines and wait for the ensuing mess when you need to archive the project at a later date.
In any case, the way to calculate how much to spend on this secondary server is as follows.
When your primary server goes down, work out the cost of the lost editing time in freelancer wages as you get things back up and running.
Now add your time, plus that of a colleague, trying to fix it/ calling IT people / buying makeshift solution. Now add the cost of finding a way of getting your rush job delivered without a shared storage device.
If you have shoot rushes, then add the time it takes to find the Lacie drive that they were inevitably delivered on and checking that your runner hasn't taken it home for their iTunes library.
3) Once you are using shared storage, all your editors will assume that you have taken responsibility for making sure their data is safe. From now on, they will not be making their own backups, unless you tell them to. So the responsibility is now on your shoulders. Your plan is their plan.
4) Your storage is most likely to stop working when you are closest to your deadline. This is the point when you are most likely to be stressing the system. You will be filling up the drives a little too high. You will have an extra person on the network. People will make mistakes and be in a more panicked state than normal. Your client will be sitting on the sofa.
Your primary server needs to be able to handle the biggest job you can envisage doing in the next 3 years. With it being 30% empty.
5) You need some proper IT support from a video specialist with in-depth knowledge of video storage, who can be there within 4 hours.
Not the in-house IT guys.
6) Your backup plan needs to be Disk-Disk-Tape. Your primary disk server backs up to a secondary disk server. This gets backed up to tape, which will be mostly unaffected by power loss, unlike your drives.
7) Instead of spending time on geeking out about SAN systems on the cheap, spend the same time on your business model and working out how you can get the money to pay someone to put all of this is for you. Unless you are planning on opening an IT business, you will not recover the time and effort that you put into building a cheap system.
It will cost you the same as buying the expensive one.
8) All shared storage systems use the same bits of technology. Drives, motherboards, RAM, operating system, etc. And then some custom software on top. You can build your own system for a fraction of the price that these guys are charging. There is a healthy premium for making this stuff work,and they charge it. It seems unfair. And yet...
Although we have successfully built 2 SANs that both work, we still bought a turnkey solution as our primary server. It has gone down from time to time. We have cursed our supplier. All of this will happen. But it is still better than the alternative.
9) This expensive system will be worthless in 3-4 years time.
10) Shared storage is STILL the best way to work.
When you buy shared storage, you need to buy a backup plan as well which may cost more than the server itself.
When your backup plan fails, your project fails.
If you are an IT whizzkid, then go into the business of designing shared storage rather than post-production. There is probably more money in it.
If you are a creative, then concentrate on working out a business model where your clients can be made to pay for the equipment properly.
This is great. Dual online or active nearline is never really discussed because of the perceived doubling of cost. Thanks for the reminder.
I'm curious, how many SAN integrators on this board install dual online or active nearline systems? Do you backup your SAN everyday? Personally, I've done none. I get my clients in the habit of putting all of their footage/media to LTO before it hits the SAN so there's a backup before work begins. Then the project files themselves are backed up multiple times during editing. Yes, it will take awhile to recover from LTO, but for many of my clients the cost/benefit of having a dual system isn't there.
Production Workflow Designer / Consultant / Colorist / DIT
this is a GREAT post Marcus !
But no one will listen -
and I have a terrible thing to post - look for the next subject.
Great post Marcus, could easily be a Cow article, blog post, storage FAQ.