Creative COW SIGN IN :: SPONSORS :: ABOUT US :: CONTACT US
SAN: SAN ForumSAN TutorialsApple Xsan Forum

Backup strategy for your SAN ?

Cow Forums : SAN - Storage Area Networks

<< PREVIOUS THREAD   •   VIEW ALL THREADS   •   PRINT   •   NEXT THREAD >>
Backup strategy for your SAN ?
by Christopher Tay on Jul 3, 2009 at 12:09:34 am

Hi all,

Now that it's alot easier to jump onto a SAN infrastructure as there are many options and cost is manageable, how much do you think about backup, both near-term and long-term ?

I'd like to hear how everyone approaches this as things can,and will, go wrong with the SAN even though there is RAID5 protection, resulting in data loss.

Cheers.

-chrispy

Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Chris Blair on Jul 3, 2009 at 2:54:21 am

We run a 4TB Apace vStor system with 4 edit workstations. Almost everything we edit is SD using DVCPro50, so it's plenty of storage for our needs. We also use a separate, shared LaCie 2TB ethernet drive array for projects. For backup, we simply added a LaCie 4TB ethernet drive array and then use Second Copy (very simple backup software that's $39) to back it all up every night. Second Copy runs from one of the workstations and it works like a charm.

All these drive array are rack-mounted in a big, cooled closet along with our 48 port managed switch, internet, routers etc.

Keep in mind, the 4TB vStor only has about 3.2TB available for actual use, and we segmented into 2 volumes, one 200GB for audio, and 1 2.5TB for video. We use the remaining space for other stuff.

We've been running this setup for 16 months and we're not even close to filling any of the drives. The backup is about 65% full, and the other drives/volumes are roughly 40-60% full.

We prefer to keep stuff as simple as possible and this is a simple solution that just plain works.

Hope that helps.

Chris Blair
Magnetic Image, Inc.
Evansville, IN
www.videomi.com

Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Bob Zelin on Jul 3, 2009 at 4:11:20 pm

the elegant solution is to use a Cache-A LTO4A tape backup system. However, at $7995 for a 800 gig tape backup solution, clients with 8 to 16 Terabytes don't seem to like that solution very much.

With the cost of 1 Terabyte SATA drives (raw) costing about $150 and under, the Wiebetech slotless removable SATA chassis (between $99 and $220) is a low cost cheap alternative to the CacheA.

And don't get me started about how "tape lasts 30 years". I put in plenty of Quantum DLT systems on AVID NLE's. How are you going to read a SCSI based Quantum DLT tape today on a modern computer system ?

Bob Zelin




Respond to this post   •   Return to posts index


Re: Backup strategy for your SAN ?
by Ben Estabrook on Jul 3, 2009 at 9:38:09 pm

Hi all,

I'm new to these forums and have loved reading through these posts. Bob, I especially enjoyed reading your tutorials about creating an affordable SAN solution. That has been a huge help to me, so thank you.

As for backing up data, does anyone have experience with Quantum's SuperLoader 3 in a production environment? It's a LTO 4 autoloader you can get configured for SAS with 8 slots for only $3,600. For those who want to go the tape route, it seems like a great option. (My friend is a media archivist and would ring my neck if I don't use tape for backup, so that's the path I'm taking, but I'm still debating which LTO drive to buy.)

Then there's the software side of backing data up. Someone recommended to me BRU PE (http://www.productionbackup.com/). It's about $500, targeted to the music and film production industries. Sounds cool, but has anyone here used it?

thanks!
Ben

Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Bob Zelin on Jul 4, 2009 at 2:49:46 pm

all interesting questions. I wish more knowlegeable people on this list would give their input, besides me, who knows very little about this.

Quantum handed off the entire LTO4A division to Cache-A, so how much longer Quantum products will be supported is yet to be seen. There are great threads on RedUser.Net on people using the Quantum LTO4. The SAS "package" comes with the LSI Logic SAS/SATA host controller card, and gets recognized by the MAC operating system - there are no drivers. You "see" the LTO4 in Retrospect or BRU. BRU (which I never heard of until I read the RED forum) apparantly has been around the Unix/Linux world for a LONG TIME - but since I am not in that world, I was only aware of Retrospect. BRU is apparantly teriffic, but Retrospect is popular (and at 80 bucks, certainly cheap enough). I understand that if you dont' get the kit, you are better off with the HP LTO4 and an ATTO R380 SAS/SATA host adaptor (althought ATTO told me that Quantum released new firmware to support the ATTO R380 card) - but I have not confirmed any of this, and with all the confusion about CacheA being the only company left really doing this - I dont' know what to say about the Quantum LTO line.

Where is Jordan Woods when you need him (or Mark Raudonis).

Bob Zelin




Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Ben Estabrook on Jul 8, 2009 at 2:06:19 am

Thanks for the insight about Quantum, Bob. In purchasing a LTO drive, I certainly hope the company that manufactures it will continue to support it for some time.

Good to know that Bru has been around for awhile, but darn, I didn't realize Retrospect is so cheap! Wow, I'll have to look into that, too. . .

I am enjoying the discussion on the rest of the thread. Such a supportive community here on Creative Cow.

thanks,
Ben

Respond to this post   •   Return to posts index


Re: Backup strategy for your SAN ?
by Bob Zelin on Jul 8, 2009 at 7:47:12 pm

if you are so hot for an LTO solution Ben, if I were you, I would call Cache-A (all ex Quantum employees), and ask them exactly WHO is going to support this hardware.

bob Zelin




Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Ben Estabrook on Jul 8, 2009 at 11:08:52 pm

I wrote Quantum about this and, for what it's worth, they said they are committed to LTO and they already have LTO 5 drives on the roadmap. I'll give Cache-A a ring too to see what they have to say.

thanks,
Ben

Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Mark Raudonis on Jul 4, 2009 at 3:35:57 pm

Chrispy,

You've hit the "hot button" for anyone entering the "shared storage" pool. Now that you've placed all of your eggs in one basket, what happens when that basket breaks? Mr. Z has outlined some of the LTO options and they are recognized by IT professionals as the most economical, viable choices for back up. I'd suggest that you choose one and use it.

The bigger your bucket, the harder/more expensive it is to have a complete back up in place. It really comes down to money. You can have a complete mirrored SAN off-site for 100% disaster recovery capability, but you've increased your costs by over 100%. Most post houses simply can't afford this. Video editors have an advantage over the IT world in that our original data (media) is coming off of an existing data storage format (videotape), and by definition, that is a backup. The project files are infinitely smaller, and therefore more easily backed up. We insist that editors back up their project files three places: locally (to their local drives), globablly (to the SAN), and mobilly (USB drives).

Our storage pool is too big to deal with LTO tapes, but we're using a reliable SAN (X-SAN) with dependable storage (Active Storage). It's been in operation for over 5 years now, and we've yet to lose any media (other than to operator error). Plenty of drives have failed, but that's what RAID 5 is for.

There are other pieces to this issue that contribute to your data integrity: Back up power supply, and most importantly, adequate air conditioning. Almost all of our drive failures can be traced to the few times that our AC has conked out.

Here's my take on it. Your original camera tapes/discs/chips are your ultimate back up. If you have a project that's close to the finish line with a deadline looming, you may want to back up just the media for that project. Unless you have a complete mirrored SAN, the time to restore from LTO tapes will take so long that you'll miss your deadline anyway. So, back up your project files regularly and to multiple places, but pick and choose what media you want to back up. That's just my opinion. I'm sure others will differ, but there is no "right choice", only what makes sense for your situation considering budget, deadlines, and "tolerance for failure".

Good luck.

mark



Respond to this post   •   Return to posts index


Re: Backup strategy for your SAN ?
by Dave Barnard on Jul 6, 2009 at 10:19:46 am

Hi Chris

A lot depends on your clients type of project and workflow - with film and animation projects there is less likely to be original media to go back to.

We have used the Superloader3 SCSI LTO3 with a couple of broadcast clients, it's not the most sophisticated tape library but does the job at a low price. We've seen some rather slow backup speeds from it, but the SAS LTO4 version should be a lot faster - see if you can get any real world performance data.

One of our clients used a Dell/IBM FC LTO4 library on a recent animation feature with excellent results - around 300GB/hr

BruPE is designed for for creating archives of media projects so doesn't have all the functionality of a network backup program like Bru Server. Should be a good tool in the right environment though.

Take a look at Archiware PreStore http://archiware.com if you are looking for network backup & archive, we've used it with good results

Dave Barnard
cinedigital
London, UK




Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Neil Sadwelkar on Jul 7, 2009 at 4:41:10 am

Chrispy,

Good to find you here.

I tend to agree that video, your original 'data', if off tapes can always be recaptured. That being said, if its a PAL film project with the average 50-100 hours of material, an equal quantity of BWF files, sub clips and FCP's fantastic inability to relink 24fps conformed clips. Recapture is not always an option.

Then there's tapeless media - AVCHD, XDCamEx, Red, Phantom, etc etc.

But most of this backups are really short term backups. Once the program is edited, the value of the original unedited media diminishes over time.

For my film DI clients, I advice a bunch of Firewire drives. If you're really paranoid, do a simultaneous backup to LTO3 - one reel per tape. In the worst case, the film is always there. Can be rescanned.

With LTOs, my observation is they're not like hard drives. You need to have the software that wrote it, just to read them. So, a Bru backup, needs Bru to restore. Retrospect needs retrospect and so on. Windows Backup (its a software built in to Windows) seems to be common across Windows systems.

Some tape libraries don't normally work like a cupboard full of drives but like a giant volume which is the sum of all LTO tape cassettes in them. So, if you need a particular file, they can get it. but only the software knows which tape or tapes it is in. You can't just write to a Library and then pull out one or more tapes and keep them away.

So unless the world comes up with a storage device that's as long lasting as film, video tape, and hard disk - in that order - we're going to pretty much take our chances with storage. Or, get two of them of two different makes... and still take our chances.

Neil Sadwelkar
neilsadwelkar.blogspot.com

FCP Editor, Mumbai, India.
Completely PAL.

Respond to this post   •   Return to posts index

Re: Backup strategy for your SAN ?
by Eric Hansen on Jul 9, 2009 at 5:26:17 pm

Neil wrote: "With LTOs, my observation is they're not like hard drives. You need to have the software that wrote it, just to read them. So, a Bru backup, needs Bru to restore. Retrospect needs retrospect and so on. Windows Backup (its a software built in to Windows) seems to be common across Windows systems."

you can look back through my post history in the various COW forums and see that i have talked a lot about SDLT and LTO based archive (not backup) systems. i personally don't believe in backing up a RAID-5 array such as Xsan. that's why its RAID-5. have a few spare drives around to hot-swap if a drive fails. i've never lost media on a RAID-5 because someone was always around to hot-swap when needed. be sure to show your staff how to do this procedure.

at a former facility, we used the SDLT-600a, an ethernet-based SDLT drive. it was slow, but it worked great for archiving projects and was accessible to all computers on the office network. it used FTP and all the employees knew how to use FTP. i had researched LTO options using BRU and Retrospect with an attached LTO4 drive, but what stopped me from getting it is that it wasnt as easy as FTP (everyone needs to know how to use this system and there were definitely a few people that would never be able to learn a Retrospect or BRU-based system), and it was only available on one computer. another thing that became a huge concern was whenever the SDLT drive went down, we would lose access to our entire archive. the few times the drive broke were very critical. i would only recommend a tape-based system if you can afford 2 decks to minimize downtime. that former facility ended up getting an LTO4 system for archiving their RED footage and its been working great for them. but i know that my counterpart over there is the only one that knows the system, and personally that would drive me mad. my goal has always been to make it as easy as possible for an editor to access old footage when needed as quickly and easily as possible.

i now recommend to my clients a bare hard drive based system, where you archive your projects to 3 different drives - 2 in-house and 1 out of house that you rotate on a regular basis. a tape library should be done in the same way. hard drives are extremely cheap and getting cheaper at a faster rate than tape. then get a hard drive bay like Bob suggested above. i have been using eSATA "toasters" at a few facilities and they have been awesome. you load 2 hard drives in the top like a toaster, copy what you need, and eject.

regarding the "30 year shelf life" that Bob pointed out, i'm sure its true. but you need to keep those old decks around. tape companies advance their technology by creating new tape systems. when hard drives advance, they usually keep the same interface. look at the last 20 years and compare the different tape formats against the different hard drive interfaces. its definitely easier to hook up an old hard drive than an old tape. but you have to remember that hard drives don't have the shelf life, so i recommend to my clients that they test their drives every 6 months and replace them when needed. thats why there are 3.

i've been talking about archive and not backup. for backup, i recommend that you have daily backups for your computer's boot drive, which in many cases contains all your current projects. i would go with a cloning system like Carbon Copy Cloner (Mac) because you can get back running right away. in my experience, i have never needed an incremental backup system because no one every says to me "hey, i need a file i accidentally deleted 3 weeks ago." its always been "crap, my hard drive died and i have to print this tape today and make FedEx."

as for the SAN, since its RAID-5, i wouldnt worry about backing it up regularly. its more important to create a workflow for dealing with your media before loading to the SAN, and archiving after the edit is done.

e

Eric Hansen, The Audio Visual Plumber - www.avplumber.com

Respond to this post   •   Return to posts index


Re: Backup strategy for your SAN ?
by Bob Herzan on Jul 13, 2009 at 6:57:54 pm

It all comes down to workflow and your budget on how best to protect your SAN. You can add a low cost NAS with RAID protection onto your network and push the data manualy, LTO4 with the appropriate cards and backup software is also a good way of doing it exspecialy if you want to keep your backup's off site or need to share them with your clients. I tried to add a quick image to this response so hopefully it will show up. If not you can contact me and I will send you over some additional detail and can hook up up with some of our customers in your area that are doing this already.

Bob Herzan
Vice President of Sales
Rorke Data Inc
952-829-0300

Respond to this post   •   Return to posts index

<< PREVIOUS THREAD   •   VIEW ALL THREADS   •   PRINT   •   NEXT THREAD >>


FORUMSTUTORIALSMAGAZINETRAININGVIDEOS - REELSPODCASTSEVENTSSERVICESNEWSLETTERNEWSBLOGS

© CreativeCOW.net All rights are reserved.

[Top]