FORUMS: list search recent posts

Long Term Storage Solutions for Red Camera

COW Forums : RED Camera

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Eric Weiner
Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 12:20:09 am

Hi, first post! (Long time searcher though.)

Hey, I work at a small production company, and we are just starting to use the Red Camera. We do a lot of work where we have to go back and find footage years later. Up to now we have used a combination of Hard-drive backups, but with all footage being "final" backed up on their shot tapes (kept in huge firesafes) in-case disaster strikes.

So we would have a working HD, a backup HD, and all footage kept on tapes. I also keep our FCP project files backed up in about 5 different places. (So we can just recapture tapes from with-in FCP)

However, red footage isn't on a tape.

What is the best long term storage solution? Part of me thinks buy a huge raid, but what if the building burns down, or the Raid Fails? I have also thought about LTO tape backup (we have some already) but that has it's own problems. Will the technology really be around in 2-5 years? I don't have an unlimited budget, but what is the safest way to keep red footage around for years? There is too much footage and my internet connection is to slow for off site server.

Do I have to do a combination of all of these, or is there a good reliable system out there for large file backup?

Thanks!


Return to posts index

Noah Kadner
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 3:12:35 am

Most people go LTO or hard drives in a vault.

Noah

Check out my book: RED: The Ultimate Guide to Using the Revolutionary Camera!
Unlock the secrets of 24p, HD and Final Cut Studio with Call Box Training. Featuring the Sony EX1 Guidebook, Panasonic HVX200, Canon EOS 5D Mark II and Canon 7D.
Watch Formosa- My indie movie shot with the SDX900 and finished with Final Cut Studio.


Return to posts index

Russell Lasson
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 3:37:10 am

We've just started using the Cache-A LTO drive. I've got to say that it's very handy for backing up anything, from R3D files to full FCP projects.

Nothing will be completely future proof, but LTO is generally considered to be the best option for now.

-Russ

Russell Lasson
Colorist/Digital Cinema Specialist
Color Mill
Salt Lake City, UT
http://www.colormill.net


Return to posts index


Noah Kadner
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 4:17:14 am

Until we get those lovely holographic memory units from Star Trek...

Noah

Check out my book: RED: The Ultimate Guide to Using the Revolutionary Camera!
Unlock the secrets of 24p, HD and Final Cut Studio with Call Box Training. Featuring the Sony EX1 Guidebook, Panasonic HVX200, Canon EOS 5D Mark II and Canon 7D.
Watch Formosa- My indie movie shot with the SDX900 and finished with Final Cut Studio.


Return to posts index

Arnie Schlissel
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 5:39:33 am

[Noah Kadner] "Until we get those lovely holographic memory units from Star Trek..."

Alas, it seems that we'll have transporter beams first...

http://www.engadget.com/2010/02/08/inphase-out-of-business-assets-seized-fo...

Arnie

Post production is not an afterthought!
http://www.arniepix.com/


Return to posts index

Noah Kadner
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 3:36:56 pm

Sadly I don't think we'll ever have time travel. I mean if we did- wouldn't someone have traveled back to let us know already?

Noah

Check out my book: RED: The Ultimate Guide to Using the Revolutionary Camera!
Unlock the secrets of 24p, HD and Final Cut Studio with Call Box Training. Featuring the Sony EX1 Guidebook, Panasonic HVX200, Canon EOS 5D Mark II and Canon 7D.
Watch Formosa- My indie movie shot with the SDX900 and finished with Final Cut Studio.


Return to posts index


Eric Weiner
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 8:13:37 pm

I feel a little better to learn other people are using LTO. I have a VXA-2 drive and library combo from Tandberg data. But the software that came with it, Retrospect, feels a little, 2003. It crashes a lot and doesn't seem to be very reliable. What programs do you guys use to interface with the LTO?

Also I'll look into Cache-A LTO. I've never heard the term before. Right now the largest tapes we can purchase with our current LTO are 160 (320 with compression). How do you guys feel about LTO compression? Does it actually make Red Camera files and projects smaller? Has anyone had trouble restoring from an LTO tape with compression? Does the compression come from the program used, or is it hardwired into the LTO tape backup. (I'm worried I'll compress a tape, and then later have a new LTO machine and not be able to get my files back.)

One more question. Drives in a vault sound like a good idea, but is there anything I should do to make sure that the drives are in good shape after a few years? We have a lot of footage that we have to go back to on a regular basis even years after it was originally shot.

Also anyone have any experience with backup up to a large Raid 6 drive array? I feel like that would last much longer because I could swap out failing drive slowly. I'm thinking something like a Raid 6, and a mirror of that raid as well. Does that seem like overkill?

I'm hoping in the future, we get huge flash storage drives. That seems more stable over a long period to me.

Thanks so far guys!


Return to posts index

Uli Plank
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 8:48:05 pm

Harddisks in storage should be started at least every 3 months or so to keep the bearings from getting stuck. Plus, you should copy them after a few years, opinions vary between 2 and 5, since magnetic information deteriorates over time.

Director of the Institute of Media Research (IMF) at Braunschweig University of Arts


Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 10, 2010 at 10:45:52 pm

I don't backup Red files, but we Back up all of our tapeless (p2) and original created media with a Cache-A LTO4 drive. LTO4 is 800GBs native (1.6TB compressed). You can't really count on the compression though (or calculate it) so just going with the native size is best. The interface is via a browser as the Cache-A has it's own web server built in. It's very easy and done all over IP if you wish. you can also transfer files to the unit over afp or smb (essentially like a network drive). What's nice about the Cache-A is that it has a searchable database for when you need something off the shelf. Whatever you have archived in the unit, it will be on the database, even when the tape is across the room or across the street.

Is it the most wicked fast thing in the world? No, but it does transfer at about fw800 speed. IT's ironic to archive all of our tapeless media back on to tape, but at least the time has shifted away from capturing and on to archiving. Archiving can happen in the background without taking up time on any other production machine.

Cost of entry can be significant, but the tapes themselves are <$40 so once the unit is purchased, the media is cheap. What's nice about LTO is that two revisions are built in to the spec, so your LTO4 tapes will work in an LTO6 machine whenever that time comes.


Return to posts index

Illya Laney
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 3:08:59 am

Why don't you back up RED files?

Motion Design, Color, Editing
Simulated Wood Grain Cabinet Inc.
Bunim-Murray Productions


Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 3:16:14 am

If we shot RED we would, no question.


Return to posts index

Illya Laney
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 4:29:48 am

Oh, I misunderstood you. Haha. I thought you meant you just didn't bother backing them up.

Motion Design, Color, Editing
Simulated Wood Grain Cabinet Inc.
Bunim-Murray Productions


Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 7:34:31 pm

[Illya Laney] "Oh, I misunderstood you. Haha. I thought you meant you just didn't bother backing them up. "

Oh yeah, we just toss those in trash. Who needs them??? We have everything we need after we log and transfer to a precious Quicktime movie. pfffff! ;-D


Return to posts index

Illya Laney
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 3:14:24 am

Our studio backs up everything to Blu-Ray.

Motion Design, Color, Editing
Simulated Wood Grain Cabinet Inc.
Bunim-Murray Productions


Return to posts index


cowcowcowcow
Bob Zelin
Re: Long Term Storage Solutions for Red Camera
on Mar 11, 2010 at 8:35:21 pm

you guys are all such big shots, advising Eric to backupt to a Cache-A LTO4A. Have you told him that this costs $7995 ? Many people faint when they hear this. And your old Quantum/Tandberg is now "obsolete", and believe me, we will look back on the Cache-A in 5 years and laugh (probably sooner than that).

Becuase SATA drives are SO cheap, a very simple, easy alternative is to create MULTIPLE SATA copies. SATA drives (like Hitachi 500 Gig) are under 60 bucks, and at that price, you can buy a LOT of drives for $7995. And as for "spinning them up every three months" - this is a little extreme. When Apple and others do a production run of their computers in China, or when Seagate, Hitachi and Western Digital do a production run, these drives sit in a hot warehouse for A LONG TIME before they ever see a computer - a lot longer than 3 months. And you know what - most computers actually work when you first turn them on (and they have been sitting around for a lot longer than 3 months before you got them).

Chassis from companies like Wiebetech make it very easy to pop in and out raw drives.

No one commented on Retrospect. The Cache-A (which is the ex employees of Quantum) use a web GUI that is built into the Cache-A to run it - you DO NOT use Retrospect or Bru with this product.

Bob Zelin




Return to posts index

Eric Weiner
Re: Long Term Storage Solutions for Red Camera
on Mar 13, 2010 at 12:24:27 am

Hey, Bob, I'm sure you don't remember me but we've talked on the phone before. (We have like 5 Evo2 Raids.)

I'm thinking your right. I don't have too much trouble with hard-drives, even some of our older stuff that we keep on HD's still works. (like 4 or more years.) I still like the idea of LTO tapes though, maybe I'll we'll end up doing both, but just sticking with our current crapy LTO-2 that we already have. And again part of me is thinking buying another huge Raid as backup. (Which I'm sure Bob Likes. ^_^)

Yeah, cloud is out, I'm talking lots of TB, and our internet connection sucks. (100 MB ~ 30 minutes for upload). We used to have Fiber, and then we moved are are stuck with dsl. I"m sure you can all imagine how painful it is to for from 100 Mbs up and down, and then go to 7 Mbs down and 700ish kbs up.

I was kind of hoping there was some magic technology I hadn't heard about yet, but it looks like there is still room for improvement for LONG term data storage.

Thanks everyone!


Return to posts index

Mark Perez
Re: Long Term Storage Solutions for Red Camera
on Mar 12, 2010 at 5:27:11 am

Its really Simple, Data is what you value you it at. Just do a simple ROI on your footage. Then ask your self, what can I NOT afford to spend.

You probably have a decent internet connection, think cloud, 1TB of storage is less than a few hundred per month, and its secure, and accessable anywhere, costs will go down over time, speeds will increase, and technology will never be outdated

Try i365.com its a seagate company, in a tier1 data cloud

save it, write it off on taxes, be safe, access anywhere.

if you want some help, mperez@bdata.com





Thanks
Mark


Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 12, 2010 at 3:55:54 pm

[Mark Perez] "You probably have a decent internet connection, think cloud, "

It sounds serene and ideal. We have over 46TBs sitting on the shelf, and amassing more every day. There's no way we could cloud it all and have it be cost effective. I'm sure one day, but that's a long way off.

Jeremy



Return to posts index

Bob Zelin
Re: Long Term Storage Solutions for Red Camera
on Mar 14, 2010 at 10:32:41 pm

Jeremy is correct. Everybody is a big shot. "just get an LTO". What is 800 Gig backup going to do for you today. There is only one way to backup 46 TB, or 64TB or 100 TB - BUY REDUNDANT STORAGE. Or you just backup your individual projects. Banks and insurance companies have REDUNDANT storage in multiple locations. LTO4A is a fantasy, for those people that have lots of time on their hands, and not much media to backup.

Will SSD take this over - of course it will, but it's expensive, and today, you deal with SATA or with LTO4A. How many people on this forum have tried to recover Quantum DLT from AVID media on a SCSI interface recently. That was less than 10 years ago. GOOD LUCK retriving this kind of media. You will have better luck with a 1" videotape.

Bob Zelin




Return to posts index

Russell Lasson
Re: Long Term Storage Solutions for Red Camera
on Mar 15, 2010 at 9:14:03 am

Bob, you're right that LTO isn't for everyone. You're also right that ten years from now, restoring an LTO4 is going to seem out of date compared to what's available even at the local electronics store. Things change quickly and I think you'd agree with me that people shouldn't invest in equipment that will not make them money.

While LTO isn't for everyone, there are some instances where it's required and makes a lot of sense. If I'm a "big shot" because some of our clients require LTO archiving, or because I like the idea more than hard drive only archiving, then great. It's part of the cost of doing business. When it really matters, redundancy and quality of archiving is really what's important.

I will say that I think that the cost of the Cache-A units are high. But when we started looking at other solutions, the Pro-Cache made quite a bit of sense for our situation. There are other LTO options out there, but many of them require proprietary software to be read. Cache-A units use industry standard TAR to ensure that compatibility is maximized. The Cache-A units also don't require a Linux geek to operate. It's really just plug and play (as much as plug and play LTO will ever be). When comparing the real costs of some other systems, I felt like the Cache-A units made sense for us.

While the Cache-A units might seem expensive, it's really just a fraction of the cost of what we used to spend on decks for making masters in the past. Buying a $30,000 HDCAM deck or a $100,000 HDCAM SR seems laughable for archiving when keeping a ProRes file is superior in so many ways now days. $8000 to expand and archiving system seems reasonable compared to those options.

Here's a little math just for kicks too.

If I had 50TB of material that needed a redundant backup, here are some cost comparisons. If I bought 100TB of hard drives (using 1TB drives at $80 each), that would cost $8000. If I bought 100TB of LTO4 tapes (using 800GB LTO4 at $35/tape), that would cost $3500. If you add in the cost of a Cache-A unit, then making LTO4 tapes is only $3500 more than using hard drives. I'm not sure cost is really that big of an issue anymore. Nor do I really feel like a "big shot" for spending $3500 more to archive to tape instead of hard drives.

-Russ

Russell Lasson
Colorist/Digital Cinema Specialist
Color Mill
Salt Lake City, UT
http://www.colormill.net


Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 15, 2010 at 4:02:05 pm

That's the same math we came up with, Russell.

In fact, here's from another recent post in the FCP forum that I wrote:

http://forums.creativecow.net/thread/8/1078318

"No doubt catdv is the bees knees.

Cache-A has an extremely simple
database that allows searching.

As far as redundancy, we keep a set
local and a set offsite.

As far as speed, it runs at about 68-70 megabytes/sec so FireWire 800 speed. When tranferring 800 gigs from a single drive of any type will take a bit of time.

We thought long and hard about what to get. We had over 35 sata drives sitting on the shelf, it was time to either purchase another set of drives to add redundancy, or look for
something else. At the time, LTO was really the only cost effective way to do it. Buying
40TBs worth of raid protection, or 40TBs of more SATA drives, plus a server to run catalog software that could be run throughout our 'facility' wasn't making much sense for a number of reasons. The LTO tapes are cheap, easily transportable and the CAche-A has it's own web server with a rudimentary catalog that can be acessed from any computer on the network. It made the most sense at the time, and it is still making sense for the forseeable future. In the process of archving the 40TBs to LTO, one of our newer 1TBs drives failed to spin up. It's gone and lost forever even after sending it to Drivesavers. Now, even if something happens to our office, the offsite tapes are safe. It's worth at least throwing it in to your pile of considerations for long term archive. "

Jeremy


Return to posts index

Russell Lasson
Re: Long Term Storage Solutions for Red Camera
on Mar 15, 2010 at 9:27:39 am

[Bob Zelin] "How many people on this forum have tried to recover Quantum DLT from AVID media on a SCSI interface recently. That was less than 10 years ago. GOOD LUCK retriving this kind of media. You will have better luck with a 1" videotape. "

Comparing the setup of a Cache-A unit to a Quantum DLT with SCSI is an extreme. Cache-A connects over ethernet using AFP, SMB, or FTP. No drivers or anything.

[Bob Zelin] "LTO4A is a fantasy, for those people that have lots of time on their hands, and not much media to backup. "

??? I'm really not sure where you're coming from on this one. I know some very large organizations that have monster LTO archiving setups in multiple locations.

-Russ

Russell Lasson
Colorist/Digital Cinema Specialist
Color Mill
Salt Lake City, UT
http://www.colormill.net


Return to posts index

Bob Zelin
Re: Long Term Storage Solutions for Red Camera
on Mar 16, 2010 at 1:38:46 am

Russ writes -
Comparing the setup of a Cache-A unit to a Quantum DLT with SCSI is an extreme. Cache-A connects over ethernet using AFP, SMB, or FTP. No drivers or anything.


REPLY -
do you think that 1 Gig ethernet will exist in 10 years ? Its a miracle that GigEthernet exists right now. Will 10Gig ethernet be "backwards compatible" with Gig ethernet ? What if "they " choose to not stadardize to RJ45 connection, and use fibre with SFP+ or XFP connectors ? (as "they" are doing RIGHT NOW). They you can take your 10 year old cache-A with it's LTO4A tapes and "you know what" with it. The whole idea of using media from 10 years ago, in todays "computer age" is a complete joke (except for archive companies that will do this for a living) and the little production companies (if they are still in business 10 years from now) will NEVER pay for these services - only big people (like Agencies and big clients) will be able to afford to do this.

What do you do RIGHT NOW if you have to get media off of a Bernouli disk, or Iomega JAZ or ZIP disk ? Or DLT. What makes Gig Ethernet on RJ45 connection any different that this. At this moment, 10Gig ethernet is NOT going to happen on RJ45 (even though I wish it would, and I am prepared for all the tooling and cabling techniques).

Be assured of one thing - 10 years from now, everyting all of us own will be obsolete (except for the racks, and AC power strips).

Bob Zelin




Return to posts index

Jeremy Garchow
Re: Long Term Storage Solutions for Red Camera
on Mar 16, 2010 at 3:16:57 pm

[Bob Zelin] "do you think that 1 Gig ethernet will exist in 10 years ?"

You bring up a good point, Bob. In 10 years it will all be different anyway. There's nothing permanent that's available for archive today. It will all change but in 10 years, and we will make yet another decision and buy something else. we will then transfer our LTO catalog to whatever that is going to be just like we are with our temporary hard drives right now. Just like people that were archiving on PATA drives, when the move went to SATA, guess who was buying SATA drives? It's the nature of the game. When iOmega 1GB or 2GB Jaz drives were suddenly too small, guess who moved to PATA?

But for now, LTO is reliable and it's so easy to make two copies and it runs in the background. The management is minimal and it sits on your network available to anyone. It's quite cool and I encourage you to go get a demo of the Cache-A products at NAB this year. You might see the temporary light.

Also, for those that are interested, LTO is bondable. Can't get that on a hard drive.



Return to posts index

Earle Nichol
Re: Long Term Storage Solutions for Red Camera
on Mar 16, 2010 at 3:45:52 pm

.
[Jeremy Garchow] "It's quite cool and I encourage you to go get a demo of the Cache-A products at NAB this year. You might see the temporary light."

Hey Jeremy, I think the point that Bob is making is that in 10 years you may not be able to transfer that media from your LTO drives.(not that I would try and read Bob's mind) The other thought is what are the possibilities that you will really want to keep a client's stuff from 10yrs ago? I just left a post company and every 7-10 years we let the clients know that we were returning their footage to them as storage space was a premium. With digital media I'm not sure that would work, but it might be something to consider. On another note I am definitley going to see the Cache-A system, as I sat in with Bob on their demo last year, and well the demo was less than stellar...from your posts it seems to have improved greatly, so I will have a look.

Great input! thanks

Earle


Return to posts index

Russell Lasson
Re: Long Term Storage Solutions for Red Camera
on Mar 16, 2010 at 6:54:25 pm

Rj45 connectors likely won't be the standard ten years from now, but with as much cat5 cable as there is in this world, I think gig Ethernet will still have a place in it.

-Russ


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]