FORUMS: list search recent posts

Backing Up cost effective plan

COW Forums : Apple Final Cut Pro X

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Craig Alan
Backing Up cost effective plan
on Jun 12, 2016 at 7:49:11 pm

I want to move to a system of using individual drives for back ups rather than using a second raid for each raid that needs a back up. I feel it will be a better and cheaper way to begin consistently backing up. Plus I have a bunch of raids whose drives are near their life expectancy thus the need to rethink what has become a mess.

However, since I will be backing up individual projects and since individual drives do not have the capacity of a raid, I need a way to clone each project rather than making copies. This will save a lot of time and I will more likely do this after every editing session. (With two connected raids, I simply cloned the editing raid to the back up raid)

I contacted CCC which I use and asked them about using the cloning program to clone libraries and project folders. But CCC cannot clone a library. But it can clone a folder that contains a library. So My back up drives will contain a folder with each project name and I can clone my raid folders to these folders as I edit. I can take a screen shot of the finder folders on each drive/raid and have a pretty solid an relatively inexpensive way to back up.

I've begun buying 6TB drives and using raid 5 rather than raid 0 for my media drives and using a thunderbolt dock and bare drives for back ups with stacks of cases to house them. The raid 5's thus have the storage of three bare drives. The dock is a lot cheaper than additional raids.

The 2 raid system was a lot more convenient but has proven unreliable. Have had too many raids go bad. Also once you have more than one raid and back-up raid, it's not all that convenient since I have to swap the raids on and off and connect them to the iMac I edit on.

Also if i want to back up a particular project to the back up raid and delete it from the media raid to give myself more headroom, well that messes up the whole cloning work flow. Ideally I think the media raid should only contain active projects - that is projects being shot or edited. So I'm thinking two copies of each finished project and only active projects on the media raid.

For important projects I can store a copy in a safe or off site.

Must add that being my own assistant editor is the only production job that I really don't enjoy. Its tedious and time consuming and life is too short.

That said: any feedback would be appreciated.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Darren Roark
Re: Backing Up cost effective plan
on Jun 12, 2016 at 8:10:35 pm

LTO tapes are still the most cost effective and safest long term backup solution.

If you don't want to invest in a system there are places that offer backup and restore services. I'm going to get an LTO-7 system once the 6TB tapes come down in price. Right now they are $135 ea.

Right now the dollars to gigabytes ratio are too close so I've been backing up to 2.5TB LTO-6 tapes @ $35 each.

One thing with LTO, for it to be a true backup it has to have a redundant set in case of a write error.


Return to posts index

Craig Alan
Re: Backing Up cost effective plan
on Jun 12, 2016 at 10:02:06 pm

What tape drive system did you buy?

Cost?

Are the LTO systems proprietary in any way? how sure is the future support of these systems?

Speed compared to hard drives?

I'm kinda feeling that as SD drives come down in price and increase in capacity and reliability they will replace other forms of storage.

$135 for 6TB is certainly more than competitive with hard drives.

But I'm seeing $5000 tape drives with two year warranties.

Don't tape head wear out?

How's the learning curve?

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index


Darren Roark
Re: Backing Up cost effective plan
on Jun 13, 2016 at 12:30:53 am

[Craig Alan] "Are the LTO systems proprietary in any way? how sure is the future support of these systems? "

LTO-7 can write LTO-7, and read LTO-6&5 tapes. It used to be the current gen LTO could read it's own version and one prior, now it's the previous two versions.

Similar to MPEG, there is an LTO consortium made up of IBM, Quantum, and HP. As long as working compatible drives exist your data can be read. The intent is long term reliable data storage for major corporations, pulling the plug doesn't seem to be on their roadmap.

[Craig Alan] "Speed compared to hard drives?"

The last time I rented some M-Tape drives it was a tad slower than a single USB 3 drive. I could play back 1080 ProRes files directly from the tape. (Pretty cool, wasn't possible before)

[Craig Alan] "I'm kinda feeling that as SD drives come down in price and increase in capacity and reliability they will replace other forms of storage. "

SSDs have components that can fail just like spinning drives. The LTO tapes are just recordings of zeros and ones and have no electrical components to fail. For now, much safer.

This is what I want: http://gizmodo.com/optical-data-storage-squeezes-360tb-on-to-a-quartz-disc-...


Return to posts index

Craig Alan
Re: Backing Up cost effective plan
on Jun 13, 2016 at 12:55:26 am

So what would you recommend for an LTO drive today. Could i use CCC to create incremental back ups of folders stored in files or is this not the way LTO is structured?
I thought for some reason that each LTO drive came with software that limited use to that brand even though the tapes themselves were used as a standard.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Darren Roark
Re: Backing Up cost effective plan
on Jun 13, 2016 at 1:36:07 am

http://www.mlogic.com/products/mtape-thunderbolt-lto-7

This reads and writes LTO 6&7 which is cool. That gives you the option to use the cheaper v6 tapes and use the 7s when the $/GB ratio starts to make sense.

My understanding is the LTFS version needs to be compatible with the OS and LTO drives you are using. I've worked with places who leapfrog their data to a more current version of LTO every few years.

I don't know about using CCC with the LTO drives as I haven't done it. I've used this for media managed projects I need to keep longterm, not as a running backup. For that I use a slowish RAID-1 six bay array (using CCC).


Return to posts index


Craig Alan
Re: Backing Up cost effective plan
on Jun 13, 2016 at 2:06:54 am

that's what i kinda understood that they serve different functions. with thunderbolt 3 if it meets its promise coming soon - LTO should be plenty fast for most uses.

So each brand of LTO drive uses its own software? That would make me nervous. Big companies just update as needed. I'm a one man band and get to stuff when money allows and the need is pressing. Right now that need is an organized fast incremental back up system. And yes finished projects off the media raids and backed up securely. LTO seems like the best way to do this but a bit pricey given the amount of media I have of vital finished projects. Just the tape drive itself would get me 20 6TB drives. My SD footage is very small by todays standards and the P2 footage/FCP X files are about 16TB total. but counting. and my next camera will be 4 k.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Darren Roark
Re: Backing Up cost effective plan
on Jun 13, 2016 at 2:55:20 am

[Craig Alan] "So each brand of LTO drive uses its own software? That would make me nervous."

My understanding is the software that allows the tape to be mounted as an LTFS volume has to adhere to a standard so that cartridges can be read interchangeably between different manufacturers.

The different software apps are different, but the data written to the tapes and the format are not.


Return to posts index

Mark Smith
Re: Backing Up cost effective plan
on Jun 13, 2016 at 1:52:22 pm

I've been using LTO 5 with Pre Roll Post software from Imagine products for years. I recently retrieved a 5 year old file from tape archive that was archive with the first beta version of this software without any issues. Tape archives for my money are way more secure than spinning platters or SSD drives for my money. I have also archived whole FCPX and FCP 7 libraries, projects to LTO and brought them back years later without any issues. Aside from the initial hardware/software investment, the POM ( peace of mind) factor with using tape archives is very high with me.


Return to posts index


Craig Alan
Re: Backing Up cost effective plan
on Jun 13, 2016 at 6:39:51 pm

Thanks Mark. Food for thought. If you were to start fresh today what brand of LTO drive would you buy? Do you use LTO for incremental backups on active projects and if so what software do you use to do this?

My debate is that $5000 for just a drive that has a two year warranty and then additional expense for the tapes which need redundancy gets expensive given the amount of media I need to back up. that said if it becomes a firm investment in the future as opposed to every five years or less making new back ups to new drives then it would be worth it.

If I could use LTO for incremental backups that would definitely help me decide to go with the preferred LTO. To use one set of back ups on drives for incremental backups and another on LTO for archiving finished projects would be too expensive for me.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Mark Smith
Re: Backing Up cost effective plan
on Jun 13, 2016 at 7:36:45 pm

There has been quite a bit of progress in the LTO world since LTO 5 . There are Thunderbolt compatible drives and USB 3 compatible LTO drives available at this point which makes the CPU interface easy.
Mdrive is a good thunderbolt option.
Tape drives are robust. I have a Quantum SDLT600A drive that has been gathering dust for some years and the other day I pulled it off the shelf and loaded a tape that must be 7 years old. My experience with spinning platters as archive devices has not been so terrific....

I have good confidence in Tape back ups vs Drive back ups. I have pulled material, even just recently off the first LTO back up I made with Prerollpost software that was in beta when I used it in 2012. That tape had been sitting in a case since 2012. I like that.

I think LTO has a good shelf life and since the road map for LTO generally supports 2 legacy generations, I think long term archival is still better than drives over the long run, and LTO media costs generally drop quite a bit over the life of a format.

Incremental back up with PRP is a possible- one can set a watch folder and a background back up time for PRP to run and it will back up only new files added to the watch folder. If an archive has diminished in size for what ever reason, PRP doesn't delete those files from the archive as its not a random access storage device like a disk drive where its possible to write over selected sectors.


Return to posts index

Darren Roark
Re: Backing Up cost effective plan
on Jun 13, 2016 at 8:40:40 pm

[Mark Smith] "Tape drives are robust. I have a Quantum SDLT600A drive that has been gathering dust for some years and the other day I pulled it off the shelf and loaded a tape that must be 7 years old. My experience with spinning platters as archive devices has not been so terrific...."

Well said. On older drives I haven't had LTO backups made I have to fire them up and "take them for a walk" or it risks the motor seizing.

Trusting teeny tiny record players with your data long term isn't a great idea. LTO is what the studios use to back up DCIs and archive editorial so that's good enough for me.

AFAIK there are other proprietary tape backup systems, but LTO is intended to be open as to not cause future compatibility issues.

The compatibility is tied to the drive's ability to read that version of LTO, not the software that transferred the data to it.

As of now you can still buy LTO-3 drives new.


Return to posts index


Craig Alan
Re: Backing Up cost effective plan
on Jun 14, 2016 at 4:05:30 am

What is PRP? The incremental backup that does not overwrite is similar to CCC incremental backup with safety net but obviously will take up more room if you can't overwrite.

Can you delete stuff from an LTO to regain space? I could do the incremental back ups that dos not overwrite and then when all is finished delete all after a fresh back up of final version.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Mark Smith
Re: Backing Up cost effective plan
on Jun 14, 2016 at 3:57:25 pm

Sorry I took the short cut with an abbreviation. PRP= PreRollPost (see imagineproducts.com) which is software that I use for LTO back up and retrieval from LTO volumes. PRP allows for incremental back up, but since tape is linear not random access, you can't "free up space by deleting a file as you would with a hard drive. Tape just doesn't work that way.


Return to posts index

Craig Alan
Re: Backing Up cost effective plan
on Jun 16, 2016 at 7:47:05 pm

So it can do an incremental back up and take much less time than a full back up of say a FC library and will delete things that have been deleted but where the deleted data was still takes up the space on the tape? Is that it? I know on tape based camcorders you could overwrite though not particularly recommended. I though I had read that some LTOs could do overwrites?

Anyone use cloud based solutions or is this just too slow and or expensive?

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index


Darren Roark
Re: Backing Up cost effective plan
on Jun 16, 2016 at 8:15:43 pm

[Craig Alan] "Anyone use cloud based solutions or is this just too slow and or expensive?"

There are various services that vary on features and accessibility to data. Some are just designed for peace of mind that your data can be retrieved for a fee by them sending you a hard drive if a disaster strikes.

I've worked out of a few facilities with fantastically fast internet speeds where I could upload a half terabyte of Red footage in a couple hours. I think this is where things are headed, but it's not practical for most video professionals.

The hybrid hard drive approach suggested by another used might be feasible with cloud backups.


Return to posts index

Craig Alan
Re: Backing Up cost effective plan
on Jun 17, 2016 at 2:25:04 am

That's what I was thinking. Use individual drives in a dock for back ups - first incremental then archive when finished. Then use cloud service for failsafe back ups just in case disaster strikes. But if cloud is too pricey then consider tape. Thing though is you have to store this stuff off site to really be safe. I would imagine tape needs to be kept in a temp controlled environment.

As of now I have maybe 20 TBs of projects sitting around and I'd guess only half of those that I really think would be important to me in the long run. Doesn't yet seem the investment in a $5000 tape drive would be worth it. But when I start shooting 4K and longer form films then yeah. Who knows what future tech might bring. Really need a revolution in terms of storage.

Mac Pro, macbook pro, Imacs (i7); Canon 5D Mark III/70D, Panasonic AG-HPX170/AG-HPX250P, Canon HV40, Sony Z7U/VX2000/PD170; FCP 6 certified; FCP X write professionally for a variety of media; teach video production in L.A.


Return to posts index

Brett Sherman
Re: Backing Up cost effective plan
on Jun 14, 2016 at 12:44:50 pm

Let me push back a bit on LTO. There are solid reasons I use hard drives over LTO for backup.

1. Universal accessibility. If it's on a hard drive it doesn't take special hardware or software to read it. What are the chances the hardware and software you use will be around in 30 years? Let's just say it's not a probability I'd be willing to risk.
2. Speed of access. If you are continually pulling material from your archives, you will waste a lot of time with LTOs. Not to mention it makes using a MAM impossible.
3. Tapes fail too. Enough said. But the other thing to keep in mind is that if you don't have two tape machines you are lacking necessary redundancy to access your material.
4. Software failure. Mirroring hard drives is dead simple. Recording to LTO is not. I know of cases where the tape backup software failed miserably and resulted in data loss.
5. Ease of duplication. Without the original files, it's a time consuming matter to make a new copy of the backup.
6. Large size volumes. That's a little different with 6TB tapes, but tapes are always lagging hard disks in size. With smaller tape sizes, it's hard to manage your tapes since you can't have a one-to-one backup of a hard drive. And when you spread out your backups over multiple tapes, you've just multiplied you risk of data loss, because one tape going down will effectively destroy your archive.

Yes we all know platter drives are volatile. And if your intention is to make a backup, stick it on the shelf and never think about it again, then LTO may be the best option. However, if you manage your disk backups and have a solid strategy, you have little to worry about.

My method is I have two hard drive backups. One kept off site at all times. Once a year I spin up all the drives (just in the off chance I didn't access one throughout the year). After 4 to 5 years I copy my disks onto a new larger volume (this reduces the number of drives I have to keep). I currently have about 30 backup drives (60 because of duplicates). I have not lost any data yet.

Some of this is dependent on what your purpose for backup is. Prevent loss for the short-term? Long-term archiving with small likelihood of needing the material? Or long-term archiving with regular access? Is it mission critical to maintain access in 50 or more years? I'd at least consider a hybrid approach. LTO with a secondary hard drive backup.


Return to posts index

Doug Metz
Re: Backing Up cost effective plan
on Jun 15, 2016 at 10:19:46 pm

[Brett Sherman] "I'd at least consider a hybrid approach. LTO with a secondary hard drive backup."

This is the route I chose... tapes off-site, hard drives in a cool, dry storage area on-site. I don't typically need to re-do a previous project, but I often need to retrieve old files for new projects.

Doug Metz

Anode


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]