Creative COW SIGN IN :: SPONSORS :: ADVERTISING :: ABOUT US :: CONTACT US :: FAQ
Creative COW's LinkedIn GroupCreative COW's Facebook PageCreative COW on TwitterCreative COW's Google+ PageCreative COW on YouTube
APPLE FINAL CUT PRO:HomeFCP ForumFCP XFCPX TechniquesFCP TutorialsFC ServerBasics ForumPodcastFAQ

Promise Pegasus R4 8tb RAID config?

COW Forums : Apple FCPX or Not: The Debate

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Share on Facebook
Mark SlocombePromise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 12:01:13 pm

I've just got a Promise Pegasus R4 8tb to use with new iMac - it comes configured as RAID 5 which I understand is a good balance of speed and protection (if 1 of the 4 drives fails, its contents are saved).

How have other users configured their drives? Does RAID 6 offer any advantages?

Mark Slocombe
http://www.creationvideo.com
London, England


Return to posts index
Reply   Like  

Oliver PetersRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 12:29:25 pm

"Does RAID 6 offer any advantages?"

Yes. It's not unheard of to have a second drive fail while the RAID is rebuilding itself from the first drive replacement. A RAID 5 array is a good workspace, but don't rely on it for true long term storage, especially if it's on 24/7.

Oliver

Oliver Peters Post Production Services, LLC
Orlando, FL
http://www.oliverpeters.com


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 1:37:45 pm

[Mark Slocombe] "Does RAID 6 offer any advantages?"

More security, less storage, less speed. But even raid 6 is not totally safe - I've had a raid get corrupted even without drive failure. You have to find your own balance.

That being said, most editors are either raid 0 for speed or raid 5 for safety.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  


Jeremy GarchowRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 2:51:42 pm

Raid6 also knocks thecaiacity down by two full drives.

Your 8TB is now 4TB formatted.

Raid5 only uses the capacity of one drive, so your 8TB is 6TB formatted.

For a four drive raid, raid 5 will be great.

Jeremy


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 2:54:57 pm

[Jeremy Garchow] "Raid6 also knocks thecaiacity down by two full drives. "

I've often had problems with my thecaiacity, but I find if I just lay down for a few minutes it clears up.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  
+3

Jeremy GarchowRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 3:25:06 pm

[Herb Sevush] "I've often had problems with my thecaiacity, but I find if I just lay down for a few minutes it clears up."

I'm so glad we are speaking the same language and we find ourselves in solidarity.

Hopefully my iPhone will someday learn thecaiacity really means the capacity, becuase this obviously not my fault.


Return to posts index
Reply   Like  


Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 3:38:28 pm

[Jeremy Garchow] "Hopefully my iPhone will someday learn thecaiacity really means the capacity, becuase this obviously not my fault."

Don't know if you've seen this site of mangled iphone texts. bound to make you laugh.

http://damnyouautocorrect.com/13603/the-25-funniest-autocorrects-of-dyacs-f...

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  

David Roth WeissRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 3:18:23 pm

[Mark Slocombe] "Does RAID 6 offer any advantages?
"


RAID-6 on a small four-drive enclosure is a waste of your resources. You'd be using the most expensive storage you have for protection that, as Herb said, really doesn't give you ultimate security, as even RAID-6 can still fail. It only makes good sense for huge facilities with massive storage that absolutely have to be running 24/7.

Making a backup of the RAID-5 protected storage to inexpensive 2nd-tier firewire drives give you real protection, because you'll have redundancy.

The best of all worlds for your data safety is three levels of storage:

1) ONLINE STORAGE: RAID-5 protected storage (or 6 if you're big)
2) NEARLINE BACKUP: a 2nd tier redundant copy to inexpensive storage
3) LONG TERM ARCHIVAL: LTO tape or BluRay disc

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com
http://www.ProMax.com

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 4:00:03 pm

[David Roth Weiss] "The best of all worlds for your data safety is three levels of storage:

1) ONLINE STORAGE: RAID-5 protected storage (or 6 if you're big)
2) NEARLINE BACKUP: a 2nd tier redundant copy to inexpensive storage
3) LONG TERM ARCHIVAL: LTO tape or BluRay disc"


I know this is the basic business archival plan but I don't really see the value in "nearline backup" if you can afford LTO. An LTO drive is on my shopping list and one of the ways I can justify it's cost is by no longer storing backups on hard drives. The whole "nearline/long term" backup system seems to be more the preserve of big business where they have off sight LTO storage, as opposed to an editorial environment.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  


David Roth WeissRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:13:57 pm

[Herb Sevush] "The whole "nearline/long term" backup system seems to be more the preserve of big business where they have off sight LTO storage, as opposed to an editorial environment."

Not exactly Herb... In case of RAID failure, LTO backup would get you back up and running in a day, or within 4 to 5 hours possibly. The near-line solution gets you up and running almost instantly, or in an hour or so for most projects.

So, the 3-tiered system I advocate is really for anyone who can't afford to be a day late or a dollar short... :)

Or, at least for your clients who think that way.

Make sense?

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com
http://www.ProMax.com

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index
Reply   Like  

Joseph W. BourkeRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:30:44 pm

And don't forget a set of drives off-site. No amount of backup will do you any good if it's melted and crispy in the basement of a burned out facility.

Joe Bourke
Owner/Creative Director
Bourke Media
http://www.bourkemedia.com


Return to posts index
Reply   Like  

Joseph W. BourkeRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:31:43 pm

And don't forget a set of drives off-site. No amount of backup will do you any good if it's melted and crispy in the basement of a burned out facility.

Joe Bourke
Owner/Creative Director
Bourke Media
http://www.bourkemedia.com


Return to posts index
Reply   Like  


Michael GissingRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:46:38 pm

Good idea to backup the comment with an extra copy Joseph :)


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:46:41 pm

[David Roth Weiss] "In case of RAID failure, LTO backup would get you back up and running in a day, or within 4 to 5 hours possibly. The near-line solution gets you up and running almost instantly, or in an hour or so for most projects."

My situation may be specific but assuming the "near term" backup is a firewire 800 drive, I would still have to transfer that material to a raid for actual use. Unless I'm misreading things Firewire 800 is 800 Mb/s (mega bits). The LTO 5 spec is for 140 MB/s (mega bytes) - which equals 1120 Mb/s or nearly 50% faster. So I don't see the time saving for hard drives. Now I could definitely have screwed up the math somewhere along the way, but for the moment I just don't see the hard drive advantage.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  

Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 9:58:10 pm

I have online storage (direct-attached external RAIDs on a Mac Pro and Z800), nearline storage (12 TBs of capacity on an iSCSI Drobo), and offline storage (LTO5 backup).

While the LTO5 tape does transfer faster than the Drobo (though not for me, as I access my LTO tape system over gigabit Ethernet), LTO is physically offline -- the tape I need is rarely the tape in the drive. You have to pull the correct tape off the shelf and load it in before you can transfer your content. If the archive is larger than 1.5 TB, you need to switch tapes (both for archive and restore). Tape is just not as convenient as true nearline storage.

I break my projects down into hot, warm, and cold -- active, recent, and old -- and sort my storage accordingly.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  


Bob ColeRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 3:59:28 am

[Walter Soyka] "Tape is just not as convenient as true nearline storage."

I agree with Weiss and Soyka. I use a similar pattern, in my case RAID 5 (CalDigit), hard drives, and LTO-3. Although the software (Retrospect) with which I copy to LTO-3 has very good indexing, it is true that for ease of retrieval, hard drives are the backup of choice. My Mac Pro's three internal "extra" SATA drives are a wonderful way to make and park quick backups of critical material, when I have "drop-dead"-lines and can't afford to lose an hour.

I am looking forward, when it gets a bit less expensive, to switching to LTO-5 or LTO-6. But even then, tape presents a dilemma as to just how to organize the backups. Currently, I backup by project or client. That makes it much easier to reload the material, because when I do have to reload data, it is always for a given project. But it is a lot less efficient to do the backup that way, which means I wind up NOT doing tape backups as often as I would like. Due to the limited capacity of LTO-3 tape, I think I'm locked into that method for now. But for people like Walter, with LTO-5, do you just have the tape system backup everything that is new, every day? And if so, do you find that to be crippling when it comes to retrieval of data, in a pinch? Because I have several projects happening at once, and if I were to backup daily, the material for any one project might be spread over several tapes, and the material for any one CLIENT would be spread over many, many tapes.

Reloading material is a regular requirement, even if your RAID never fails. Whenever I reload material from an LTO tape, it is because I thought the project was finished, and had deliberately deleted it from the RAID -- and not once because the RAID had failed. My CalDigit has been fantastic; when a drive died, I had a new one ready to pop in and it rebuilt the RAID redundancy in a couple of hours.

Bob C


Return to posts index
Reply   Like  

Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 5:51:20 am

[Bob Cole] "But for people like Walter, with LTO-5, do you just have the tape system backup everything that is new, every day? And if so, do you find that to be crippling when it comes to retrieval of data, in a pinch? Because I have several projects happening at once, and if I were to backup daily, the material for any one project might be spread over several tapes, and the material for any one CLIENT would be spread over many, many tapes."

I consider backup to be separate from archive.

I run continuous backups to the cloud, nightly backups of active projects to local hard drive, and a scheduled weekly backup to tape.

Having only one copy of a project on one tape (in one location) would only make me feel marginally safer than one copy on hard drive. You're just a tape failure away from having no copies of the project.

When a project is finished, I archive the entire project twice: once onto a tape set for the current year's projects, and once onto a tape set for the client's projects. One set stays in the office, and one set comes home with me. It will also stay on nearline storage as long as there is room, and is ultimately removed on a FIFO basis.

It's a lot of work, but I think my diligence about project organization and data safety gives me fast, strong retrieval capabilities, and I consider them to be competitive advantages.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 2:28:50 pm

[Walter Soyka] " You're just a tape failure away from having no copies of the project."

I look at back-up differently. For me there are 2 different things to back up. 1 - Original camera sources recorded tapeless, 2 - Project media.

For project media, everyday backup to external hard drives and thumb drives works fine. At the end of the project I've been consolidating each job on hard drives; soon I hope to have a dedicated LTO tape for each major job to include all projects, graphics and music.

But it's backing up tapeless camera sources that has me most concerned. We started going tapeless last season and I have a set of five 3tb drives in two different locations as backup. I envision changing that workflow so that as soon as I dump everything to my main raid I then copy all the camera sources to LTO5. Once that's done and the tapes are in storage I can breathe easy again. I've never lost anything to tape loss in 30 years of editing, the same can not be said about hard drive failure. In this new workflow I can see no reason to also back up the camera sources to hard drives. If my raid fails it's going to take a day to restore and rebuild no matter what the source. If it's a matter of restoring old material, that never happens without some notice, and I don't see the big speed loss with tape since everything I work with has to be moved to the raid before I edit.

I'm definitely not as thorough as Walter in my backup plan, and one day it will probably come back to haunt me, but backup strategy like everything else is a matter of balance - how much risk vs. how much cost.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  


Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:14:50 pm

[Herb Sevush] "I look at back-up differently. For me there are 2 different things to back up. 1 - Original camera sources recorded tapeless, 2 - Project media."

I was dreadfully unclear in my original post. I was using the word "project" in the sense of all files associated with a particular job, not in the sense of an NLE application's file (like a .FCP or .prproj) that is distinct from its associated media.

I keep tapeless camera originals filed with other assets in its associated project folder.

I used to let FCP manage its render files (keeping them outside of my project folder structure and abandoning render files when the project concluded). With Premiere Pro's ability to manage render files on a project basis instead of a system basis, I can now keep render files in my project folders. I manually manage my C4D and AE renders, and they all stay with their associated project, too.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:23:06 pm

[Walter Soyka] "I used to let FCP manage its render files (keeping them outside of my project folder structure and abandoning render files when the project concluded). With Premiere Pro's ability to manage render files on a project basis instead of a system basis, I can now keep render files in my project folders."

Considering the nature of the work you do I would imagine some of your renders would be well worth worth saving.

AS for me, with FCP I would treat certain important renders as clips, copying them from the renders folder into the projects scratch media folder and renaming them before using. For the most part I'm happy to blow them away and let the program re-render as needed. I'm just looking for a way to recreate the security of a tape workflow in a tapeless world.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  

Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:54:59 pm

[Herb Sevush] "Considering the nature of the work you do I would imagine some of your renders would be well worth worth saving."

I generally don't care about the NLE renders; those tend to be less involved and pretty fast to re-render, and I'm usually not finishing in the NLE anyway. I do care a great deal about preserving AE/C4D mograph and animation renders, though, because the cost for disk space is negligible, but the cost for re-render time is high.


[Herb Sevush] "AS for me, with FCP I would treat certain important renders as clips, copying them from the renders folder into the projects scratch media folder and renaming them before using."

That's a pretty clever idea. It's kind of funny how we can turn FCP7's weaknesses in media management into a strength.


[Herb Sevush] " I'm just looking for a way to recreate the security of a tape workflow in a tapeless world."

I think dumping all your camera originals onto LTO5 as you've suggested sounds like the perfect solution for your workflow.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  


Bob ColeRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:23:52 pm

This has been an interesting thread. I would appreciate more details regarding hardware and software. It's all about data management, and the very specific approaches used are quite important. Clearly Walter and Herb are way ahead of me on this score. Help me catch up please!

Bob C


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:51:20 pm

[Bob Cole] "Clearly Walter and Herb are way ahead of me on this score. Help me catch up please!"

Actually Walter is way ahead, he actually has an LTO tape drive and uses it regularly. I've been researching this for about 6 months and I'm ready to take the plunge, just waiting to see what shakes out at NAB before I make any computer purchases.

LTO drives are a tape backup medium. The latest generation, LTO-5, can handle 1.5 Tb per tape (there is a compression scheme that can squeeze data 2:1 but apparently it doesn't squeeze much out of video files so I probably won't use it.) Most of the drives need a PCI SAS or fibre channel host card, although Walter has his hooked up over ethernet. The transfer speed is around 1 TB an hour using a SAS card.

These drives are mostly intended for large networked facilities and can come with cassette auto-loading so you can back up huge amounts of data unattended. That's not what I'm looking for, I just want to be able to back-up my media files to tape. Simple single external drives seem to cost around 2500 - 3500, the tapes around 90 each. You also need software to run the drives, BRU seems to be the most recommended on the MAC side, I'm not sure about what is best on Windows. There's also a company called Cache-A that makes what they call appliances where the LTO drive is teamed with it's own hard drive for buffering and has multiple ways to connect to a system - they have their own software and go for around 6K - 7k. The Cache-A system is very highly recommended.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  

Frank GothmannRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 11:38:08 pm

[Herb Sevush] "You also need software to run the drives, BRU seems to be the most recommended on the MAC side, I'm not sure about what is best on Windows. There's also a company called Cache-A that makes what they call appliances where the LTO drive is teamed with it's own hard drive for buffering and has multiple ways to connect to a system - they have their own software and go for around 6K - 7k. The Cache-A system is very highly recommended."

Just as a little additional info in case someone considers LTO5 for their backup needs. You don't absolutely need additional software for your backups (which in turn is also required to read back, sometimes an issue in cross platform environments etc.). You can also use LTFS (Linear tape file system, driver & utility comes free with the drive) and the tape will mount on the desktop just like any other media. Writing to it is a drag and drop operation, as is reading back to your machine.


Return to posts index
Reply   Like  


Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 11:53:05 pm

[Frank Gothmann] " You can also use LTFS (Linear tape file system, driver & utility comes free with the drive)"

Does this utility come with the Cache-A system, or with all LTO drives?

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  

Frank GothmannRe: Promise Pegasus R4 8tb RAID config?
by on Mar 2, 2012 at 12:05:25 am

[Herb Sevush] "Does this utility come with the Cache-A system, or with all LTO drives?"

Comes with all LTO 5 drives, it doesn't work with previous generations. Different vendors have different utilities, but the actual file system as such is not vendor specific. Also BRU PE has the ability to read and write LTFS. The win equivalent to BRU would be XenData Workstation, also does LTFS.
It's convenient, and free. Drawback of course is that there is no management in restoring, ie. BRU etc. can restore to the exact location where the original file were. If you copied it via the finder to LTFS you'd have to remember where it was in order to restore it to the same place (which isn't always important but can be for NLEs, render stuff, fonts etc.).


Return to posts index
Reply   Like  

David Roth WeissRe: Promise Pegasus R4 8tb RAID config?
by on Mar 2, 2012 at 12:09:46 am

[Herb Sevush] "The Cache-A system is very highly recommended."

Cache-A is da bomb as they say, and it's the only LTO system that's fully integrated with CatDV, which is an incredible DAM (Digital Asset Manager), and fully integrated with FCP legacy.

Check this webinar out, CatDV will amaze you:
http://www.promax.com/s-108-catdv-digital-asset-management-webinar.aspx

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com
http://www.ProMax.com



Sales | Integration | Support


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index
Reply   Like  


Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 9:57:19 pm

[Bob Cole] "Help me catch up please!"

Herb gave a great overview of your LTO5 options.

If you've got more questions, start up a thread on the Archiving and Backup forum [link]. I'll follow you there, and you'll likely get some other very good opinions as well.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  

Jeremy GarchowRe: Promise Pegasus R4 8tb RAID config?
by on Mar 2, 2012 at 9:30:47 am

David highlights the distinct difference between backup and archive. It's a big difference.

I use LTO as well, but it's "lowly" lto4 to a Cache-A. It's actually been a pretty wonderful solution.

I will highlight my workflow do you can pick and choose what's relevant.

We shoot, we make backups of the cards in the field to at least two drives if not a third.

That media then comes to the edit suite and gets copied to
local storage and starts the project and stays in the media folder that I create for that project.

At that point, we can plug a hard drive right on to the ache system and archive straight to tape. I tend to "stage" media in tape size pieces. Once I have around 750 GBs of data, I make a tape, then I immediately make another one which is taken home (offsite). Depending on the number of files, a full tape takes about 3-4 hours and runs at or usually over 80MB/sec.

The footage is now backed up and archived.

I edit the project. Once that's done, I network transfer that to the cache-a drive and backup all media save render files. This includes the original camera media, any transcodes, and all other ancillary media. This creates yet another copy of the original camera media, but it is now stored with all the project media, so when it comes time to restore, I have everything in one convenient package.

It works well for us.

I can't recommend cache-a systems enough. They are flexible and pretty easy to use. They keep an internal catalog of all media which serves as a very rudimentary catalog. There's no software, you plug it in to the wall, plug it in to your network, and power it up. You can even restore back to an attached hard drive if you want. They aren't cheap, though, so it's a big consideration. For us, the reliable archive system is worth it.

Ltfs represents a step forward, but since we are lto4, we don't have that capability so I can't speak to that.


Return to posts index
Reply   Like  

Frank GothmannRe: Promise Pegasus R4 8tb RAID config?
by on Mar 2, 2012 at 1:11:35 pm

[Jeremy Garchow] "Ltfs represents a step forward, but since we are lto4, we don't have that capability so I can't speak to that."

LTFS is still fairly young and there are a few things one should be aware of but it opens up new backup possibilities and an interesting roadmap with specs for LTO6 and 7 already in the can plus cartridges are quite cheap.
While it is of course not recomended nor something that should be done in production, it is possible to play HD Prores HQ streams in QT player from an LTO5 tape in realtime without dropped frames. So in terms of speed that's pretty amazing compared to tape based backups a few years ago.


Return to posts index
Reply   Like  


Bob ColeRe: Promise Pegasus R4 8tb RAID config?
by on Mar 3, 2012 at 12:14:21 am

Several questions:

1. There are heavy costs to archiving: a heavy initial investment, time, attention, and even the tapes themselves. If you archive a client's work, do you bill the client for that service, or is this part of your overhead?

2. It is often stressed, when transferring XDCAM footage, for example, that verification of the copied media is critical. If you are just using drag-and-drop to send data to an LTO-5 tape, is there any verification?

3. You write:
[Jeremy Garchow] "At that point, we can plug a hard drive right on to the ache system and archive straight to tape. I tend to "stage" media in tape size pieces. Once I have around 750 GBs of data, I make a tape, then I immediately make another one which is taken home (offsite)."

Are you saying that you make an LTO tape immediately, or that you don't make any LTO backups until you have accumulated 750GB of data? I guess that would be okay, if you had a very steady and predictable flow of data. But it would mean being exposed to the vagaries of hard drive failure for some period of time, at a stage when the project was most important.

(The sad truth is that for the vast majority of commercial projects, the finished product is the only thing the client really has much interest in, once the project is done. I would think that the most critical period for LTO backups would be during the editing phase, not after.)

Thanks for sharing the information about this important subject.

Bob C


Return to posts index
Reply   Like  

Frank GothmannRe: Promise Pegasus R4 8tb RAID config?
by on Mar 3, 2012 at 12:54:50 am

[Bob Cole] "1. There are heavy costs to archiving: a heavy initial investment, time, attention, and even the tapes themselves. If you archive a client's work, do you bill the client for that service, or is this part of your overhead?

If a client wants a back-up to LTO5 I will bill him. Most of my clients are regulars and I know I need to repurpose and revisit stuff all the time so this is part of my overall agreement with them.

2. It is often stressed, when transferring XDCAM footage, for example, that verification of the copied media is critical. If you are just using drag-and-drop to send data to an LTO-5 tape, is there any verification?

If you use tools such as BRU then verification is an option. If you just use drag-and-drop LTFS then there isn't. I never had an issue restoring a backup from tape.

3. You write:
[Jeremy Garchow] "At that point, we can plug a hard drive right on to the ache system and archive straight to tape. I tend to "stage" media in tape size pieces. Once I have around 750 GBs of data, I make a tape, then I immediately make another one which is taken home (offsite)."

Are you saying that you make an LTO tape immediately, or that you don't make any LTO backups until you have accumulated 750GB of data? I guess that would be okay, if you had a very steady and predictable flow of data. But it would mean being exposed to the vagaries of hard drive failure for some period of time, at a stage when the project was most important.

In my case, I wait till I can fill-up a tape. I am working off RAID6 so I'd have to experience three drive failures for things to go up in smoke.

(The sad truth is that for the vast majority of commercial projects, the finished product is the only thing the client really has much interest in, once the project is done. I would think that the most critical period for LTO backups would be during the editing phase, not after.)"


Depends on the nature of your projects and what you do. My stuff ends up on HDCAM-SR for the client plus Blu-ray for replication/stores. I know I will need to revisit the projects all the time, often just the final product for slight modifications. So I could either put another 250 dollar SR Tape for 120 minutes on the shelve and recapture when necessary, pile up external hard drives or use an 80 dollar LTO5 to store 10 hours without the need to recapture. Obviously the latter makes more sense and is more economical both for me and the client. They save money, I save time.


Return to posts index
Reply   Like  

Jeremy GarchowRe: Promise Pegasus R4 8tb RAID config?
by on Mar 5, 2012 at 5:53:41 pm

[Bob Cole] "If you archive a client's work, do you bill the client for that service, or is this part of your overhead?"

Mostly overhead at first, but you can certainly add an "archiving" fee to help defray the cost. It all comes down to how much your archive is worth to you, and perhaps to your clients. You don't necessarily have to ask your clients how much it's worth to them as they probably couldn't put a value on it, nor should they. When we get repeat business, our clients expect us to have the footage, and we also charge for the time that this archive takes to restore. So, you can make it a line item, or simply wrap it in to the cost of the overall project. Either way, you should certainly be able to add this to your invoices in some way as it is part of the service you are providing.

[Bob Cole] "If you are just using drag-and-drop to send data to an LTO-5 tape, is there any verification?"

We use LTO4. The way I do it is that I setup the "tape sized" bits on the SAN (or it could be local storage as well). I then use Shotput Pro to run a verified copy to the Cache-A hard drive (known as a "Share" in Cache-A terms). This way I know the media is good from the SAN to the Cache-A. The Cache-A has it's own verification system. You have the option to turn it off to gain a little bit of speed, but I leave it on as this is an archive after all! This way I have a verified copy to the share, then the Cache-A runs the verification to the tape.

[Bob Cole] "Are you saying that you make an LTO tape immediately, or that you don't make any LTO backups until you have accumulated 750GB of data? I guess that would be okay, if you had a very steady and predictable flow of data. But it would mean being exposed to the vagaries of hard drive failure for some period of time, at a stage when the project was most important. "

Well, we have the original shoot drives that are in essence a backup, and any and all project media is backed up. Once it comes time to archive, I then archive to LTO. I don't have to have 750GB sized bits. I archive to tape when the project is ready to archive. Sometimes, I have to split a project up as all of the media is more than 750GBs so it will go across two tapes (four with the offsite backup copies).

[Bob Cole] "(The sad truth is that for the vast majority of commercial projects, the finished product is the only thing the client really has much interest in, once the project is done. I would think that the most critical period for LTO backups would be during the editing phase, not after.)"

Then it sounds like for you, you need a solid backup system rather than an archive system. There's a big difference that DRW touched on.

We need an archive system and clients do return to modify projects, or use existing footage in new projects, as well as backup. It has worked out great for us, but perhaps it doesn't make sense for everyone.

AN LTO system could sort of be used as a backup system, but it's more of an archive system. Tape backups would be fairly inefficient due to their linear nature.

Jeremy


Return to posts index
Reply   Like  


moody glasgowRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 10:00:11 pm

Actually FW800 is 400 Megabyte/sec or 3200Mbit/s.
http://en.wikipedia.org/wiki/IEEE_1394#FireWire_800_.28IEEE_1394b-2002.29

LTO5 is not faster then FW800.

moody glasgow
smoke/flame
http://www.thereelthinginc.com


Return to posts index
Reply   Like  

Walter SoykaRe: Promise Pegasus R4 8tb RAID config?
by on Feb 29, 2012 at 10:16:09 pm

[moody glasgow] "Actually FW800 is 400 Megabyte/sec or 3200Mbit/s. "

The spec provides for speeds up to 3.2 Gb/s. I'm not aware of any devices or controllers that actually support the full spec. Everything I know of in production does top out at 800 Mb/s.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index
Reply   Like  

Andrew RichardsRe: Promise Pegasus R4 8tb RAID config?
by on Mar 1, 2012 at 2:20:02 pm

[moody glasgow] "Actually FW800 is 400 Megabyte/sec or 3200Mbit/s. "

Read your link again:

"IEEE 1394b-2002 introduced FireWire 800 (Apple's name for the 9-conductor "S800 bilingual" version of the IEEE 1394b standard). This specification and corresponding products allow a transfer rate of 786.432 Mbit/s full-duplex via a new encoding scheme termed beta mode."

The part later that reads "The full IEEE 1394b specification supports data rates up to 3200 Mbit/s (i.e., 400 megabytes/s) over beta-mode or optical connections up to 100 metres (330 ft) in length" references the IEEE standard, but is not representative of the FW800 that you would ever see in actual use. In any implementation on the market, FW800 means 786.432 Mbit/s or 98.304 MBytes/sec.

So yeah, LTO-5's 140 MBytes/sec data rate is faster than FW800.

Best,
Andy


Return to posts index
Reply   Like  


Todd GillRe: Promise Pegasus R4 8tb RAID config?
by on May 22, 2012 at 1:36:45 pm

WOW! Such great information on LTO archiving.

I'm in the process of going to LTO for archiving tapeless footage and projects/elements. I know it is going to be a huge investment. But I'm sitting on a lot of footage and projects that need backing up. Thank you to everyone on this thread for such great info. I'm looking at getting at the Tolis Group's The Edit Bay Cube.

http://ww2.productionbackup.com/hardware-bundles/desktop-bundle-packages

Due to that lack of PCIe slots I'm also going to need to get a Atto R680 for both my SAS connected RAID and LTO drive. But it will be worth it in the end and I can breathe easier once all is archived on mirrored tapes.

Todd Gill
Editor.Colorist
Digital Post Ink
http://www.digitalpostink.com


Return to posts index
Reply   Like  

Fabrizio D'AgnanoRe: Promise Pegasus R4 8tb RAID config?
by on Oct 25, 2012 at 10:46:38 pm

Backups and archives have become pretty crucial with the tapeless workflow. I still don't feel safe the way I used to be when I had all the dv or hdv tapes named and lined up and two or three copied of the project I could batch capture and recover from. The LTO looks like a great choice, but the price is still very high. You save about 20,00/30,00$ on a 1,5 Tb tape compared to a HD drive of the same capacity, so it's about 60 1,5 Tb drives before you pay the LTO driver. Years of backups in my line of work, even if I double security. And archived HD drives let sitting on a shelf should be less likely to go off than the actually used ones. I shoot outdoors documentaries so I first have to store the footage where I find myself working. I build a .dmg of each card daily on a RAID1 Western digital portable array from my MBP, and also copy them on one more drive. I keep the logged and transferred clips (I check and line up the footage daily) on the same RAID1, without copying them on the second drive (I still have the .dmg's in case it fails). When I'm back home, I copy the RAID1 drive content onto a raid 5 "cold" NAS storage array, then the transferred clips for actual editing to the MP internal RAID, that gets synch'd onto another external backup system, so that if the RAID0 fails I do not have to find, copy and reconnect all the clips, and moreover, the graphics, voiceovers and so on. Just to feel more at ease, I keep a drive loaded with OS and FCP only, so that if I have a problem with the current system disk I can just connect that one and go on with the projects on the fly without having to wait for Time Machine restores or reinstalls (it saved my ### yesterday when FCP7 went out and wouldn't launch or reinstall with a delivery to be completed tomorrow). When I'm done with a project I consolidate it using media manager into a drive I keep away. One drive per three or four projects. And time Machine takes care of the OS and projects on one more external drive. So far, it worked fine and I could survive to more than one drive or RAID failure. My next choice would be keeping the active projects media on an external RAID5 array like the Pegasus instead than on the RAID0 4 TB internal one.

Fabrizio D'Agnano
Rome, Italy


Return to posts index
Reply   Like  

Herb SevushRe: Promise Pegasus R4 8tb RAID config?
by on Oct 26, 2012 at 11:41:06 am

[Fabrizio D'Agnano] "You save about 20,00/30,00$ on a 1,5 Tb tape compared to a HD drive of the same capacity,"

Your getting great prices on your hard drives. LTO tapes go for about $45.

[Fabrizio D'Agnano] "And archived HD drives let sitting on a shelf should be less likely to go off than the actually used ones"

Hard drives that aren't spun up at least once a year are much more likely to fail then drives that see regular use.

The deal with LTO5 is that it gives you the safety of shooting on tape. Instead of consolidating on an external Raid5, which can and does fail - the raid can get corrupted even if the drives don't fail, it has happened to me - archiving to LTO is like having it stored on DV tapes - it can sit on your shelf for 5 years and still work perfectly. The tapes are small and easily stored or shipped.

I will say that dealing with the LTO drives and the software that supports them is not as quick and easy as copying files to drives. The A-Cache system is probably the easiest way to go, although it costs almost twice as much.

Herb Sevush
Zebra Productions
---------------------------
nothin' attached to nothin'
"Deciding the spine is the process of editing" F. Bieberkopf


Return to posts index
Reply   Like  


Max SugermanRe: Promise Pegasus R4 8tb RAID config?
by on Apr 8, 2014 at 6:02:28 pm

I think it's important not to underestimate the power of long-term cloud storage as a viable tier 3 option going forward. The fact of the matter is, the cost benefit towards moving completely digital is fairly substantial on both a small and a large scale. On top of that you're looking at a virtual fail safe. I think Amazon S3 storage is something like 1 file lost in over 10,000, and correct me if I'm wrong but I believe they claim to have never lost a file in Glacier Storage. And we're talking pennies on the dollar for storage space as compared to LTO.


Return to posts index
Reply   Like  

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Share on Facebook


FORUMSTUTORIALSFEATURESVIDEOSPODCASTSEVENTSSERVICESNEWSLETTERNEWSBLOGS

Creative COW LinkedIn Group Creative COW Facebook Page Creative COW on Twitter
© 2014 CreativeCOW.net All rights are reserved. - Privacy Policy

[Top]