FORUMS: list search recent posts

HP StoreOpen LTFS

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Bob Cole
HP StoreOpen LTFS
on Feb 7, 2013 at 11:05:57 pm

From HP's website, describing HP StoreOpen with LTFS:

"Linear Tape File System (LTFS) is an open format for storing data on tape that makes LTO-5 and 6 tapes self-describing and file-based."

This sounds great, but I'm sure there are aspects to this that I don't know enough about even to ask the right questions.

We have all the "ordinary" needs of a normal business (backing up our system drives, accounting and document information), but we also have massive amounts of video data. In addition, we need both to back up current work, and also to archive completed projects. The latter, it seems to me, is where the LTFS system would be wonderful.

Considering that the video post-production environment (esp. with today's tapeless video) is very different from the average business in terms of its data flow, what are the pros and cons of LTFS? And, is there anything special about HP StoreOpen as opposed to other flavors of LTFS?

Please forgive my ignorance and decrease it a bit. Thanks!

Bob C


Return to posts index

Tim Jones
Re: HP StoreOpen LTFS
on Feb 7, 2013 at 11:25:01 pm

Of course, this is a bit slanted by our stand on the recoverability of all things written to tape, but for more depth on the subject, you can start here:

http://knowledgebase.tolisgroup.com/?View=entry&EntryID=263

Tim

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Kevin Francis
Re: HP StoreOpen LTFS
on Feb 8, 2013 at 3:26:15 am

Hi Bob,

Not sure if you are Mac or PC. We've been using BRU PE for a couple of years and for the most part it works. Helps to have knowledge of Terminal commands because you will definitely need it from time to time to restore a tape or for other functions. Their support is good but not fast- if you have an urgent issue you're pretty much out of luck. The interface has quirks and bugs that usually get addressed eventually but sometimes get broken again on a new update. When your archive library starts to get big, it takes a long time for the app to boot up as it reads all the archive data. If I had it to do all over again, I would have invested a little extra in one of the Cache-A appliances and saved myself a lot of time and headaches.

Using LTFS from the Finder in Mac isn't ideal, but it is an up and coming standard which nearly every vendor supports at this point, even BRU. But since you probably read the article Tim mentioned above, you know all about the idiosyncrasies of LTFS.

Hope that helps.

Kevin



Return to posts index


Bob Cole
Re: HP StoreOpen LTFS
on Feb 8, 2013 at 4:44:07 pm

[Kevin Francis] "Their support is good but not fast- if you have an urgent issue you're pretty much out of luck. "

Well, you've hit the nail on the head. Few of us have our own IT Department - we're "it" so to speak. So, finding a solution with outstanding support is of paramount importance. That's why I bought a CalDigit - at the time anyway, it seemed the best at support. And that is also why so many of us have bought AJA products. I would love to see a ranking of how well various companies "back up" their backup solutions.

Thanks for the warning about needing to learn Terminal commands. I'd just as soon spend that time getting better at After Effects. If I can avoid Terminal, I'd like to try to do that!

Thanks too, to Tim, for the very helpful white paper. The premise about support for LTFS is valid. (It reminds me of the classic investment advice: "Put ALL of your eggs in one basket - and watch that basket like a hawk.") One of the big arguments in favor of LTFS is that an open standard does not depend on any one company's support; but perhaps it is not such a bad idea to have one company solidly behind a product (unless that company goes belly-up, leaving you without support). I would place very low confidence on getting any support from HP. When I've succeeded in reaching the right person at HP, they've been excellent; but getting to that person has been problematic.

BRU PE looks pretty impressive, but it may be academic for me, as BRU PE is Mac only (right?). I'm currently in a mixed environment, but am leaning toward PC, due to the relative performance of the Mac Pro. I already own Retrospect Professional, which has been okay, though I've never really attempted anything sophisticated with it.

Bob C


Return to posts index

Frank Gothmann
Re: HP StoreOpen LTFS
on Feb 9, 2013 at 6:07:23 pm

Throwing in my 5 cents:
Bru PE is a very reliable app, there are some minor oddities when you run it via the GUI but no deal breakers there. The biggest problem of Bru is that it's Mac and Linux only.
That's also the main reason why we've been looking deeper into LTFS as we wanted a small, dedicated machine sitting in the machine room doing nothing but writing to and from tape which anyone, even without Terminal knowledge, can operate. We also wanted something that wasn't tied to any specific backup software or platform. Now that HP has it LTFS libraries ready for windows we started doing all our backups on LTFS on a 300 dollar pc rather than going with a Macpro or a Mini with a 1.000 dollar TB breakout box.
So far, we're happy with LTFS. Again, it's early days, there some quirks with HP's implementation in it's current 2.0 flavour, but nothing major so my suggestion would be to give it a try.

------
"You also agree that you will not use these products for... the development, design, manufacture or production of nuclear, missiles, or chemical or biological weapons."
iTunes End User Licence Agreement


Return to posts index

Bob Cole
Re: HP StoreOpen LTFS
on Feb 9, 2013 at 7:31:08 pm

[Frank Gothmann] " we started doing all our backups on LTFS on a 300 dollar pc rather than going with a Macpro or a Mini with a 1.000 dollar TB breakout box."

Interesting! How exactly did you set this up? I've seen this done as two-step process. Make one PC an LTO station (call it PC-LTO), write over the network to the PC-LTO's hard drive, then let the PC-LTO write to its attached LTO at its leisure. Or, one-step: write over the network directly to the PC/LTO station. I assume that if you are using LTFS, you're writing directly to the LTO over the network.

Currently, we backup up individual computers separately, with a combination of external hard drives and LTO. I love the idea of using a single LTO tape station to make regular backups of all (five) computers on our network. I don't want to have to think about backups, because if I have to think about it, then one day I won't think about it, and that will be the day a critical hard drive fails.

LTFS sounds very appealing, but I just wonder about the details. Tim's white paper pointed out LTFS's inability to span, for example.
On this forum, some bright people have mentioned that there is a difference between backups and archives. I wonder whether proprietary software (BRU PE, ArchiWare, Retrospect??) might be better for everyday, system-wide backups (for disaster recovery), and LTFS might be better for archiving completed projects.

Comments welcome! I need to replace our old LTO drive, and I want to set this up right, to carry us through the next several years.

Thanks!

Bob


Return to posts index


Frank Gothmann
Re: HP StoreOpen LTFS
on Feb 9, 2013 at 9:44:30 pm

Yes, we're writing directly to LTO5 over the network. We have all our projects stored on Linux servers sharing out via 10GB Ethernet. The LTO PC simply hooks in via regular 1GB Ethernet.
LTFS cannot span, that's true, BRU can. However, with the arrival of LTO6 and the even greater capacity I am not sure if that's really a big issue. I had to "span" only a couple of times and I did what you'd also do if you were to copy the files to a hard drive - ie. divide the files in two folder chunks and copy one after the other.
Regarding system back-ups:
LTFS is really like copying data to an external hard drive. Simple. If you want to back-up your systems for disaster recovery you want software for that.
From what you have written, I'd consider the following: Install some dedicated hard drives for online availability according to your needs on the LTO-PC (I don't know how big your business is but system back-ups are usually not that big in size), back-up your system drives with something like Acronis True Image (incremental or whatever you want/need). Have the LTO-PC write them to LTFS.
If one of your systems need a fix you can simply restore from the Acronis images on the LTO-PC over network in a very short period of time. Since it's hooked up to the network, you can access it from any machine in you facility. If something happens to the LTO-PC or the backed-up images or drives, you have the golden LTFS back-up on the shelve.
Projects and video stuff you can write directly to LTFS. We don't even have a monitor attached to it. We access it via remote desktop and simply copy the files from the source locations on the servers to the attached tape drive once a project is out of the door. Once a tape is full the next tape goes in.

------
"You also agree that you will not use these products for... the development, design, manufacture or production of nuclear, missiles, or chemical or biological weapons."
iTunes End User Licence Agreement


Return to posts index

Kiki Muchtar
Re: HP StoreOpen LTFS
on Feb 22, 2015 at 10:07:16 am

Hi Frank,

If I have to make a double copy of everything I want to archive, how can I do that smoothly and fast? Let's say I have 2 PC-LTOs and 2 LTO drives. Can both PC-LTOs access the same storage to archive to LTOs at the same time? Is it maybe a mirror RAID network drive can be used for the purpose?

Thanks,
Kiki


Return to posts index

Tim Jones
Re: HP StoreOpen LTFS
on Feb 22, 2015 at 4:16:11 pm

Hi Kiki.

This is what BRU PE's "Doubler Mode" is specifically designed for. If you have 2 drives connected to the system, there will be a checkbox enabled on the QuickArchive mode settings panel that will write your selected data to two tapes at the same time.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Kiki Muchtar
Re: HP StoreOpen LTFS
on Feb 22, 2015 at 5:07:18 pm

Hi Tim, does that mean the process time is just as long as if you copy to one LTO tape?


Return to posts index

Tim Jones
Re: HP StoreOpen LTFS
on Feb 22, 2015 at 5:27:57 pm

Correct. The only slow down would occur if your disk drive(s) can't supply the data fast enough.

I recommend looking at BlackMagic's Disk Speed Test tool to get an idea of how fast your disks are. If you can't read 180MB/sec off of the disk, you'll never be able to stream 1 LTO-6, let alone 2. A 4 disk, USB 3 Striped array using 7200RPM drives is capable of delivering the data fast enough to feed 2 LTO-6 drives. On the other hand, a mirrored, 2 drive USB solution isn't.

Remember, if your backup tool can't read the data faster than the tape drive can write it (180MB/sec with BRU for example), then the backup application (BRU, LTFS, etc.) will spend most of its time waiting on the filesystem rather than writing to the tape.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Kiki Muchtar
Re: HP StoreOpen LTFS
on Jan 11, 2016 at 3:03:26 pm

"If you can't read 180MB/sec off of the disk, you'll never be able to stream 1 LTO-6, let alone 2."

Our HDD (as source) speed is more than 150 MB/s but never reach 180 MB/s. We archive from that HDD (1 tape at a time), and we've tried to retrieve the data back from the LTO tape, and it worked fine. We use HP StoreOpenLTFS & Terminal app. Is there any possibility that something actually isn't right?


Return to posts index


Tim Jones
Re: HP StoreOpen LTFS
on Jan 11, 2016 at 4:00:22 pm

Hi Kiki,

The issue is simply one of performance. That comment was made in reference to running 2 tapes simultaneously on 2 drives.

The point is that if you can't read the data from the source disk faster than the tape drive can write, the tape drive will suffer from underruns which causes shoe-shining and slows the writes even further. The slower your disk, the slower your backup / restore operations.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

David Roth Weiss
Re: HP StoreOpen LTFS
on Jan 28, 2016 at 6:48:12 pm

[Tim Jones] "The point is that if you can't read the data from the source disk faster than the tape drive can write, the tape drive will suffer from underruns which causes shoe-shining and slows the writes even further."

Love the term "shoe-shining" Tim, can you define that term for me so I can use it properly?

TIA,
David

David Roth Weiss
Director/Editor/Colorist & Workflow Consultant
David Weiss Productions
Los Angeles


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Kiki Muchtar
Re: HP StoreOpen LTFS
on Jan 19, 2016 at 5:13:14 pm

Hi Tim,

Understood, so disk array with USB 3.0 or, I assume, Thunderbolt are good enough. What about speedy server (RAID, 7200 rpm) with 1 GigE connection, will it gain the speed we're looking for?

Cheers,
Kiki


Return to posts index


Tim Jones
Re: HP StoreOpen LTFS
on Jan 28, 2016 at 5:23:04 pm
Last Edited By Tim Jones on Jan 28, 2016 at 5:25:43 pm

Nope - your limitation there is 1GbE. That results in maximum of 88MB/sec cooked data (data after network packet infrastructure is stripped off).

It's always the slowest component that affects your throughput:

Slow 5400RPM drive = 60MB/sec
Single 7200RPM drive = 88MB/sec
Singe 10KRPM drive = 120MB/sec
2, 3, 4, etc. - multiplied throughput up to the speed of your interconnect (net, SAS, FC, Thunderbolt, USB-3)

USB-3 = 430MB/sec MAX
eSATA = 520MB/sec MAX
Thunderbolt 1 = 800MB/sec MAX
Thunderbolt 2 = 1.54GB/sec MAX
SAS 6Gb = 580MB/sec per channel MAX
SAS 12GB = 1.1GB/sec per channel MAX
FC 8Gb = 775MB/sec MAX
FC 16Gb = 1.52MB/sec MAX

1GbE Network = 88MB/sec
10GbE Network = 920MB/sec

A 1 drive, 7200RPM drive will do 88MB/sec on any of those connections
An 8 drive, 7200RPM RAID 0 chassis that COULD do 700MB/sec connected via Thunderbolt 2 would top out at 700MB/sec, Thunderbolt 1 would top out at 700MB/sec, and via USB-3 would top out at 430MB/sec.
A 4 drive, 7200PM RAID 0 chassis that could do 352 MB/sec would run at 352MB/sec regardless of the interconnect layer.

* all numbers are ±5% and come from real world lab tests.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

David Roth Weiss
Re: HP StoreOpen LTFS
on Jan 28, 2016 at 6:52:41 pm

[Tim Jones] "It's always the slowest component that affects your throughput:

Slow 5400RPM drive = 60MB/sec
Single 7200RPM drive = 88MB/sec
Singe 10KRPM drive = 120MB/sec
2, 3, 4, etc. - multiplied throughput up to the speed of your interconnect (net, SAS, FC, Thunderbolt, USB-3)

USB-3 = 430MB/sec MAX
eSATA = 520MB/sec MAX
Thunderbolt 1 = 800MB/sec MAX
Thunderbolt 2 = 1.54GB/sec MAX
SAS 6Gb = 580MB/sec per channel MAX
SAS 12GB = 1.1GB/sec per channel MAX
FC 8Gb = 775MB/sec MAX
FC 16Gb = 1.52MB/sec MAX

1GbE Network = 88MB/sec
10GbE Network = 920MB/sec

A 1 drive, 7200RPM drive will do 88MB/sec on any of those connections
An 8 drive, 7200RPM RAID 0 chassis that COULD do 700MB/sec connected via Thunderbolt 2 would top out at 700MB/sec, Thunderbolt 1 would top out at 700MB/sec, and via USB-3 would top out at 430MB/sec.
A 4 drive, 7200PM RAID 0 chassis that could do 352 MB/sec would run at 352MB/sec regardless of the interconnect layer.

* all numbers are ±5% and come from real world lab tests."


What a beautifully compact and efficient "factoid." Maybe Tim Wilson should encourage you to think about putting this in an FAQ at the top of this forum (and possibly others too)???

David Roth Weiss
Director/Editor/Colorist & Workflow Consultant
David Weiss Productions
Los Angeles


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]