FORUMS: list search recent posts

LTFS - Offload Times

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
George Boyd
LTFS - Offload Times
on Nov 4, 2013 at 9:23:52 pm

Hello all,

I am doing some test with LTFS. I am offloading a 350GB folder from an LTFS tape onto a mac, the folder on the tape is filled with 1-20GB movie files. It was very quick to load the folder onto the LTFS tape. (approx 45 minutes-an hour)

However offloading this 350GB folder from the LTFS tape onto my Mac is moving very slow, it says it will take 12 hours to complete.

Does anyone familiar with LTFS know if I am experiencing slower offload speeds, or does it seem right that 350GB should take approximately 12 hours to load off of the tape onto a Mac using LTFS Manager.


Thanks,

George


Return to posts index


Doug Hynes
Re: LTFS - Offload Times
on Nov 5, 2013 at 12:39:42 pm

Hi George -

When you are writing to an LTFS tape, the files are being written sequentially - so the speed at which these are copied is only gated by the connectivity to the storage where the original files reside on spinning disk and the write speed of the tape drive.

On restore, the OS generates a list of files to be copied from tape back to disk. The issue with restore is that the OS does not track how the files were written to tape and where they reside and therefore the list is typically in random order, which is causing the tape to be scanned for each file in the list and is causing the back and forth movement that many have called "shoe-shining".

Archiving solutions that incorporate an intelligent software layer that can track where files reside on tape and provide most efficient movement based on that location are available today. These solutions can move content to/from tape at almost line speeds of the LTO Drives (140MB/Sec LTO-5, 160MB/Sec LTO-6), meaning an hour's worth of content can be transferred in 10 minutes.


Return to posts index

Tim Jones
Re: LTFS - Offload Times
on Nov 5, 2013 at 4:06:37 pm

Yes - what Doug said. The Cow's threading model hides these if you're just replying to the original posting...

BTW - our BRU family is one such group of applications that he mentions that properly manages the tape for file positions within an archive.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index



Tim Jones
Re: LTFS - Offload Times
on Nov 5, 2013 at 4:04:07 pm

Hi George,

I suspect that Finder is getting in your way in this case. When you drag and drop the files from the LTFS volume, Finder sorts and copies those files one at a time. This results in a lot of seeks on the tape as Finder goes after each file separately (the same thing happens with Explorer on Windows).

You can remove some of the seeking (and resulting delays) by copying folders instead of files. But, if you need to select specific files, this is one of the caveats of dealing with LTFS.

If you are familiar with the OS X / Unix command line, you can also reduce the time for restore by using the Unix "cp" command instead of Finder, but that leads to a whole new world of confusion if you're not familiar with the Terminal.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

George Boyd
Re: LTFS - Offload Times
on Nov 5, 2013 at 7:10:54 pm

Thanks guys,

Very helpful information.

Tim, I am familiar with terminal commands regarding TAR. Not so familiar with the "cp" command. I'm looking into it now, thanks for the recommendation.

It would be great to find a workaround to retrieve these files without waiting 12-24 hours. I am interested in software which could potentially speed things up!


-George


Return to posts index

Tim Jones
Re: LTFS - Offload Times
on Nov 5, 2013 at 7:47:31 pm

Unfortunately, there's no way to speed the existing tapes up because of the limitations of the LTFS format. You will need to have written the tapes in an encapsulating, container format (a la BRU) to deal with the data as if it were on a tape instead of a disk. This is another limitation of trying to treat tape like disk.

It's like the man says, "ya gets what ya pays for."

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Doug Hynes
Re: LTFS - Offload Times
on Nov 5, 2013 at 10:08:33 pm

George,

As I said previously, there are archive solutions available today that use LTFS and achieve near line speeds of the LTO drives during both archive and restore functionality. These solutions are in use everyday and include workflows for feature film, episodic, reality, documentary, sports programming and more.

You can contact me if you'd like to learn more.

Best Regards,

Doug


Return to posts index

Bob Zelin
Re: LTFS - Offload Times
on Nov 6, 2013 at 1:44:03 am

so George - exactly what software are you using to do this ? Just the Finder ? Are you using the free HP StoreOpen LTFS software ?

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

George Boyd
Re: LTFS - Offload Times
on Nov 6, 2013 at 1:48:16 am

Bob - I am using LTFS Manager which is essentially HP LTFS Storeopen, and then using the finder for all the fun drag & dropping.
I have been tinkering with the demo of PreRoll Post by Imagine products - and it is doing a great job at pulling these large folders off quickly.

I have reached the conclusion that the free software gets the job done - however it does not proved one with quick retrieval times.
I have now offloaded 100's of GB within 2-3 hours where the free software was taking 12 hours to do so.


Return to posts index


Bob Zelin
Re: LTFS - Offload Times
on Nov 6, 2013 at 5:03:02 pm

Hi George -
I found the exact same thing that you did. I have only used Tolis Group BRU, or Cache-A, and just wanted to play with the LTFS stuff, "just for fun". And I found that the free HP StorOpen software took FOREVER, and simply trying PreRoll Post dramatically changed that - as you have now discovered too.

As the expression goes - you get what you pay for.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Kiki Muchtar
Re: LTFS - Offload Times
on Feb 22, 2015 at 7:53:14 am

[Bob Zelin] just wanted to play with the LTFS stuff, "just for fun"

Hi Bob, did you use Finder or Terminal command?
If you used Terminal, would you kindly share the command line?
Then, how do you verify the copy when you use LTFS either via Finder or Terminal?


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]