FORUMS: list search recent posts

LTFS writing XML to index partition

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Ashley Lynch
LTFS writing XML to index partition
on May 30, 2013 at 8:41:26 pm

The network for one of our shows has mandated that all media be backed up to LTO5 in LTFS format and that a custom formatted metadata.xml file be written to the index partition.

This is where my life has begun to unravel.

I've managed everything to do with backing up our media to the LTFS, but I haven't successfully gotten this file into the index partition. Our HP LTO deck is hooked up to a OSX 10.6 tower.

Through research, I managed to put together the following code to format the tape.

mkltfs -d 0 -f --rules="size=1m/name=metadata.xml"

The metadata.xml file is currently sitting in C:\

The network said the tape didn't have the metadata.xml file and didn't pass QC.


Does anyone have any experience with this and is able to give me some help?

Ashley Lynch


Return to posts index

Kevin Francis
Re: LTFS writing XML to index partition
on May 31, 2013 at 1:33:11 pm

Can someone at the network point you to a resource that details how to do this? Is this an industry standard practice?



Return to posts index

Ashley Lynch
Re: LTFS writing XML to index partition
on May 31, 2013 at 9:41:41 pm

It's part of Discovery's new LTO specs for digital media storage. They've basically pointed me to HP's manuals which list all of the options, but I'm unclear on the exact syntax to get the xml file to the index partition of the LTO. Unfortunately it doesn't seem to give me any verification that's it's written the file I want there and there doesn't seem to be any way to check afterwards (at least not on our end.)


Return to posts index


Tom Goldberg
Re: LTFS writing XML to index partition
on Jun 2, 2013 at 3:40:56 am

Hi Ashley,

If you want to take an easy way out, use a Cache-A appliance - we have built the Discovery requirement into our products with an option to automatically put that metadata.xml file into the index partition.

If you want to check if your implementation was able to do that, here is how you can look at which partition a file is on:

getfattr -n user.ltfs.partition pathname_to_file
Where pathname_to_file is the full pathname of the file in question.

Partition "a" is the index partition and "b" is the data partition.

Good luck!

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index

Jonathan Applebaum
Re: LTFS writing XML to index partition
on Jul 9, 2013 at 2:20:25 pm

Tom,

Can you provide the command equivalent for Mac OS? I've tried xattr, but am not receiving what I believe is the proper information.


Return to posts index

Tom Goldberg
Re: LTFS writing XML to index partition
on Jul 9, 2013 at 4:02:50 pm

Jonathan,

When mounted to a Mac, the partition is invisible, thus there is no attribute that can be seen by the MacOS.

I would note further that even using getfattr under Linux can give you misleading results as LTFS automatically puts copies of anything written to the Index partition also onto the Data partition.

If anyone out there knows of a better way to do this, please post on this thread!

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index


Jonathan Applebaum
Re: LTFS writing XML to index partition
on Jul 9, 2013 at 4:20:48 pm

Actually, just learned how to do this from the folks at Discovery. On Mac OSX one would use the xattr command, along with an extended attribute. So, for example, if you have an LTFS volume mounted as "ltfs", the command to check to see which partition the metadata.xml file is in would be:

xattr -p ltfs.partition /volumes/ltfs/metadata.xml

The command will return either an "a" or a "b", where "a" is the index partition and "b" is the data partition. So if you receive an "a", the metadata file is correctly placed in the index partition.

Important note: The index partition is typically not updated immediately, so don't run this command until you're sure that has occurred (though a more frequent interval can be specified, the index is always written when a tape is unmounted).

- Jonathan


Return to posts index

Hans Hoffman
Re: LTFS writing XML to index partition
on Jul 19, 2013 at 8:19:44 pm

Thanks for this. Did anyone ever get back to you about how to actually place the metadata.xml file into the index partition in the first place using OSX?


Return to posts index

Kevin Francis
Re: LTFS writing XML to index partition
on Jul 24, 2013 at 12:46:05 am

Imagine Products PreRollPost now has an option to archive LTFS in the format specified by Discovery as well.



Return to posts index


Ashley Lynch
Re: LTFS writing XML to index partition
on Jul 24, 2013 at 1:44:25 am

Here's what we were able to solve with help from Discovery. We're on Mac systems.

mkltfs -d 0 -f -r size=10k/name=metadata.xml
mkdir /mnt/[LTO NAME]
ltfs /mnt/[LTO NAME]

This prepares the LTO to look for metadata.xml and put it in the index partition. You just copy the metadata.xml file along with all the other data and it will parse that one out. A good poor-man's test to see if it's in the index partition is to write it last. Then load the LTO and open the xml file. If it's in the index, it will load right away. If it's not, the tape will have to hunt for awhile to find it.


Return to posts index

Jon Berry
Re: LTFS writing XML to index partition
on Aug 19, 2013 at 11:12:40 pm

Ashley-

Tried that on our systems here, we're still running Snow Leopard, and I had to interchange the mnt command with Volumes. Everything looked fine until I double checked that the metadata file was on the index partition, but I came up with it being on the data partition instead. Here's the sequence of events I took:

1. Loaded up the LTO into the tape drive.
2. mkltfs -d 0 -f -r size=200k/name=metadata.xml
3. Shows the data partition being written on b, while the index partition on a.
4. mkdir /Volumes/MM_LTFS_01
5. ltfs /Volumes/MM_LTFS_01
6. Copy the metadata file onto the drive.
7. xattr -p ltfs.partition /Volumes/MM_LTFS_01
8. Thats when the metadata file shows up as being on partition b instead of a.

Any ideas? Using a HP LTO5. Thanks!


Return to posts index

Tom Goldberg
Re: LTFS writing XML to index partition
on Aug 20, 2013 at 3:30:49 pm

Jon,

I was also confused as I was getting the same result in my tests - this thread actually contains the answer:

[Jonathan Applebaum] "Important note: The index partition is typically not updated immediately, so don't run this command until you're sure that has occurred (though a more frequent interval can be specified, the index is always written when a tape is unmounted)."

So try ejecting, reinserting and then check your xattr's (thanks Jonathan!)

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index


Greg Luce
Re: LTFS writing XML to index partition
on Sep 13, 2013 at 7:18:22 pm

For those using linux, the appropriate command is
attr -g ltfs.partition /location/filename.extension

It should say
Attribute "ltfs.partition" had a 1 byte value for /location/filename.extension:
a


Return to posts index

Joe Kaczorowski
Re: LTFS writing XML to index partition
on Oct 31, 2013 at 1:36:55 am

Tom,

I am wondering when this discovery option was built into the cache-A appliance? I have recently been trying to develop workflow for my company to make deliverables to Discovery and trying to figure out what equipment to buy. I rented a Pro Cache LTO deck and found it clunky to use at best. I was very disappointed in the connectivity of the drive to the machine and the graphical user interface of the software.

I am under the impression that Cache-A has some of the best stuff out there and have to believe that what I was using was hopefully just out dated? Would you be able to point me in the direction to find more information on seeing how this workflow actually works? I would love to speak to someone in more detail in regards to our specific workflows and needs and how cache-A could help with that?

Thank you very much for your time,
Joe


Return to posts index

Tom Goldberg
Re: LTFS writing XML to index partition
on Oct 31, 2013 at 2:54:59 pm

Joe,

Cache-A added the Discovery Format Rule option as of our version 3.2.4 in May of this year.
Our releases and release notes are always posted on our Support web page at http://www.cache-a.com/support.php and software updates are available for download by request also from our web site.

If you'd like to discuss operations or drive connectivity further, I'd be happy to help you out, please drop me an email at the address below.

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index


Vincent Lavares
Re: LTFS writing XML to index partition
on Dec 19, 2014 at 12:00:44 am

Thank you all for this information. I found this to be such a tremendous help in writing out these tapes! One question though. If I am verifying that the metadata.xml file lives on partition "a" by doing the following:

attr -g ltfs.partition /mnt/tape/ltfs0/metadata.xml
and the return message is "a"

Do I then delete the file from /mnt/tape/ltfs0 after this "validation" in order to comply with Discovery's tech specs? Because if I don't, it appears that the xml file resides on the same partition (b) as the data.


Return to posts index

Tom Goldberg
Re: LTFS writing XML to index partition
on Dec 19, 2014 at 12:38:47 am

Hi Vincent,

If you are using the Cache-A Discovery format, the metadata.xml file will be on the index partition and you really don't need to check.

If you insist on double checking, be sure to eject the tape and reinsert after the last write to ensure that the information about where that file is written has been updated.

We recommend never deleting anything from a tape - in fact data will still be wherever it was, just no longer visible through LTFS.

Tom Goldberg
TGCS
30201 Rainbow Hill Rd.
Evergreen, CO 80439
mailto:tomgoldberg@gmail.com
http://tomgoldberg.net




Return to posts index

Vincent Lavares
Re: LTFS writing XML to index partition
on Dec 19, 2014 at 1:32:29 am

Thanks Tom! But what if I'm not using Cache-A? If I'm listing the volumes in linux, the metadata.xml file appears to reside on the same partition as the MAIN and AUDIO data. I looked over the Cache-A LTFS file layout and that apears to normal. I'm just questioning it against the spec sheet I was given.


Return to posts index

Greg Luce
Re: LTFS writing XML to index partition
on Dec 19, 2014 at 1:17:30 am

Hi Vincent,

If you're using the standard Linux binaries you also should not delete metadata.xml
The linux ltfs mount command lists the index partition with everything else, so you don't have to worry about metadata.xml showing up in ls or what have you. As long as your attr -g ltfs.partition command shows what you want it to, you should be good to go!


Return to posts index

Vincent Lavares
Re: LTFS writing XML to index partition
on Dec 19, 2014 at 6:22:07 pm

Thanks for the info Greg!


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]