CatDV and Cache-A Archiving Questions

I'm hoping someone can help with some archiving questions that have recently arisen.

First, let me say that we are using CatDV 8.1.1 Pro/Server with the accompanying Cache-A plugin. Our workflow had been working fine up until about a month ago:

We would add media files to our catalogs, use the Archive Media function under Tools. Choose the Cache-A as the path. CatDV would correctly pick up the LTO tape name. We'd tell it to archive and the files would be immediately copied to the VTape in a mirrored (Volumes/Hard Drive/Media folder/Media File) pathway. And then Cache-A would automatically copy the files to the LTO.

I discovered recently, that the Cache-A was no longer copying the files to the LTO after CatDV had copied them to the VTape. Why the Cache-A is not syncing the VTape is a matter I am discussing with Cache-A Support, so I don't want to focus on that for this post.

What I am trying to determine is if CatDV is actually doing what it is designed to do and whether I correctly understand what it is doing.

The reason I ask this is because Cache-A Support is telling me that it is a bad idea to modify folders within the VTape. They suggest staging the files outside of the VTape and then copying over complete folders to the VTape root. When we archive files from the Finder, this is exactly what we do, however, this is not the way that the CatDV/Cache-A plugin works, at least on our system. On our system, it copies the files directly to a subfolder structure on the VTape and if you then archive other files onto the same tape, it just adds them to the existing folder structure.

My first question is: Is this actually what is happening, or is there something more complicated happening behind the scenes?

My second question is: Can the plugin be reconfigured to do the staging that Cache-A Support recommends?

And lastly: We're debating about upgrading to the latest version of CatDV. Would upgrading to the new Cache-A plugin do what Cache-A Support is advising us to do?

I appreciate any help with this.



Gerald Goldman
Director of Post Production
Grace Creek Media

Hi Gerald,

I will defer to the CatDV experts on the advantages of v9.x vs. v8.x and focus on the Cache-A side of your questions.

The reason we say it is a bad idea to modify anything inside the VTAPE is that the VTAPE is a watchfolder. Your Cache-A appliance is constantly monitoring that folder for changes and will archive anything it sees that is new. You move a file - it will get re-archived. You rename something - we see it as new and will archive it again.

Bryson's comment that LTO doesn't like a lot of little files is not quite correct - we handle small files fine, what causes problems are lots of little archive sessions. So the easiest way to make CatDV prevent small sessions is to get all your content organized and ready for archiving and select a large data set before you select Send to Archive. If you can run the job with 10 GB or more per session, you will have fewer problems and get better tape utilization.

There is a work-around if you want to manage your content within the VTAPE: First, insert the tape you will be archiving to, the Cache-A will create a VTAPE for that tape. At this time you can check available space, rename the volume, etc. Then eject the tape but select "Keep the VTAPE." Now you freely move your stuff around manage to your heart's content, and when you're all done, re-insert the tape. The Cache-A will tell you that the tape's contents doesn't match the VTAPE and gives you the option to Sync the contents - select that button and everything that wasn't in the VTAPE when you ejected will get added to that tape.

Finally, I'd like to take issue with Bryson's assertion that "you cannot perform "staging" through the SDK" - you certainly can. Our SDK will allow you to archive content from any folder on any volume it can see, and will allow for lists of files or folders to be archived at once (which is effectively staging). CatDV chose to not use that class of command but rather initiate archiving by moving data to the VTAPE. That said, even though I disagreed with this one point, I want to thank Bryson for being a knowledgeable resource for this community (and an all-around great guy).

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com

I recently sent an email to Cache-A support about this, but I have yet to receive a response. This seems to be along topic (and probably answers it), but I will post here to make sure:

We recently realized that our archive was only being copied to VTAPE and not actually writing to LTO. We then reinserted a tape and it started archiving, but we ended up tape spanning, which CatDV does not support. So then I deleted the VTAPE and started re-archiving to avoid tape spanning. However, I am getting "lost space" and would like to try and avoid/minimize this. On a daily basis, we would not archive that much. Therefore, what is the best workflow? If I archive through CatDV, then it will automatically write to tape; at least it does now the way it is set up.

I see in the manual it mentions creating a 'Staged Data' folder on the Cache-A Share. Should I do this and "archive" the files from CatDV to this folder and then when that has a decent amount of files on it, I then drag the contents to the VTAPE folder? If so, then what? Will it automatically write to LTO or is there another step?

Cheers!

Cache-A version 2.1.17
CatDV version 9.0.6

The main difference between CatDV 8 and CatDV 9/10 is that we use the Cache-A API to restore files. When we first developed the archive integration the archive API wasn't available which is why we use the vtape watch folder, but obviously that is supposed to work - the fact that you can just copy files to the Cache-A disk, secure in the knowledge that they will be archived, is Cache-A's selling point.

If you're worried about fragementation (in the sense of lots of separate sessions on the tape) you can create another folder on the Cache-A drive and choose that as the Archive Location (and select Other rather than Cache-A for the Archive System when you configure the archiving in CatDV). Then it will use the folder name as the "tape name" and you can manually drag the contents of that folder onto the vtape to archive it. Better yet, you can follow Tom's suggestion of ejecting the tape without erasing the vtape, though I haven't personally tried that myself.