FORUMS: list search recent posts

Archive Status and Cache-A

COW Forums : Square Box CatDV

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Whitney Sickels
Archive Status and Cache-A
on Jul 31, 2012 at 2:05:54 pm

Hi there,

I see many people have been having issues with Cache-A, but I have a couple of questions that can hopefully be answered. (I just asked for an upgrade so maybe that will fix them?)

I've been concerned about the differing messages in my archive summaries. It is not the same on every clip after archiving the media. It says either: 'Already copied to...', 'Archived to...' and 'Copied to...' followed by the serial number of the first LTO I put in (more on that below) and then either 'currently on tape' or 'currently offline'. I assume the ones that say currently offline is from any LTO tapes that are not currently in the deck? But why are these statuses different when I do the same exact thing each time? (Just select clips and Archive media files...)

Once you "Archive" the files how do you tell it to write it to the LTO?

This morning I ejected the LTO that was in the deck, with nothing on it apparently, then pushed it back in. It then started transferring files and became full. It named it by it's serial number (AC38E6RAR4). Then it ejected the tape and asked for another. I put another LTO in and "Initialized" it. However, it named it the serial number from the previous tape followed with '_2'. Then, that LTO filled, asked for another and stopped (not enough left to fill) And again, it named this one the same serial number as the first LTO followed with '_3'. I checked CatDV and there are no records with LTO '_2' in the Archive Tape, Details or Summary info. Only AC38E6RAR4 and AC38E6RAR4_3 (the LTO currently in the deck). Yet, when I 'Check Archive Status' in CatDV of one claiming to be on AC38E6RAR4_3 it tells me it's on '_2'. What is the deal? And why isn't it correctly naming it the serial number that is on the actual LTO?

I have many more questions but am still trying to wrap my head around this and have a tech guy helping tomorrow. Any help on this is appreciated!

Currently on Version 2.0.22 Cache-A
9.0.6 CatDV


Return to posts index

bryson jones
Re: Archive Status and Cache-A
on Jul 31, 2012 at 3:14:57 pm

No ideas on the tape ID thing, let's see what the Cache-A guys have to say.

The official word on the copied vs archived is this. "Copied means it's copied it to the vtape. Archived means that additionally it has received confirmation (by reading the transfer log file) that the file has successfully been written to tape. You won't always get the latter, for example if you archive files by copying them to the vtape, then immediately close the CatDV catalog before it's written to tape."

This is from Squarebox and should explain a lot. Tape moves slowly, the confirmation takes a while and since the unit writes through the client app, the catalog can't close before the whole operation is done.

The archive, however is most likely fine since it made it to the vtape.

bryson

bryson "at" northshoreautomation.com

northshoreautomation.com


Return to posts index

Rolf Howarth
Re: Archive Status and Cache-A
on Jul 31, 2012 at 4:58:16 pm

The Cache-A device is supposed to automatically write files to tape once they've been copied to vtape, so to archive a file CatDV copies it to the vtape. If you quit CatDV immediately then the archive status will be "Coped to tape XYZ' (where CatDV gets the current tape identifier from the Cache-A device), but if you leave it running long enough for CatDV to get confirmation from the Cache-A transfer log then it updates the status to say "Archived to tape XYZ".

It says "currently on vtape" if the file exists in the vtape folder (ie. the hard disk cache that Cache-A uses). The file normally stays on the Cache-A hard disk until you load a new tape into the drive.



Return to posts index


Whitney Sickels
Re: Archive Status and Cache-A
on Aug 1, 2012 at 9:13:41 am

Ok - now I understand the difference! Thanks!

So I need to select the clips, 'Archive Media Files' and then leave CatDV open so it can then copy to LTO. Is there a specific setting we need to let it know to "automatically" transfer to LTO or once it's on vtape it knows to copy over? Should I re-open the catalog to see if the status has changed from Copied to Archived or is there a better way of knowing once it's on LTO? (Check the Cache-A Tape Directory?)

What should the workflow be like? Do I archive files (daily) as I create them, check the LTO periodically and once it's full, delete the contents of the vtape, and then put in a new LTO? Or do I have to wait until I have 1.5 TB worth of footage on the vtape and then put a new LTO in?


Return to posts index

Tom Goldberg
Re: Archive Status and Cache-A
on Jul 31, 2012 at 9:38:59 pm

Hi Whitney,

I'm not sure why these files weren't going to tape as you sent them, but that is the root cause of your issue. You had apparently pushed more than 3 TB of data to our VTAPE but the Cache-A did not start actually archiving until you reinserted the tape, whereupon we did what we normally do with more data than what would fit on a tape - tape spanning. When tapes are spanned, they all form a single logical volume and thus are named with the naming convention you noted (i.e. first_tape_name, first_tape_name_2 and so on).

Unfortunately, CatDV does not at this time accommodate spanned sets, so although apparently all your data is safely on tape, CatDV doesn't know which volume it landed on.

You can either keep this 3 tape spanned set as it is and use the Cache-A's search function to restore content when you need it or, if you want CatDV to remain in charge, re-archive your stuff to individual tapes.

The CatDV Archive interface nicely reports how much space you've used on each tape as you go - in your case, it must have constantly been reporting 0% which should have been a good clue that something wasn't quite right! If you start archiving to a new tape, note whether those items are actually getting written to tape - you can also easily tell from the Cache-A user interface by noting the Transfer Summary. If it still doesn't seem to be happening properly, let us know and we'll look into what might be causing your problem. Don't forget that your Cache-A share is probably close to full from that last spanned set so you'll need to be erasing data to make room for new archiving.

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index

Whitney Sickels
Re: Archive Status and Cache-A
on Aug 1, 2012 at 9:19:32 am

Hi Tom,

Thank you for your response - very helpful.

We would prefer not to span the tapes and re-archive to individual tapes. What is the best way of doing this? Is there a way to tell it not to span or do I need to clear the vtape and essentially re-archive everything via CatDV?

Also, what is this archive interface in CatDV that lets me know how much space is on each tape that you speak of? :) I see the information on Cache-A in Tape Information/Summary, but not in CatDV.

Cheers!
Whitney


Return to posts index


Tom Goldberg
Re: Archive Status and Cache-A
on Aug 2, 2012 at 12:57:40 am

Hi Whitney,

The best way to avoid spanning is to not put too much on a tape. You can turn it off under System Tools page > Settings tab > Multiple Volumes off. However, if you overfill a tape in this case, you end up with a tape that has no TOC - not good either.

In the CatDV "Archive Media Files" dialog there is a line that displays the Current Tape and in parentheses after the Media ID of the tape is a percentage displayed (45.6% used). To be safe, I would tend to move on to another tape once you get above 90%.

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index

Whitney Sickels
Re: Archive Status and Cache-A
on Aug 2, 2012 at 8:13:31 am

Ah yes! Thanks for the help and advice, much appreciated.


Return to posts index

Whitney Sickels
Re: Archive Status and Cache-A
on Aug 22, 2012 at 1:06:25 pm

Hi again,

I deleted the VTAPE and am re-archiving everything to avoid tape spanning. However, I just checked some and they say "Archived to (NEW TAPE NAME)". Sounds good. But when I "Check archive status" it says it's on the old (spanned) tape number. How do I get it to reference the new tape number?

I checked the LTO and as of now it has about 800 MB lost space... and growing as I continue archiving. What is causing this? It's Cache-A format (tar).

2.1.16 Cache-A
9.0.6 CatDV

Cheers!


Return to posts index


Rolf Howarth
Re: Archive Status and Cache-A
on Aug 22, 2012 at 2:05:11 pm

Check Archive Status uses the Cache-A API to query the tape a given file is stored on. Unfortunately I believe the API only returns the first tape, whereas it would be more useful if it returned all the tapes, or the last tape.



Return to posts index

Whitney Sickels
Re: Archive Status and Cache-A
on Aug 22, 2012 at 2:23:55 pm

OK, thanks. That's a shame.

On another note, I just checked the ones I (re)archived yesterday and there seems to be an issue. On some of them, on the Archive details they say 'Copied to' (as opposed to 'Archived to'), even though they all said Archived before I left work yesterday (and I left CatDV open to make sure it had time to copy to LTO). However, I checked the LTO and the file seems to be there. I was told Copied means they are on vtape and Archived means they are on LTO. So why do some still say copied when they are on the LTO? I checked a few files that are marked as copied and they are also on LTO.

After the 'Archived/Copied to' there is a semi-colon and something that either has said "currently on vtape" or "currently offline". What is this referencing? If the former, then it's still on vtape and the latter it is no longer on vtape?

Cheers.


Return to posts index

Tom Goldberg
Re: Archive Status and Cache-A
on Aug 22, 2012 at 2:34:54 pm

Hi Whitney,

I will defer any questions about how CatDV is tracking archiving tapes to those experts, but would like to provide answers about lost space.

Data tape can lose space for a wide variety of reasons:

Dirty heads or poorly performing tape decks as well as network connection issues can cause large amounts of lost space due to the need to re-write data that didn't get written correctly (LTO does read-verified writes, checking every bit and rewriting it if it didn't get onto tape perfectly).

Small amounts of lost space such as your 800MB out of hundreds of GB are usually due to less serious issues. Every time you write a single file, the deck must rewind, find the end of data and leave an empty block and markers for the start of new data. Each write session creates a tar package (what we call a tarball) and those tarballs have wrappers that use up a small amount of space as well. Thus the more sessions you have, the more these small losses add up and the more net loss you will see.

If you grab a whole stack of files and archive them together, the less space loss you are likely to see. You might even see a negative space loss (space gained through lossless compression in the deck).

You are welcome to contact our support group and have us evaluate the health of your tape drive if you have concerns about that (as long as you are under warranty).

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index


Whitney Sickels
Re: Archive Status and Cache-A
on Aug 22, 2012 at 2:45:22 pm

Ah, that answers it then. I was archiving often, instead of all at once. Would it make sense to consider the vtape the backup until we reach almost 1.5 TB and then archive to tape all at once?

Regardless, I think we need support! I just checked the first LTO we put in (that I thought was full when it asked for another tape - as we were spanning), but there was only a very small amount of data on it!

Will you be at IBC in Amsterdam this year by any chance?


Return to posts index

Tom Goldberg
Re: Archive Status and Cache-A
on Aug 22, 2012 at 3:20:12 pm

We recommend a max target of 1425GB per LTO-5 tape to assure room for the TOC, and yes, doing it in fewer sessions is better, although you don't have to do it all at once.

We are happy to look at your deck and any suspicious tape - please contact our support group as outline on our web site here

Yes, I will be at IBC in our new stand in hall 7 booth 7.E06!

Tom Goldberg
Cache-A Corporation
602 Park Point Drive
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index

Aat Aar
Re: Archive Status and Cache-A
on Nov 2, 2013 at 12:25:12 pm

I am facing a problem of 300Gbs to 400Gbs of Space lost in LTO 5.


Return to posts index


Tom Goldberg
Re: Archive Status and Cache-A
on Nov 3, 2013 at 3:56:31 pm

Large amounts of space lost are typically due to a tape drive that needs to have a cleaning tape run on it. Give that a try and see if your next archives aren't better... if not, you have have a hardware issue and will need a support connect session so we can evaluate the health of your drive.

Note that if you are seeing a negative space lost, it simply means that you have compressible material and the tape drive's internal lossless hardware compression engine is saving you room on the tape.

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com

(FYI - this really should be posted as a new topic and in the "Archiving and Backup" Forum)




Return to posts index

Aat Aar
Re: Archive Status and Cache-A
on Nov 3, 2013 at 6:11:01 pm

Thanks for the reply

I connected the machine with cache-a support they tested all the hardware, and found that the 1st drive is failing,
But is that why i am facing Large amount of Space Lost ?


Return to posts index

Tom Goldberg
Re: Archive Status and Cache-A
on Nov 3, 2013 at 7:11:21 pm

I'm sorry but there is no way to tell from this amount of information - please email me your case number and I'll look into it.

Tom Goldberg
Cache-A Corporation
433 Park Point Drive #285
Golden, CO 80401
mailto:tom.goldberg@cache-a.com
http://cache-a.com




Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]