FORUMS: list search recent posts

BRU PE to LTO workflow, share your methods?

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Prodigi House
BRU PE to LTO workflow, share your methods?
on Jul 13, 2013 at 7:15:56 pm

Hi guys,

I've been trying to wrap my head around BRU PE workflow especially the part where i can document and retrieve the files inside the LTO archive, and it eludes me.
I wonder how you guys do it, how do you manage to keep track of what's archived to which tape? What did you prepare beforehand? Do you use any library/catalogue softwares?


Thanks
Oleng

Prodigi House
Jakarta


Return to posts index

Oleng Tjioe
Re: BRU PE to LTO workflow, share your methods?
on Jul 13, 2013 at 10:56:13 pm

I just remember that i probably should ask Tim Jones first..

Hi Tim...
Is there any kind of walkthrough in using BRU PE, like a video tutorial, that explains what to do and what to avoid?
I usually left the archiving overnight, and I found that i get confused on a lot of details, like how to verify if the tape we used has the data (i remembered that in BRU Server there's an information block about the tape ID & related infos of the loaded tape), or if the archiving actually succeeded.

What i do is I usually waited to see that the job is running, when it started to writing, i just left the job and check the following day. When i come back & see that the archiving is done, i exported the pdf to check the job details and i can't find any list of files in that pdf, which is weird because i saw that BRU reported the file name & what size being archived in the progress window.
But yeah, there's a lot of things that i can't wrap my head around, i think that a video tutorial or a walkthrough might help answer some of them so i can find out if there's some real problems that i encountered instead of problems caused by my own dumb mistakes.

Thanks.
Oleng

Oleng,
Prodigi House
Jakarta, Indonesia


Return to posts index

Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Jul 15, 2013 at 2:54:39 pm

Ah - we are actually working on a couple of different scenarios - Archival and Long Term Storage, Archival of projects, and Archival for delivery.

We're heads-down on BRU PE for Windows right now, but the videos will come shortly after that release.

Does anyone have a specific scenario that they would like to see covered?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Oleng Tjioe
Re: BRU PE to LTO workflow, share your methods?
on Jul 15, 2013 at 8:42:41 pm

I think it would be better for common people like me if the tutorial divided in 3 parts, Setting up the job, Archiving, and Post-Archiving.

In the screenshot i uploaded here 6266_brutapeinfo201307153.41.44pm.png.zip, i find the terms in that notification a bit confusing, especially if the archive isn't in separate sessions.

I created a naming system, it's ArchiveDate_ArchiveOrder_ProjectNumbers. This system doesn't contain any archive id since i couldn't tell what it will be beforehand.

That screenshot is of a tape that contains several projects in a single archive, hence [date]_[archive#]_[projectnumbers] 20130608_001_9-13-21-24-35.
I believe the Label is for archive/job label, and that i wasn't really noticing why there's two labels there when i ran the job, lol.

Now with separate sessions archive, i get a better idea of what archive id and how to name them, but i hope you can see my confusion here, i'm still not clear on what i should do

Oleng,
Prodigi House
Jakarta, Indonesia


Return to posts index

Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Jul 15, 2013 at 2:50:38 pm

What are you specifically trying to achieve? Are you trying to create a DAM-like environment?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Oleng Tjioe
Re: BRU PE to LTO workflow, share your methods?
on Jul 15, 2013 at 3:45:06 pm

I'll just recycle this post

To put it short, i want to know the methods to match the content of the tapes with the files archived in the job. I think several things that confused me are:

- At first i thought that the saved job definitions is used to compared the content of the tape (for Restore). Now i realised that the catalog tar is the document that BRU PE look at when restoring files.
- When i create a job definition in Advanced View, there's a base job definition, which correlation i don't understand
- I have several jobs that resulted in 0 files written, i even uncompressed the TapeInfo and open the catalog file, but there's no file records there.

But overall i haven't get the whole picture how to make BRU PE works for us after archiving, the parts of verifying, documenting and restoring the tapes.

Oleng,
Prodigi House
Jakarta, Indonesia


Return to posts index


Oleng Tjioe
Re: BRU PE to LTO workflow, share your methods?
on Jul 15, 2013 at 8:03:46 pm

Hi Tim

so basically we're archiving projects and HD clips/image sequences, the projects are archived up to a year, then we would reuse the tape for new projects.

I don't really have that much problem with using the software up to archiving part, but when it's about double checking the job, verifying existing tape, it's not so easy to grasp. (I used BRU server before, but that's some time ago)
I struggled with the terms (like tape set, tape name/id, archive id) and how the informations of the files inside the tape or about the tape are presented. Not to mention i can't really see inside the tape to check if the files are there if the catalog records that there's no files written in that job.

And on top of it all i also haven't have a good workflow for documenting the archives. Do i use another cataloging software, so that all the tapes & archives are listed in tabular data, search-friendly & sortable without spending much time doing it manually (and risking bigger probability of human error)

I'm not sure if this is the right place to suggest a software feature/changes, but these questions might include some in here

1. Verify archive in the tape
I understand that it actually verify the archive structure or integrity, but i'd like it if there's a notification of what archive is verified in the tape. Now it's just put out a notice that the tape is verified successfully. And to get the information about the tape, i have to go to Restore section which was a bit counter intuitive i think?

2. Do you think i should change to LTFS, for more ease of use? I mean in the future, someone else might have to do all the archiving, doublecheck, restoring and documenting the project files, and i know for sure 90% chance is that guy wouldn't be as knowledgable at this as me. I'm not bragging, but it's a fact of life here, it's hard to get a technical savvy personel. I just happened to end up in this industry myself and stuck around.

3. Is there a way to get the file path in the pdf catalog shorten, or maybe formatted like for every files in the same parent folder they would be grouped like for example:

(parent folder as head)
- (list of the files inside)

/Volumes/Archives/ProjectFolderwithLongNames/DepartmentNameFolder/2013/06/16/
- image0000001.dpx
- longnameoftheframeimage0000002.dpx
- longnameoftheframeimage0000003.dpx
- longnameoftheframeimage0000004.dpx
- longnameoftheframeimage0000005.dpx
- longnameoftheframeimage0000006.dpx
- longnameoftheframeimage0000007.dpx
...


Now what i see is a really long list of truncated path names
/Volumes/Archives/ProjectFolderwithLongNames/Dep.../.../...oftheframeimage0000001.dpx
/Volumes/Archives/ProjectFolderwithLongNames/Dep.../.../...oftheframeimage0000002.dpx
/Volumes/Archives/ProjectFolderwithLongNames/Dep.../.../...oftheframeimage0000003.dpx
/Volumes/Archives/ProjectFolderwithLongNames/Dep.../.../...oftheframeimage0000004.dpx


PS:
I read the user guide today, and i got a better understanding of the terms, like archive id (in separate sessions), etc. but i still have some confusions. maybe i should divide this questions into several shorter posts

Oleng,
Prodigi House
Jakarta, Indonesia


Return to posts index

Cody Foltz
Re: BRU PE to LTO workflow, share your methods?
on Jan 19, 2015 at 4:22:02 am

After I complete a project, I typically try to keep everything within one root directory folder to make it easier for archiving. I am using BRU-PE 3.1.7 and LTO 5 tape/drive.

I do have a question as far as when it is safe to delete assets from the system drive after it has ben written to tape. If I am archiving a full LTO 5 tape, and the verification process has begun (typically about 3hr. or more) at that point is it safe to delete the data from the system drive since it has been written to tape but hasn't completed verification?

Thanks.

__________________________
Cody Foltz
http://www.leftdigitalmedia.com
Editor | Producer

MacPro V.3,1
2.8 GHz Quad-Core Intel
16GB 667 MHz RAM
OSX 10.6.8


Return to posts index

Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Aug 31, 2015 at 1:44:35 pm

Hi there,
I am Rumen from Bulgaria. I am using LTO4 in HP drive SCSI since 3.5 years with BRU PE last vers. We using to archive TV production sources audio and video. I have a 2 questions:

1. Is there have some way to compress data so that it will useful after that. We have not any additional hardware for that and should be use software compression. What kind?
2. Is it normal on Maxell LTO Ultrium4 tape 800/1600 to collect only 500GB. I did quite testing but without success.


Return to posts index


Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Aug 31, 2015 at 3:02:02 pm

HI Rumen,

The compression in LTO drives is automatic. It's also lossless, so you get back exactly what you put in. There is no further compression in your situation because the data is already compressed by either your camera or your CODEC during transcode.

As for capacity, BRU does incur around 18% overhead to provide the reliability factors that are included in its container format, but loss of 300GB is 38%, so something is going on at the drive level. A couple of questions:

When was the last time you ran a cleaning cartridge through the drive?
Have you downloaded HP's Library and Tape Tools and run a diagnostic test on the drive?
(depending on the number of tapes you've used, the drive may be exhibiting head wear with will reduce tape capacity).
What version of OS X are you using and which pSCSI host bus adapter are you using?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Aug 31, 2015 at 5:50:35 pm

Hi Tim! Thank you for the answer!

If there no compression what is mean 1.6TB on the tape and which way can I fill in?


Your Questions:
1. Never clean
2. Didn't download and run diagnostic.
3. Usualy run about 1.5 TB on 3 tapes.
4. MAC OS X 10.8.5
Areca Technology Corporation
Product: ARC-1320
Revision: version 2.0.3
Initiator Identifier: 255


Return to posts index

Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Aug 31, 2015 at 6:48:14 pm

That explains a lot.

First, the Areca HBAs / drivers don't properly support tape. I've totally surprised that you've had any success at this point. The only pSCSI HBA that we had success with back in the day were the ATTO UL4 and UL5 cards.

Next, run a cleaning cartridge in the drive.

Finally, go to the HP website and search for "L&TT for Mac OS X".

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Sep 1, 2015 at 3:53:16 pm

Hello Tim, I'm sorry that I caused your headache.

I use this device /HP StoreEver LTO-4 Ultrium 1760 SAS External Tape Drive/ since I had no problems, except that with the reduced size of recorded information. I Would keep your recommendations and give feedback.

Unfortunately I can not find in any way that software HP's Library and Tape Tools. Please if anyone has this software, send me to mail / rmitovv@gmail.com/ .

So far, there is no way to change the SAS controller. I did not understand whether the problem is with the controller or with areca driver and how this can affect the amount of recorded information?


Return to posts index

Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Sep 1, 2015 at 4:23:15 pm

Hi Rumen,

No apologies required :).

The link to the L&TT can be found by going to http://www.hp.com and searching for "L&TT for OS X".

The Areca issue is a combination of both their chipset and the OS X driver because the chipset is not configured to handle large, contiguous block reads and writes that occur when using tape. When you transfer data to a disk, the data blocks are broken up into what are called scatter-gather segment - each segment usually the size of a disk block or small multiple of the disk block - and written through the filesystem's IO manager. Tape, on the other hand, requires large contiguous blocks for proper I/O operations (we actually recommend 1024k for LTO-4). The fact that you've been successful using tape with the Areca HBA to this point is a serious stroke of luck.

To explain how BRU works with the tape for determining capacity - a tape drive always knows where it is while writing to a tape. As we get close to the end of the tape, the drive sends us an "Early Warning - End of Tape" (EWEOT) status. We don't control that, but once we receive that status, we write the current block and then write the end of volume marker to tell the BRU restore process that there is another tape and then continue to the next tape. This is why I indicated that I suspect a drive issue in causing your short writes - the drive decides when it's written all that it can write, not BRU.

The L&TT diagnostic tool from HP is how we can determine what's going on with the drive.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Sep 1, 2015 at 5:24:26 pm

I coudn't find the software. The expert from HP told me that I need SID servise ID. So this software is not free.
This is the final link where they stop me:
http://www8.hp.com/us/en/software-solutions/free-enterprise-software-downlo...


1. So never mine if the main problem is ARECA controller I will change. Can you tell me one model?
2. In my device BRU I/O bufer size is set to 128K - ntape0 and ntape1. Is that OK or is too large. My be that is the problem. Can I change to 32K or 64K.
The most strange thing is that before year and a half on the same tapes capacity was near 800GB - 730GB.


Return to posts index


Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Sep 1, 2015 at 5:39:56 pm

We recommend the HighPoint Technologies RocketRAID 2722 if you're just using the tape, or the HighPoint RocketRAID 4522 or the ATTO Technologies R680 if you're using hardware RAID on disk with the same card.

As to the buffer size, 128K is pushing the limits of the Areca and 64K is better, but you will notice a reduction in performance on the drives (80MB/sec down to around 65MB/sec). I don't believe that this will resolve the capacity issue since you've had success to this point.

Since you've previously gotten the full rated capacity with your tapes, I'm again brought to an actual issue with the tape drive itself. You really should run a cleaning cartridge and also run the diagnostic test with the L&TT tool.

We have an article in our knowledgebase on how to run the diagnostics and pull the support ticket for the HP team is an issue is reported.

Here are the steps to run a drive assessment test with the "hp_ltt" tool.

http://support.bru.com/howto/ltt_drive_assessment_test.php

Here are the steps to perform the drive support ticket extraction:

http://support.bru.com/howto/ltt_ticket_extraction.php

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Sep 1, 2015 at 5:57:21 pm

Thank you Tim! That is lot of information so need time. Best for now!


Return to posts index

Rumen Mitov
Re: BRU PE to LTO workflow, share your methods?
on Sep 7, 2015 at 5:04:14 pm

Hello Tim. Since buying the ATO controller is costly. I would like to have every confidence that this change was not due to the version of the BRU.
I think we started with version 2 and then switched to 3. Could this be the reason.

I bought a cleaning cartridge. Is there anything in particular in the process of cleaning. Is there a tool in BRU about it.

I did check the archives from 2013 and there everything was fine. I let the cleaning cartridge and did test recording without compression software and change the buffer. Nothing.

I am still not able to download diagnostic software L&TT and turn to the supplier to clarify the case. While not released diagnostics I don't take steps to buying a new controller.
Best!


Return to posts index


Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Sep 8, 2015 at 2:10:45 am

You could also use the HighPoint Rocket RAID 2722 unit. It's much less expensive than the ATTO, is usually more available in the international market, and works wonderfully for tape.

Since the BRU engine itself didn't change (the part of the product that performs the actual tape I/O), I really believe that you're looking at a drive issue.

You still couldn't find the L&TT package on HP's site using that search term I listed above?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Paul Katzman
Re: BRU PE to LTO workflow, share your methods?
on Oct 13, 2015 at 5:41:35 pm

Hey all, been a lurker for a while and this is my first post here! Been having some similar issues to the OP with tape capacity (going to reach out to TG support, Tim.)

The HP website seems to be very wonky lately, maybe relating to their Consumer/Enterprise split; one link I found is hp.com/therestoftheurl and another is hpe.com/therestoftheurl. Anyway, hope this helps? http://h20564.http://www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=406731&swLangOid=8&...


Return to posts index

Tim Jones
Re: BRU PE to LTO workflow, share your methods?
on Oct 13, 2015 at 7:47:00 pm

Paul, that link is invalid. What were you specifically searching for on their site?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Paul Katzman
Re: BRU PE to LTO workflow, share your methods?
on Oct 13, 2015 at 9:21:37 pm

Huh, strange... I was just searching for L&TT Mac OS X, as you suggested. Still, I only had success with that when on the hpe.com/... website and not hp.com/... When using the "consumer" HP website, I can get search results for the software but the link doesn't work. The different URL seems to be the key for me, and maybe others as well?


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]