FORUMS: list search recent posts

Slow verify with BRU-PE

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Neil Sadwelkar
Slow verify with BRU-PE
on Feb 20, 2016 at 12:41:35 pm

With BRU-PE 3.1.17 on a Mac under OS X 10.10.5, SAS LTO-6, I'm seeing excruciatingly slow verify speeds particularly with drives with a huge number of files. For eg. .ari, .dpx, .exr file sequences

Write is fine, though. Exact same setup I don't remember such slowness prior to this version. Will try one more tape with the same data and a prior version.

From the report, timings are like...

Example 1
20160220 11:26:17 - write start
20160220 12:29:17 - write end
20160220 12:30:10 - verify start
20160220 16:54:27 - verify end

wrote 262479872 blocks (524959744 KBytes) - 139025 KB/sec
read 262479872 blocks (524959744 KBytes) - 33105 KB/sec

So, 1 hr to write 500 GB, and 3.5 hrs to verify

Example 2

20160219 10:59:57 - write start
20160219 15:03:52 - write end
20160219 15:05:32 - verify start
20160220 10:50:18 - verify end

wrote 1058849792 blocks (2117699584 KBytes) - 144701 KB/sec
read 1058849792 blocks (2117699584 KBytes) - 29791 KB/sec

So, 4 hrs to write 2.1 TB, and 19 hrs+ to verify

During verify there's no drive access. In fact, we can even unmount the drive during verify.

Anyone else observe this?

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

Bjorn Frodason
Re: Slow verify with BRU-PE
on Feb 20, 2016 at 2:51:44 pm

Experiencing the similar thing, but only slower. 100GB verify in 3 hours.
I'm testing BRU PE and it seems to be nice piece of software in many areas.
But this morning I started to do a verify on a 1.6 GB data on LTO-6 tape in IBM TS3100 lib and it is very slow. Writing was normal where the bottleneck was the network.
This is just normal BRU verify, i.e. not comparing to the actual files.
I'm testing BRU because of a problem I ran into with Retrospect, when upgrading a failed LTO-5 drive to LTO-6. I think I will make a specal post with that one, so I will not hijack this one.

I'm "hoping" that the slow verify is connected to some underlying problem, SAS controller, failed LTO-6 drive, OS X or the HW. But I'm not sure what the best method is to diagnose.

System information
MacPro 1.1 Intel - 32 bit HW
OS X 10.7.5
6GB RAM -
Sys HD Mirror w. SoftRaid v. 4.53
SAS Controller Atto Express H380 driver 2.02, firmw June 2011
Library IBM TS3100 - Firmw. D.00 / 3.20e - 4 years Old
Drive-ULT3580-HH6 - Firmw. F9A1 - 6 months old

------------------------------
Bjorn Frodason
Technical department
ODDI Printing & packaging.


Return to posts index

Tim Jones
Re: Slow verify with BRU-PE
on Feb 22, 2016 at 2:22:19 am

Hi Guys,

We're looking into this, however, it's odd since the back-end bru engine component hasn't changed in quite some time.

What is the BRU version (not BRU PE) that you have?

In a Terminal type:

bru -h

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Bjorn Frodason
Re: Slow verify with BRU-PE
on Feb 22, 2016 at 8:04:39 am

Hi there

BRU -h results

release: 18.1
variant: 0.25
serial number: UNSERIALIZED
bru id: mac-osx
config: Jan 4 2016 10:19:21
capabilities: DEMO Edition
archive: ntape0
buffer size: 2048k bytes
device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
max warnings: 1000 [ BRUMAXWARNINGS ]
max errors: 500 [ BRUMAXERRORS ]
copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.


------------------------------
Bjorn Frodason
Technical department
ODDI Printing & packaging.


Return to posts index

Bjorn Frodason
Re: Slow verify with BRU-PE
on Feb 22, 2016 at 8:27:54 am

Here are the Results from the Verify of the 1.6 TB data set, it was error free, but not fast.
As I mentioned Writing speed was normal, and controlled by the network speed.
N.b. Can the reason be that I stopped the verifying in the Archive stage?
This is just "solo" verify, wich I ran afterwards.


serial_number: UNSERIALIZED
device: ntape0
user: root
group: wheel
system: Darwin mubarak. 11.4.2 Darwin K i386
bru: mac-osx
command_line: /Applications/BRU PE/BRU PE.app/Contents/MacOS/bru -c -m -vvvvvvvvv -j -O -A -QB -f ntape0 -QX -L 2
**** end info ****
archive ID = 56c3517a35fe
label = 2016-02-16 PG_Lok22 Backup
buffer size = 2048k bytes
media size =
í málfríði.doc
í tölum.docX
089_i2.jpg
**** bru: execution summary ****
Started: Sat Feb 20 11:04:11 2016
Completed: Mon Feb 22 05:32:23 2016
Archive id: 56c3517a35fe
Messages: 0 warnings, 0 errors
Archive I/O: 0 blocks (0KB) written
Archive I/O: 875259904 blocks (1750519808KB) read
Files written: 0 files (0 regular, 0 other)
Files read: 457065 files (406117 regular, 50948 other)
Files skipped: 0 files
Volumes used: 1
Write errors: 0 soft, 0 hard
Read errors: 0 soft, 0 hard
Checksum errors: 0


------------------------------
Bjorn Frodason
Technical department
ODDI Printing & packaging.


Return to posts index

Neil Sadwelkar
Re: Slow verify with BRU-PE
on Feb 23, 2016 at 3:00:19 am

Mine says...

release: 18.1
variant: 0.25
serial number: ____-____
bru id: mac-osx
config: Jan 4 2016 10:19:21
capabilities: PE TOOLS
archive: ntape0
buffer size: 2048k bytes
device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
max warnings: 1000 [ BRUMAXWARNINGS ]
max errors: 500 [ BRUMAXERRORS ]
copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.
All Rights Reserved.

There is a eight digit serial number which I've blanked above.
I can't remember if I uninstalled the older 3.1.12 version before installing this 3.1.17 version.

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index


James Vorley
Re: Slow verify with BRU-PE
on Jan 19, 2017 at 12:18:31 pm

I'm seeing what appears to be a similar issue intermittently. Did anyone ever get to the bottom of this?

For example 167 MB/s write but only 53 MB/s verify:

++++++++++++++++++++++++++++++++++
Last BRU PE Archive Execution summary:

20170111 13:19:52|3855|root|[L163] START (r 18.1.0.25), CMD = '/usr/local/bin/bru -c -m -vvvvvvvvv -j -O -A -QB -f ntape0 -QX -L STV-6-0006 -'
20170111 13:19:52|3855|root|[L167] device = ntape0, buffer = 2048K bytes, media size = , archive id = 587630f80f0f, (Op)
20170111 17:25:28|3855|root|[L182] wrote 1108950016 blocks (2217900032 KBytes) on volume [1], 4:05:36, 150508 KB/sec
20170111 17:26:36|3855|root|[L165] FINISH - 0 warnings, 0 errors, exit code = 0
++++++++++++++++++++++++++++++++++
Last BRU PE Verify Execution summary:

20170111 17:26:54|5487|user|[L163] START (r 18.1.0.25), CMD = '/usr/local/bin/bru -ivvvvjf ntape0'
20170111 17:26:56|5487|user|[L167] device = ntape0, buffer = 2048K bytes, media size = , archive id = 587630f80f0f, ()
20170112 05:09:51|5487|user|[L182] read 1108950016 blocks (2217900032 KBytes) on volume [1], 11:42:57, 52585 KB/sec
20170112 05:09:51|5487|user|[L165] FINISH - 0 warnings, 0 errors, exit code = 0


release: 18.1
variant: 1.28
serial number:
bru id: mac-osx
config: Jul 27 2016 13:47:11
capabilities: PE TOOLS
archive: ntape0
buffer size: 2048k bytes
device table: /Library/Application Support/BRU PE/etc/brutab[ BRUTAB ]
incl/excl file: /Library/Application Support/BRU PE/etc/bruxpat[ BRUXPAT ]
max warnings: 1000 [ BRUMAXWARNINGS ]
max errors: 500 [ BRUMAXERRORS ]
copyright: Copyright (c) 1985-2016, TOLIS Group, Inc.
All Rights Reserved.


The issue appears randomly and often not at all on the same data and tape set. When the issue appears it seems to be on restore and verify operations.

I had the issue disappear for days and ~20 TB of read operations, then randomly reappear.

System is Tolis Argest LTO-6 (HP drive), ATTO H680, Early 2008 MacPro, OS X 10.10.5. I have also tested on an early 2013 Mac Pro with HighPoint RocketStor 6328 and see the same thing.

When the system is in whatever mode it gets in to that results in slow read/verify times if I switch to demos of other LTO software (YoYotta, PreRoll Post) I still see the slow read/verify times.

TOLIS are suggesting possibly the Mac's memory was fragmented at the time of that job resulting in lots of small memory reads during the tape operation. But a reboot doesn't always seem to fix the problem. All I'm running on my system apart from BRU is Media Encoder which is used a handful of times a day for the odd transcode of ~ 1 minute long commercials.

TOLIS also got me to export an L&TT support ticket which didn't show any hardware faults with the drive.

After previous nightmare issues with LTO (a hand-me-down NovaBackup and LTO-4 system) this system is *almost* perfect, but would love to get to the bottom of these random slow read times.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]