FORUMS: list search recent posts

Offloading Speeds on Mac Pro with mTape to LTO-6..

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
suny behar
Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 4:48:14 am

Hey Folks,

I have about 20TB of 4K DPX files (in separate folders). So each folder contains about 1 minute worth of 4K DPXs (about 55MBs each x 1 minute) so about 600GB.

I have had ZERO luck trying to archive these to LTO.

I have a 24TB ARECA RAID 5 Drive (thunderbolt), and an mTape (thunderbolt enclosure). Installed latest drivers, running 10.9.5 on a Cylinder Mac Pro...

First tried PreRollPost, going the LTFS route. Would set it to offload, and would wake up in the morning to either 1) a frozen Mac, (off we go with a restart) 2) a message that said I had only transferred half the media and had 40,000 errors in copy and in verification. When I click "Details" , got nothing...

Then downloaded BRU PE to try my luck there... and well.. not a ton better...

1) First attempt Froze mac after about 60GB...

then I re-installed Mac OS X, re-installed all drivers, re-installed BRU PE...

2) This time, offload has proceeded... but at a SNAILS pace.... it offloads for about 5 seconds... pauses for about 30... then offloads for another 5 and pauses for 30... I'm getting around 100GB per hour of archive... At this pace, I think I will die of old age before I am prompted to insert the next tape...

Any help would be appreciated ;)


Return to posts index

David Roth Weiss
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 6:32:38 am

Did you download the latest drivers from mTape? I believe there may be a special one compatible with BRU PE.

Have you called tech support at mTape or Tolis (BRU)?

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

suny behar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 6:43:42 am

I did download the latest drivers from mTape. I could not find a special one, there is only one driver on mTape's website...

I contacted tech support at BRU and so far was only told to make sure I have the latest drivers...

that's getting pretty old ;)


Return to posts index


David Roth Weiss
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 6:45:59 am

I understand your frustration. You should definitely call mLogic.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Martin Greenwood
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 10:45:39 am

As you have tested with two different archive apps it could be a faulty drive, to help troubleshoot you could download our app YoYottaID LTFS.
When run it will check that you have all the correct frameworks for LTFS and driver for mTape.
If anything is needed it will download and update to the latest versions.

http://yoyotta.com/help/demo.html

If you have the same problems, then please contact me and I can take a look at the Mac system log to troubleshoot.

Martin Greenwood
martin@yoyotta.com
CTO YoYotta


Return to posts index

suny behar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 7:15:46 pm

Hey Martin,

Thanks for the offer, that's very kind of you.

I will download the demo and see what I get.

Cheers!

Suny


Return to posts index


Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 4:45:25 pm
Last Edited By Tim Jones on Feb 10, 2015 at 4:56:31 pm

Hi Suny,

Unfortunately, all that we can offer is to insure that you've downloaded their latest firmware and drivers. Unlike ATTO and HighPoint, the Areca team has been very strict with their work in resolving the tape I/O issues that we identified in their driver / firmware stack over 5 years ago. It wasn't until they discovered that their drivers didn't work for the mTape unit that they even tried to make any effort in resolving the issue.

I have contacted the mLogic team. In the mean time, our engineers are building a version of BRU PE that may provide a more detailed examination as to why the data flow is so slow. Check your support ticket for a download link later today.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 6:09:50 pm
Last Edited By Tim Jones on Feb 10, 2015 at 6:11:20 pm

One additional bit of information that our lab team just shared with me - the Areca RAID cards are known to have heat problems when backups are running. Unlike normal disk I/O when you are writing data in bursts, the act of reading data from the device for backup is a continuous stream of data with very few breaks (especially with LTO-6). He have had our Mac Pro and Windows systems physically shut down because the Areca card overheats. When we shared this issue with Areca US support, their guidance was to open the case and aim a desk fan at the system.

While it is unfortunate and something that I don't like to have to do, we now recommend to our customers to stay away from Areca RAID adapters for non-server installation as they are simply not stable in extreme I/O situations where you are not running them in a controlled server infrastructure environment (forced cooling, humidity control, etc.).

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

suny behar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 10, 2015 at 6:52:21 pm

Thanks for the info Tim!

In this case, the ARECA Raid controller card is in the free standing 8 Bay RAID Drive chassis (not inside the cylinder Mac Pro which does not accept PCIe cards obviously). Does this advice still apply? The RAID drive chassis is connected to the Mac Pro via Thunderbolt. Are they recommending that I remove the case from the RAID Drive chassis to expose the drives and card? Thanks for clarifying.


Return to posts index


Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 11, 2015 at 4:15:36 pm

Suny,

It could very well be their enclosure. Since you're running with 8 drives, there's a lot of heat generated - especially during a backup / archival operation where you're not simply copying files. In our tests, we were able to burst write or burst read from an Areca-controlled array, but when we started continuous and contiguous reads from the controlled array, the HBA would overheat and the Mac wold either hang or fully gray-screen.

As contrived as this may sound, if you have a small desk fan, you might try placing it in front of the case (open the front door if there is one) and blow air across the drives to reduce the heat that they output. This may be enough to reduce the heat profile within the case and keep the Areca controller under critical temps. It's what Areca's tech rep suggested to us when we reported the issue and it did work when the fan was on medium or high speeds.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Neil Sadwelkar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 17, 2015 at 5:12:09 pm

I haven't used the Areca Thunderbolt RAID 8050 myself, (http://www.areca.com.tw/products/thunderbolt.htm) but have known other DITs use it as a on-location backup for camera data and it has been reported to work flawlessly even in 'non-ideal thermal' environments. And they've used the Areca along with StorageDNA so Areca to LTO is also a working combo.

What you could try is to disconnect the mLogic LTO drive, and instead connect any USB3 drive to the MacPro and, of course, the Areca 8050 to the Thunderbolt port of the MacPro.

Next, simply do a Finder copy of about 1.5 TB of data from the Areca to the USB3 drive.
This will sort of simulate an LTO backup. If the Areca successfully streams this large amount of data to the USB3 drive, then the Areca is fine and capable of reading large amounts of data.

Next, you take this same 1.5 TB data now on the USB3 drive, and, with the Areca disconnected, copy it over to LTO tape in the mLogic drive. Using Bru-PE. If this too works, then there's is some compatibility issue between the Areca and the mLogic.

If, on the other hand, this copy (Areca to USB3) doesn't succeed, then the Areca might need to be replaced.

BTW how did this data get into the Areca in the first place. If it copied this huge amount of data successfully, then the unit seems to be working fine.

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 17, 2015 at 5:57:51 pm
Last Edited By Tim Jones on Feb 17, 2015 at 5:58:54 pm

[Neil Sadwelkar] "BTW how did this data get into the Areca in the first place. If it copied this huge amount of data successfully, then the unit seems to be working fine."

Actually, while that seems to make sense, it's not a good measure of stability. When copying data to and from the filesystem, applications read and write data in chunks and the filesystem manager is caching even larger segments of data for burst writes and reads. Also, the low level drivers utilize memory management capabilities known as "scatter gather segmentation" to provide both improvements to observable performance and memory utilization. If you tracked actual drive I/O, you would see that this allows the drive to be quiesced for large periods relative to the actual application use. A quick test copying 300GB of data from one array to another at over 400MB/sec showed disk activity in the 42% average range (use cycle). This is the same mechanism that LTFS-based solutions use since they are running through the FUSE filesystem manager.

When BRU is reading data from the drives, we are using a much more direct API and there is no caching or scatter gather operations. Using BRU and reading that same 300GB of data from the array resulted in a disk use cycle of over 88%. This is also creating a much higher level of I/O stress on the RAID controller chipset. Combining these two heat generation situations will easily overdrive the Areca chipset's heat profile limits. This is why simply adding a fan blowing over the cards resolved the crashes in our lab.

There is a very big difference in heat generation between 42% and 88% usage cycle for 8 spinning hard drives.

For the record, we have never seen these types of crashes / hangs with ATTO or HighPoint RAID HBAs or chipsets.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Neil Sadwelkar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 17, 2015 at 6:50:03 pm

Oh wow. That figures.

Thanks Tim for the detailed explanation.

So that means the OP is out of luck being able to use this RAID for LTO backup with Bru-PE, since the RAID has an Areca (Thunderbolt to) SAS RAID card built-in and cannot be separated from the drives.

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

David Roth Weiss
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 17, 2015 at 7:23:53 pm

When I was at Promax we always used two controller cards for every NAS system we would build that included an LTO, a RAID controller for the hard drives and a separate HBA controller for the LTO. Would that not solve the problems in this case?

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions


David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 17, 2015 at 7:42:55 pm

Hi David,

In this case, the Areca HBA is a separate controller. In our lab environment, the Areca fell over when it was ONLY used to control the DASD.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Bob Zelin
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 18, 2015 at 1:27:19 am

boy - I must be missing something here. I know that I am getting old - but..........

1) The Areca ARC-1880x, 1882x, 1883x does not work with HP Ultrium tape drives. BUT you are NOT working with these cards. You are working with an MLogic MTAPE with PreRoll Post on a Mac Pro cylinder - is that correct ? So you don't need any host controller card - it's inside the MTAPE. this plugs directly into the Thunderbolt buss on the Mac computer.

2) Your Areca disk drive array (ARC-8050T2) has the equivalent of an ARC-1883x inside it - this is completely independent, and should be on an independent buss on the Mac Pro Cylinder.

If you DISCONNECT your ARC-8050, and just try to use the MTape with PreRoll Post to backup anything (like a USB drive) - does this work ?

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com


Return to posts index

Bob Zelin
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 18, 2015 at 1:30:19 am

http://www.imagineproducts.com/index.php?main_page=product_info&cPath=9&pro...

If the price of the product I have listed above is too expensive, then I will have to kill you. Try this with your MTAPE, and let me know what happens.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
bobzelin@icloud.com


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 18, 2015 at 4:56:31 am

Hi Bob,

The issue isn't the Areca-powered mTape unit. The Areca mTape firmware and OS X driver (it's the combination) for that specific chipset has been tested and in fact, part of the Thunderbolt approval involved both Apple and Intel testing the unit with BRU PE. That unit works properly as we worked with mLogic and Areca to get it to where it is.

The frustrating result is that even though the Areca team now have the answer to the tape compatibility issue, they're not expanding the changes to include their general population RAID HBAs.

They're also apparently aware of the heat profile issues as it was their USA rep that recommended the fan solution to us 3 years ago.

I'm hoping that we hear more from Suny and his results with the new BRU PE build and using alternative source drives for the tests to fort her isolate the Areca array from the equation.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


suny behar
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 8:17:16 am

First of all. THANK YOU all for your input, ideas and suggestions.

I have been sidetracked on a job, and have not been able to get back to the thorough testing I was hoping to yet. But plan on setting the next couple of days aside to do just that.

To answer a few questions: The ARECA RAID got filled with about 12TB of DPX files at a scanning facility that offloaded the DPX scans from their SAN to the ARECA RAID via a Thunderbolt connection between the ARECA RAID and a Fiber connected Mac Pro.

As has already been answered, the Controller card is built into the mTAPE. So currently I am dealing with essentially a hardware RAID controller inside the free standing Thunderbolt ARECA RAID, and a Thunderbolt controller card inside the mTape.

I have currently transferred around 4 TB to an external USB 3 drive in preparation for tests backing from that drive to the mTape while taking the ARECA RAID offline.

Right now I am testing: BRU PE, PreRoll Post, and YoyottaID LTFS.

Right before I got pulled away from this task, the last thing I did was try Yoyotta (since the 2 other systems would not let me get through 1 tape successfully- this was prior to receiving an update from the gentlemen at BRU).

I can report that YoyottaID did allow me to offload to the mTape from the ARECA RAID without any crashes (Martin thanks for the assistance!). However, the current offload speed seems quite unpredictable. All of my DPX frames are the same size : 55MB/frame. Yet, with verification turned on. One tape offloaded around 200GB/h, the next dropped down to 125GB/h, then the following was back up to 195GB/h.... but they did offload without hanging or crashing the system. I watched the tape and noticed it take several pauses, then restart itself. At one point, it even spit out the tape by itself, re-ingested it by itself, and kept offloading... All of this is probably adding to the erratic offload speeds.

I plan on trying these same files one more time with PreRollPost and the new build of BRU PE I got from Tolis and will report back.


Return to posts index

Simon Blackledge
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 6:08:45 pm
Last Edited By Simon Blackledge on Feb 19, 2015 at 6:09:11 pm

Surprised by this. Have 2 of the 8050T2 units and they have been great. The drives on the other hand! :)

How can an LTO6 mech pull data that a 900MBs+ raid can't handle with ease ?

I know with Archiware P5 Backup the sw has a buffer. Do these apps not ?

Or is it that if you pull a steady streams from the 8050T2's the raid card just overheats ? Surely thats not right :-/

So a Mac mini with an 8050T2 and an MTape daisy chained off the back of the 8050 is a no no ? And no manufacture is stating this compatibility issue?



Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 7:43:25 pm

[Simon Blackledge] "Or is it that if you pull a steady streams from the 8050T2's the raid card just overheats ? Surely thats not right :-/"

Unfortunately it's right. But, it ONLY applies to the Areca HBAs / Chipsets. The HighPoint and ATTO HBAs do not suffer from the issue. There's a reason the Areca cards are so must less expensive...

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Simon Blackledge
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 8:51:26 pm

So if I attach a usb3 1tb drive to another port on newmacpro and copy 1TB from the 8050 to it the 8050 would in theory overheat?

Or is this just streaming to tape using Bru? Archiware has a sw buffer that fills then streams. Is Bru similar?

Wouldn't say Areca are cheap compared to Highpoint.


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 9:21:30 pm

Nope - that copy operation still uses the filesystem caching and scatter-gather that I discussed earlier. So that we can demonstrate that it's not BRU, try this:
cd /Volumes/1st_Areca_Array
sudo tar -cvf - | (cd /Volumes/2nd_Areca_Array ; tar -xf -)

Give it 20 minutes or so and - Boom!

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Simon Blackledge
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 9:33:53 pm

Can I use a usb drive as 2nd volume using that command?

All one command?

Will it hose the areca or just need a restart?


Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 19, 2015 at 9:48:00 pm

It shouldn’t hose the Areca, but may as it can actually gray-screen the Mac.

Make sure that the secondary drive is fast enough to handle the output of the original array, otherwise, the read speed will be reduced enough to invalidate the test. I would recommend an SSD via TB or at least a 2 drive stripe USB-3 array.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Simon Blackledge
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 23, 2015 at 12:05:18 pm

Are MAXX DIGITAL aware of this ? I think their 8 bats are Areca also :-/

s



Return to posts index

Tim Jones
Re: Offloading Speeds on Mac Pro with mTape to LTO-6..
on Feb 23, 2015 at 4:54:46 pm

Not sure - I'll ping Devon.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]