FORUMS: list search recent posts

LTO-7 and Mac Platforms

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Tim Jones
LTO-7 and Mac Platforms
on Sep 30, 2016 at 5:30:15 pm
Last Edited By Tim Jones on Sep 30, 2016 at 6:56:27 pm

Hi Folks,

I wanted to provide an update about using LTO-7 tape drives on a Mac or Linux system. As always, my team continuously tests tape technology against all platforms and we have identified a serious data reliability issue if you're using LTO-7 with a Mac or Linux platform. Regardless of the software that you use - BRU, Retrospect, or an LTFS-dependent solution, the only HBA / Thunderbolt solution that is working properly is the HighPoint Technologies RocketRAID 4522 or their RocketSTOR 6328 with the 1.8.11.1 firmware. Any other SAS HBA / Thunderbolt solution will result in serious speed issues (at best) and randomized, unrecoverable read errors from the tape (at worst).

In addition to the resolution that HighPoint has provided, we are working with ATTO to help identify and resolve the issue with their H680 and R680 HBAs, but we don't have an update or ETA for a resolution.

At the present time, there is no chance for a resolution with the ARECA or generic LSI SAS HBAs since they don't provide proper tape support under OS X.

If you are using an mLogic mTape LTO-6 or LTO-7 Thunderbolt drive, be sure that it is a revision 2 drive (uses the HighPoint HBA internally instead of the ARECA) and that you get the latest firmware and drivers from mLogic for OS X.

If you are currently using LTO-5 or LTO-6 drives, you are okay with either the ATTO or HighPoint solutions. However, if you are looking at using LTO-7 for your backup, archival, or interchange needs, please be sure to only consider HighPoint SAS connectivity until ATTO provides new firmware and drivers to resolve this problem.

If you are currently using TOLIS Group's ArGest tape solutions with the HighPoint RR4522 or RS6328, please open a support ticket at support.bru.com for information on how to download and apply this latest firmware and driver to your system.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Tim Gerhard
Re: LTO-7 and Mac Platforms
on Oct 24, 2016 at 10:08:41 pm

You're saying that LTO7 on MAC's and Linux are all having speed issues, is there a specific version level of MAC OS or Linux that you're seeing a decrease in speed?

I frequently test IBM, HPE, and MagStor tape drives with PreRollPost, YoYotta, and the basic HPE StoreOpen software, and haven't seen any issues, nor have I had any customer complaints. All of my customers use the mentioned software with an ATTO H644, H680 or Thunderlink 2068. Is this just an isolated issue with just BRU?

Tim Gerhard
Magnext
614-433-0011 x114
tgerhard@magnext.com


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Oct 25, 2016 at 4:36:26 pm

[Tim Gerhard] "You're saying that LTO7 on MAC's and Linux are all having speed issues ..."

Yes - IF you're not using the HighPoint RocketRAID 4522 or RockstSTOR 6328 with their 1.8.11.1 firmware (and the 4.4.1 IOP driver on Mac).

Are you saying that you're seeing 300MB/sec+ with the ATTO solutions? Interesting since ATTO can't even get there. Even with specifically tuned data sets and I/O setups, we're maxing out in the 220-230MB/sec range. Also, both we and ATTO are witnessing read checksum errors with the R680 specifically (I've updated my original post to remove that comment WRT the H680).
[Tim Gerhard] ... is there a specific version level of MAC OS or Linux that you're seeing a decrease in speed?

The version of OS X doesn't matter - 10.9.5 through 10.12.1, nor does the Mac system type. For Linux, we've tested with kernel versions 2.6, 3.X, and 4.X (tape I/O is kernel-dependent, not distribution dependent). Under Windows, we've used Windows XP, Server 2008, Server 20012, Windows 7, Windows 8.1, and Windows 10.

It's not surprising that you'd not witnessed what we're seeing if you're only testing with LTFS-based tools. Since LTFS-based solutions such as the ones that you describe don't manipulate the tape drive directly, they are only aware of issues if the tape layer reports them. Unfortunately, LTFS doesn't provide the levels of reporting from the tape layer that would truly identify the problems that we are witnessing. They can only report absolute failures or "after the fact" failures based on any form of checksum sidecar that is created (MD5, SHA1, etc.). Because BRU is talking directly to the tape drive at the SCSI / IOP layer with no "middle-man", we are always aware of the state of the drive through both sense request and Tape Alert calls on our command channel for both write and read operations.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Luke Mullen
Re: LTO-7 and Mac Platforms
on Jan 16, 2017 at 9:57:31 pm
Last Edited By Luke Mullen on Jan 16, 2017 at 10:01:16 pm

Hey Tim...

I'm curious if you've gotten any farther with your testing. I'm working with a local non-profit to help them set up an LTO-7 archive solution. They purchased a standalone HP LTO-7 Ultrium 15000 tape drive. I have it running on a custom built Windows 10 system through an ATTO H680. I've downloaded and installed the latest firmware for both the H680 and the LTO drive, but it's running slow. Using LTFS and PreRollPost. They have lots of large DPX sequences that they need to archive, and I was thinking the nature of trying to backup tens of thousands files was part of the slow down and beyond that maybe PreRollPost itself was having issues writing that many files, but after reading your post I'm wondering if there's something else going on. PreRollPost doesn't give a speed in it's copy details but just based on the time, it's taking waaaay too long. And way too long just for the copy step before we even hit the XXhash checkum verification step. For example, we tried a 130GB folder last week, around 11,000 12MB files plus a 125MB sound file and it took over 2 hours for a full copy and verify. Conversely, just dragging and dropping the folder to an LTFS formatted tape in Windows Explorer took about 20 minutes and claimed to be running around 150MB/sec...still way slower than LTO-7 should be able to achieve. I haven't talked to Imagine yet, and I know that's my next step, just gathering as much info as I can first and also checking the performance on the Mac build of PreRollPost before I send that email. But I came across this post and wondered if you'd gotten anywhere with your testing in the last few months. Any info you have would be appreciated.


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Jan 17, 2017 at 2:55:19 am

Hi Luke,

No changes. Right now, only the HighPoint RR 4522 (PCIe) or the RS 6328 (Thunderbolt) options will actually drive the LTO-7 devices at top speeed on a Mac. ATTO has resolved the lost/bad block read issue with the R-series cards, but the speed is still where it is.

And, since PreRoll Post and many others are dependent on LTFS for tape I/O - they don't actually write to the tape drive - they will only be as robust as the LTFS layer for the vendor's drive that you are using. We've been testing all versions of LTFS along with BRU through all of this and until something changes at the core of LTFS on OS X, you're going to be stuck with what it is.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Neil Sadwelkar
Re: LTO-7 and Mac Platforms
on Feb 24, 2017 at 3:34:40 am

Has this issue been resolved. The speed, or lack of it, I mean, on a Mac with Atto HBAs

I'm about to place money on a Tandberg LTO-7 drive along with an Atto ThunderLink SH 2068 TB2-SAS adapter, (which is suspect is internally an H680). Mainly for its size as this will be a 'travelling' setup.
But, since I expect to be backing up Thunderbolt or USB3 RAIDs, a speed cap of 230 MB/sec would be a disadvantage.

Which comparable Highpoint HBA would be a better choice? How about a Rocketstor 6328 or 6328L/6324L since I don't really need the RAID capabilities while driving an LTO-7 drive.

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index


Tim Jones
Re: LTO-7 and Mac Platforms
on Feb 24, 2017 at 5:59:36 am

Nope, and according to ATTO it won't be. They seem to be satisfied with the lower tape speeds. Since you're buying new, I really recommend the HighPoint RocketSTOR 6328. Do not use the L versions as they are "limited" and don't support tape. This will give you both hardware RAID and top speed for tape and disk.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Neil Sadwelkar
Re: LTO-7 and Mac Platforms
on Mar 7, 2017 at 10:34:14 am

I just came across this page at Yoyotta's site which bears what Tim said about speed and LTO-7 drives...

https://yoyotta.com/help/installRocketStor.html

To quote...

"The RocketStor does not support the TLR protocol so it is not compatible with the higher speed of LTO-7 drives
If using an LTO-5 or LTO-6 library use firmware v1.8.1.0 as newer firmware v1.8.8.0 will not format and mount tapes.
There are other models like the 6328L, which will not work with LTO drives."

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Mar 7, 2017 at 4:32:34 pm

... unless you're using a RocketSTOR 6328 or RocketRAID 4522 purchased from TOLIS Group as part of one of our bundles:
bru: [I181] wrote 53680128 blocks (107360256 KBytes) on volume [1], 0:05:34, 321437 KB/sec

That 100GB quick test was a combination of R3D, DPX, and M2TS files located on one of our ArGest RackMOD arrays.

We have developed additional firmware and drivers for our units that does resolve this issue.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Michael Cinquin
Re: LTO-7 and Mac Platforms
on Mar 22, 2017 at 8:37:01 pm

Tim,
are your advices up to date?
I just checked the checksums of my first 5.54TB of data stored on an IBM LTFS LTO7 drive, through an Areca SAS HBA, and not a single bit was corrupted.
Write speed was 230 MB/s (room for improvement, but not horrible for ltfs), read speed was slower than ltfs on LTO6, with only 100MB/s.
So I'll be looking next at changing the SAS card. The RR4522 you recommend is EOL though; do you know something about the RocketRaid 3740A that you could share?
thanks!

Michael Cinquin

Charbon Studio, film finishing and DCP mastering
Post-production tools


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Mar 22, 2017 at 10:42:04 pm

Hi Michael,

Still no change to this status. And, if you're not having issues with an ARECA card, it is truly luck that is driving your efforts and not technology.

While the RR4522 is reported as EOL, there are still plenty available. We have not had a chance to test the 3740 series as yet. I will post a new thread once we know more.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Mar 22, 2017 at 11:19:56 pm

Additionally, I forgot to mention that the RR4522 is EOL, but that it has been replaced with the RR4522SGL. It's the same chipset and card components, simply in a different carrier and retail box.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Michael Cinquin
Re: LTO-7 and Mac Platforms
on Mar 23, 2017 at 2:25:07 pm

[Tim Jones] "it is truly luck that is driving your efforts and not technology."

Well, you call it luck, I call it expected behaviour of a brand I appreciate.

I might have mis-read your first post, it really seemed like every SAS card was malfunctioning except the RR4522.
Now it appears you have no feedback from using current HighPoint cards nor current Areca non-raid SAS card.
(Of course you know this but for others reading: Areca SAS card come in Raid and non-Raid versions, and the Raid versions are known not to work with tape drives).

I have not yet resolved the speed issues with ltfs-7 reading, but the HBA card doesn't seem to be the culprit… as Bru writes and reads from the LTO-7 deck at blazing speeds.
20170323 14:33:39|2603|charbon|[L163] START (r 18.1.1.28), CMD = '/usr/local/bin/bru -c -m -vvvvvvvvv -j -O -A -QB -f ntape1 -QX -L 2017-03-23 A073 Backup -'
20170323 14:33:39|2603|charbon|[L167] device = ntape1, buffer = 2048K bytes, media size = <unknown>, archive id = 58d3ceb30a2b, (Op)
20170323 14:36:17|2603|charbon|[L182] wrote 21757952 blocks (43515904 KBytes) on volume [1], 0:02:38, 275417 KB/sec
20170323 14:36:20|2603|charbon|[L165] FINISH - 0 warnings, 0 errors, exit code = 0
++++++++++++++++++++++++++++++++++
Last BRU PE Verify Execution summary:

20170323 14:37:42|2813|charbon|[L163] START (r 18.1.1.28), CMD = '/usr/local/bin/bru -ivvvvjf ntape1'
20170323 14:37:42|2813|charbon|[L167] device = ntape1, buffer = 2048K bytes, media size = <unknown>, archive id = 58d3ceb30a2b, ()
20170323 14:39:41|2813|charbon|[L182] read 21757952 blocks (43515904 KBytes) on volume [1], 0:01:59, 365679 KB/sec
20170323 14:39:41|2813|charbon|[L165] FINISH - 0 warnings, 0 errors, exit code = 0

Congratulations on Bru by the way, if we didn't have our own LTFS and cataloguing workflow, would certainly be an option, the software looks very well written.

Bru would also be an option if it was fast and good at ltfs, but for the moment I can't get Bru to format a tape as LTFS, it fails to find the open-source mkltfs binary. (I wrote your support about this)


Return to posts index

Allen Baldwin
Re: LTO-7 and Mac Platforms
on Apr 14, 2017 at 1:49:42 pm

Hey Tim! I need those new files, firmware and IOPS. The drivers and firmware available for download on Highpoints website are outdated. Where did you get those? could you post a link?


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on Apr 18, 2017 at 9:04:28 pm

To Allen and all -

Because of the level of added support required when dealing with customers and tape devices, HPT has decided to NOT provide the firmware and drivers that we worked on to resolve the issues with the RR 4522 and RS 6328. While TOLIS Group has them, we are only allowed to provide them with units purchased as part of TOLIS Group kits; this was not my call.

Therefore, if you're looking at running LTO-6 or LTO-7 drives on a Mac platform, TOLIS Group is the only source for solutions that will work properly under all situations.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Lucas Meyer
Re: LTO-7 and Mac Platforms
on May 29, 2017 at 6:19:36 am

Hi Tim,

I tried one of your competitors LTO-7 systems (which used LTFS and hooked up to my iMac via a Thunderbolt 1 cable). I found that I could back up individual files or small libraries (under 50 GB), but I got errors backing up regular sized libraries of 100 GB or more. The competitor's support guy thought that the problem was with the tape drive hardware, but after having to send two of the drives back, I gave up. I am wondering if I was having the problem that you are describing.

So I am looking for a LTO-7 to plug into my iMac (with Thunderbolt 1) or my MacBook Pro (with Thunderbolt 2). If I order the ArGest (LTO-7) with the RocketStor 6328, will it have the necessary firmware or will I need to open a ticket? (I am hoping that I can just plug it in and use it...)

Also is the correct order number #80710-TB? I just want to check that I have the right package.


Thanks
Lucas


Return to posts index

Tim Jones
Re: LTO-7 and Mac Platforms
on May 29, 2017 at 6:52:47 pm

Hi Lucas,

Our kit is complete - proper firmware and proper drivers. The problems that you are witnessing are so frustrating because we know that tape is a reliable mechanism for both backup and archival. Unfortunately, so many people run into what you're seeing and write tape off as a non-starter.

Just to be a bit more clear on the issues that you witnessed - you will see this with LTFS. Many will argue this with me because they've had no problems - and to them I say fantastic, but we see these on all four platforms we test - Linux, Windows, OS X, and Solaris.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Lucas Meyer
Re: LTO-7 and Mac Platforms
on May 29, 2017 at 9:21:56 pm

Hi Tim,


From what I was seeing, the LTFS system was working perfectly for individual files and very small libraries. However, when I started backing up libraries of 100 GB or more, I got loads of errors and never succeeded even once to get a big library to backup properly after weeks of trying. I think, if people just tested the system with one or two small files, they might not know that they had a problem.

Just to make sure that I understand:

1. The kit comes with BRU and BRU does not use LTFS. Correct?

2. And the order number for correct kit (The package with the ArGest Tape Drive, RocketStor 6328, BRU and connecting cables for Thunderbolt 1 or 2 that works with Macs.) is #80710-TS. Correct?

Sorry, ask this, but you are right that it has been frustrating having to sort through all of these problems and send two units back. I want to make sure that everything is OK, so I can just plug it in and use it when it arrives.


Thanks
Lucas


Return to posts index


Tim Jones
Re: LTO-7 and Mac Platforms
on Jun 6, 2017 at 6:20:51 am

Hi Lucas,

Sorry, I thought that I'd replied to this.

Yes, that's the complete part number.

The kit comes with either BRU Producer's Edition or BRU Server M&E Edition.

BRU PE "supports" LTFS in that we will format and mount LTFS tapes, or catalog tapes you've received from others to make them searchable within BRU PE's restore environment, but we do not USE LTFS for actual backup/archival. LTFS is designed so that you don't need extra software to copy files onto and off of tape, but it also lacks the features for management and reliability that using a format like BRU provides.

BRU Server does not support LTFS in any manner.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]