Please explain how LTO/Tape drives archiving work
I landed a job on a post-production house where one work I'm assigned to do is manage the files generated from years of Pro-Tools and Davinci Resolve sessions, including raw DPX and ProRes for some of past DCP mastering projects. I'm looking at about dozens of Terabytes in need of archiving, and my boss told me to look into tape-drive backups.
I've done some reading into the matter, and by now I've learned that I need (1) the tape drive itself to insert the cartridges into, and (2) the tapes themselves and ultimately (3) the knowledge on how to make it all work (lol). We're looking into buying LTO-6 for our archiving since LTO-7 as I've heard is still crazy expensive nowadays.
Can someone please explain how backing-up files into tapes work using LTO-6? Do I just plugin the tape drive, insert the tape and it will appear as if it's a normal drive using Windows Explorer/Apple Finder, and proceed to drag and drop files into it? How does most tape drives connect into a PC/Mac? (USB3/Thunderbolt). I have also noticed that most tape cartridges include in their description text that says 2.5TB native/6 TB compressed, what does this mean?
Lastly, what brand do you suggest we buy? Thanks in advance
I got started with this super helpful Cow article a few years ago:
Regarding price differences: media is about the same no matter what. A $20 LTO-5 tape holds 1/4 the data as an $80 LTO-7 tape. The cost of the drive is the money sink; if you can buy used you might find a great deal.
> Do I just plugin the tape drive, insert the tape and it will appear as if it's a normal drive using Windows Explorer/Apple Finder, and proceed to drag and drop files into it?
You can do it that way using LTFS. After you format the drive (which requires an LTFS software compatible with your drive, often free) the tape appears as just another drive on your system. You can drag and drop onto it or whatever.
> How does most tape drives connect into a PC/Mac? (USB3/Thunderbolt).
SAS (serial attached scsi) but there are SAS-Thunderbolt adapters on the market (see article above)
> I have also noticed that most tape cartridges include in their description text that says 2.5TB native/6 TB compressed, what does this mean?
This is weird if you try to think of a tape like your HDD. Here's how it works: As you copy data onto a tape, the tape drive does what it can to compress that data. Sometimes the 10GB file you copy onto the tape can take up as little as 5GB on the tape. But we work in compressed media (video and audio is already compressed most of the time) so there's almost nothing the drive can do to compress it.
That in mind, 2.5TB is the realistic capacity of the drive BUT sometimes I'll copy ~2.5TB of data to a tape and it will still have some capacity (a few hundred GB) available after copy because some of that content was compressed while copying.
Magnext specializes in new & refurbished tape storage products and can walk you through the whole process. Give me a call at 614-433-0011 x114.
Regarding brands, all the tape drives are pretty equal except the interfaces.
The lowest priced brands are for NEW:
Quantum for SAS
MagStor for Thunderbolt 2
Unitex for USB 3
I always recommend picking up PreRollPost as well for LTFS software.
Both Benjamin and Tim have partially answered your questions but there is more to it. Here are some additional data points:
Yes, you can use LTFS and see tapes like a disk volume but a word of caution is in order when you are talking about using tape for raw DPX files. LTFS is a great solution for large files like your ProRes projects, but really thrashes when you are storing hundreds of thousands of DPX files per tape. Third party software can significantly improve this.
LTO Tape drives are only made by IBM and HP - Quantum, Tandberg and others just put their names on them.
The drives themselves only have SAS connections but the companies mentioned by Tim (Magstore and Unitex) package them with built-in interfaces to Thunderbolt and USB3. So do mLogic and 1beyond. Or you could connect a tape drive to any thunderbolt capable computer with an "ATTO SH 1068" which is ATTO H680 SAS card in a thunderbolt box. Companies also package the drives into complete archiving solutions that include server functionality and solve the connection/interface challenges more easily than trying to roll-your-own, but at a cost.
If you are a geek and like dealing with arcane command-line programs, the original Unix/Linux "tar" is probably best for DPX and other frame based source material. Third parties bring a lot to the LTO environment in terms of ease of use and are quite a bit better than the free LTFS software, but a lot of it is still based on LTFS or has it's own issues with "large number of files" archiving so depending upon how much of this you do, you may want to talk to vendors in depth to ensure that won't be an issue.
Some of the LTO archiving software vendors out there are active on this forum and should jump in here to make the case for their solutions. Amongst the Mac-based software solutions, I'd mention Preroll Post as did Tim, but also YoYotta, and the venerable Bru. I only know of one PC-based software solution - Archiware. Most of these vendors also sell hardware/software packages. And then there are a whole bunch of vendors who have "appliances" - all in one boxes that you network to or plug storage into. Some of these include ProMax, Xen-Data, Storage DNA, Codex, and 1beyond. Finally more and more of "shared storage" vendors can include LTO with their solutions - the list for this is long and includes the likes of Quantum StoreNext, Edit Share, and Facilis, as well as many of the appliance vendors mentioned above.
30201 Rainbow Hill Rd.
Evergreen, CO 80439
Don't forget Fibre-Channel for the connect options :).
CTO - TOLIS Group, Inc.
BRU ... because it's the RESTORE that matters!