FORUMS: list search recent posts

'Offline' tape backup, or 'image' and backup

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Neil Sadwelkar
'Offline' tape backup, or 'image' and backup
on Jun 27, 2018 at 3:55:51 am

Just like for DVDs or Blu-rays, one could take some data and make an image out of it, and then other software could write this image to disk. Is there something similar for LTO tapes?

So, if the LTO drive is not accessible, one could rapidly offload data to a drive as a 'tape image'. Then once that drive with the 'tape image' was connected to a system with an LTO drive attached, this image could get written to tape.

Is this feasible on any software? Or is this even possible within something like Bru?

The use case would be to use now available portable drives of 5 TB to create images with. Then ship these drives, and at the destination, write the image to LTO tape. Reason for shipping an image is that its more secure than just files. So if the drive were to be misplaced, the finder wouldn't be able to get at the data very easily.

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

Jerzy Zbyslaw
Re: 'Offline' tape backup, or 'image' and backup
on Jun 27, 2018 at 9:29:04 am

Its INCREDIBLY EASY if you use the ZFS file system that comes with either Solaris, one of the BSD's like https://www.freebsd.org/ which are the rock solid ZFS systems, you can also install ZFS On Linux (ZOL) http://zfsonlinux.org/ which should be OK but it's not regarded as production stable as the first two options.

The reason I say this is that you can create a ZFS file system on either entire disks, or slices (Solaris way of doing partitions) or just plain ORDINARY FILES, see here for an explanation using Solaris https://www.thegeekdiary.com/zfs-tutorials-creating-zfs-pools-and-file-syst...

You then mount the file and access it as you would do a regular disk, so in your case where you said you had a 5TB disk you could for example create say a 2.45 TB file and mount it, you then copy data to it until full and then you unmount it, you then copy the 2.45 TB file to a 2.50 TB LTO6 tape in your tape drive using regular unix TAR, DD, or CPIO commands and then post the LTO6 tape in the mail (it doesn't only have to be an LTO tape as it could just as easily be an RDX cartridge)

The recipient at the other end just has to reverse the process by loading the tape into their drive and copying the monolithic file to their hard drive or raid array and simply mount it (obviously using ZFS as well, and preferably using the same version of ZFS) and then they can easily access the data.

Because you are creating ZFS using files and not disks or partitions and therefore you don't have to worry about passing through disk accesses through to the VM itself (e.g. fiddling with VM and BIOS settings for LSI SAS cards and SAS drives) then the easiest way to do this using your existing computer would be to get a free hypervisor like https://www.virtualbox.org/ which runs on "Windows, Linux, Macintosh, and Solaris hosts" or you could use Hyper-V with Windows and I think the Mac has https://www.parallels.com/ available and simply install one of those three ZFS capable systems although to use Solaris professionally it would have to be licensed whereas the BSD's and ZOL would be entirely free, you don't have to have a 2.5TB VM as you can just set aside say 50GB for the VM's operating system and just give that VM network access to let it create a 2.45TB file somewhere else on the computer, network or NAS, once your running the VM and in there you mount the file, then copy 2.45TB's worth of data from somewhere else to that filesystem (the 2.45 TB file), unmount the file and shutdown and exit the VM.

So to answer your question "Is this feasible on any software?" yes it can and it can even be totally free but the only downside is that suddenly you have to learn one of either unix, Solaris, linux, or FreeBSD but this would be a very good investment for yourself now and in the future especially as data storage sizes are set to explode with 4K and 8K coming in, another benefit is that with ZFS each block in the filesystem if fully checksummed so if it gets sent to the destination and has any errors you'll know about it if you do a Zpool Scrub command on the entire filesystem, more introductory information about ZFS is here https://pdfs.semanticscholar.org/presentation/bba6/a91ab9573e4284d041b0ae1c... and it is far better than Windows NTFS, Linux's Ext4 and the Mac's APFS.

Lastly, a second answer to your question is if you don't want to use ZFS you could just as easily create a 2.45 TB VM file using whatever hypervisor you want containing say an OS of your choosing and as an example together with say a free version of Blender and the Blender files you are working on and then you shut the VM down and then send that actual VM file (together with any relevant config files) and just send all that on the LTO tape, the recipient just has to import the VM into their hypervisor and fire it up. e.g. this article here https://www.techrepublic.com/article/how-to-move-virtualbox-vms-from-one-dr... and although this is just as do-able you miss out on the integrity features that ZFS provides and so isn't my preferred recommendation but this would suffice for say an interim period until you get up to speed with ZFS.

I think the ZFS file and VM image is the only real neat and tidy way to virtualize file systems as these can be any size whereas actual disk images are usually for set sizes (unless you just create a primary partition that doesn't fill the entire disk) and also contain pesky information like MBR and GPT partition table information that you also have to deal with when transferring which makes actual disk images a bit messier to deal with.

Cheers


Return to posts index

Tim Jones
Re: 'Offline' tape backup, or 'image' and backup
on Jun 27, 2018 at 3:27:31 pm

Hi Neil,

The key to such a mechanism is in the format that the archive container is written. As long as the container format for the two types of containers is the same, it is quite "doable". You just need a tool that can tape the file and write it as raw data to the tape so that the format and any metadata in the container is retained.

BRU Server already does this (D2D and D2D2T) and you can do it through a script with BRU PE, disk archiving and the tapewrite command line tool that is included. Drop our support team a note for more details.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


Neil Sadwelkar
Re: 'Offline' tape backup, or 'image' and backup
on Jun 28, 2018 at 11:56:10 am

Thanks Tim.
I was wondering if I make a disk backup using Bru-PE, can I take the resulting file (its named .bru?) and then write that to tape somehow?

-----------------------------------
Neil Sadwelkar
neilsadwelkar.blogspot.com
twitter: fcpguru
FCP Editor, Edit systems consultant
Mumbai India


Return to posts index

Tim Jones
Re: 'Offline' tape backup, or 'image' and backup
on Jun 28, 2018 at 3:42:13 pm

Hi Neil,

Yes - the BRU container format is the same on disk and tape, so you would use the "tapewrite" command that is part of BRU PE.

There are a lot of more advanced things that you can do, but the simplest mechanism is:
cat /PathTo/archive.bru | tapewrite -b 128k -f ntape0
If you shift to BRU Server, it's an automatic process called "UpStage". When you set the BRU Server job for Disk 2 Disk, you can then follow up at a later time with an UpStage that automatically transfers the disk file to tape with the options of deleting the original disk archive or retaining it for local operations while sending the resulting tape(s) offsite.

As always, contact the support team if you have any specific workflow questions.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2018 CreativeCOW.net All Rights Reserved
[TOP]