FORUMS: list search recent posts

Re: 'Offline' tape backup, or 'image' and backup

COW Forums : Archiving and Back-Up

Respond to this post   •   Return to posts index   •   Read entire thread

Jerzy Zbyslaw
Re: 'Offline' tape backup, or 'image' and backup
on Jun 27, 2018 at 9:29:04 am

Its INCREDIBLY EASY if you use the ZFS file system that comes with either Solaris, one of the BSD's like which are the rock solid ZFS systems, you can also install ZFS On Linux (ZOL) which should be OK but it's not regarded as production stable as the first two options.

The reason I say this is that you can create a ZFS file system on either entire disks, or slices (Solaris way of doing partitions) or just plain ORDINARY FILES, see here for an explanation using Solaris

You then mount the file and access it as you would do a regular disk, so in your case where you said you had a 5TB disk you could for example create say a 2.45 TB file and mount it, you then copy data to it until full and then you unmount it, you then copy the 2.45 TB file to a 2.50 TB LTO6 tape in your tape drive using regular unix TAR, DD, or CPIO commands and then post the LTO6 tape in the mail (it doesn't only have to be an LTO tape as it could just as easily be an RDX cartridge)

The recipient at the other end just has to reverse the process by loading the tape into their drive and copying the monolithic file to their hard drive or raid array and simply mount it (obviously using ZFS as well, and preferably using the same version of ZFS) and then they can easily access the data.

Because you are creating ZFS using files and not disks or partitions and therefore you don't have to worry about passing through disk accesses through to the VM itself (e.g. fiddling with VM and BIOS settings for LSI SAS cards and SAS drives) then the easiest way to do this using your existing computer would be to get a free hypervisor like which runs on "Windows, Linux, Macintosh, and Solaris hosts" or you could use Hyper-V with Windows and I think the Mac has available and simply install one of those three ZFS capable systems although to use Solaris professionally it would have to be licensed whereas the BSD's and ZOL would be entirely free, you don't have to have a 2.5TB VM as you can just set aside say 50GB for the VM's operating system and just give that VM network access to let it create a 2.45TB file somewhere else on the computer, network or NAS, once your running the VM and in there you mount the file, then copy 2.45TB's worth of data from somewhere else to that filesystem (the 2.45 TB file), unmount the file and shutdown and exit the VM.

So to answer your question "Is this feasible on any software?" yes it can and it can even be totally free but the only downside is that suddenly you have to learn one of either unix, Solaris, linux, or FreeBSD but this would be a very good investment for yourself now and in the future especially as data storage sizes are set to explode with 4K and 8K coming in, another benefit is that with ZFS each block in the filesystem if fully checksummed so if it gets sent to the destination and has any errors you'll know about it if you do a Zpool Scrub command on the entire filesystem, more introductory information about ZFS is here and it is far better than Windows NTFS, Linux's Ext4 and the Mac's APFS.

Lastly, a second answer to your question is if you don't want to use ZFS you could just as easily create a 2.45 TB VM file using whatever hypervisor you want containing say an OS of your choosing and as an example together with say a free version of Blender and the Blender files you are working on and then you shut the VM down and then send that actual VM file (together with any relevant config files) and just send all that on the LTO tape, the recipient just has to import the VM into their hypervisor and fire it up. e.g. this article here and although this is just as do-able you miss out on the integrity features that ZFS provides and so isn't my preferred recommendation but this would suffice for say an interim period until you get up to speed with ZFS.

I think the ZFS file and VM image is the only real neat and tidy way to virtualize file systems as these can be any size whereas actual disk images are usually for set sizes (unless you just create a primary partition that doesn't fill the entire disk) and also contain pesky information like MBR and GPT partition table information that you also have to deal with when transferring which makes actual disk images a bit messier to deal with.


Posts IndexRead Thread 

Current Message Thread:

© 2020 All Rights Reserved