How Does Partitioning Impact RAID Arrays?
I'd like to partition my new RAID array (Pegasus2 R8 32 TB).
I'm thinking I'd like to segregate video media from everything else. (audio, photos, etc)
My reasoning is the make backups easy. Video will get archived as necessary to LTO. While the rest (on the other partition) will get backed up much more frequently (cloned, Time Machine, etc), since these files will be changing more frequently. (with periodic LTO backups)
What are the issues/concerns to partitioning RAID arrays?
I wouldn't do it unless you have a use case that benefits from it. You can point LTO backup software at specific folders, excluding others, the same as you can for Time Machine. Also, if you don't properly predict their relative consumption requirements, you'd going to run out of space on one before the other and regret the layout. This isn't a problem with folders. So I don't see an advantage to partitioning based on what you've said so far.
However, since the inner part of the platters have lower performance than the outter part, you could choose to short stroke the array. Normally this means using only up to the first third or so of each disk, "wasting" the last 2/3rd. This is often still cheaper than getting 10K or 15K RPM disks, which are smaller anyway, and not short stroking them. In your case, maybe you'd go 50/50, or even 70/30, which is perhaps a faux short stroke. Instead of taking the best performing part of the drives, you're eliminating the worst performing part of the drive. Also, you wouldn't be discarding the worst performing part, you'd use it. If you actively use both partitions, then you've lost the performance benefit, and possibly have made it worse because of the additional seeks that will happen from one end of the platter to the other more often than it otherwise would, at least until the array is 2/3+ full.
Anyway, to do this in Disk Utility, click on the top most icon for the array device so that the UI shows a Partition tab. Change the scheme pop-up from Current to 2 Partitions. The first partition will have the best performance because that will use the outer portion of the drive. And have the 2nd partition take up the rest. You'll put video files on the 1st partition's volume. And the things that tolerate lower performance on the 2nd volume. Whether you do this depends on your workload. If you really have distinctly separate workloads, this two partition layout will help overall performance. If you combine them, it'll make them worse. How much worse just depends on what the usage pattern is, and hence whether the disks will be seeking to death or not.
More relevant for performance though, is the chunk size for the array. I don't know what the default is for Promise arrays but for this use case I'd say 512KB or larger is reasonable. If we're talking about a bunch of emails, mp3s, and JPEGs, well that's a different story. But you might just accept the write/rewrite performance hit you get with those. Chances are that's a one time thing, and from then on they just get read.
Another thought I had is that I'd like to use the RAID array for Time Machine backup, without partitioning.
It's my understanding that I can limit the size of a Time Machine backup by using sparse image bundles. (set the size limit to something like 1.5 x what I'm backing up)
Do you have any experience with this setup?
Any concerns having sparse image bundles residing on a array?
Yes, sparse bundles can be used for Time Machine backups - this is how Time Machine backs up over a network. There are guides on the internet on how to create them with hdiutil. The problem is that once configured, backupd automatically makes an authenticated connection to the remote host, and also mounts the disk image file, does the backup, then unmounts the disk image. All automatically. You won't get this behavior locally, you'll have to manually mount (double clicking) the disk image and probably always just leave it mounted. So now you have two mounted file systems, one on top of the other, and if you have a crash or panic during the backup there's a higher chance of one of the file systems being corrupted. And Time Machine backups, I find, are already really sensitive to file system problems which is why I don't consider them worthy of being an archive. It's just a matter of time before a Time Machine backup starts to exhibit problems, and before it exhibits problems backing up, it's not assured it won't have a problem with a restore.
Time Machine is the backup/restore strategy I wish worked. I use it because it's brain dead simple, and if the boot drive dies and the restore works, I'm up and running much faster than alternatives. But I simply don't trust it to be the only backup, and I don't trust it at all to be an archive.
What you're best off using Time Machine for is making a "snapshot" of your boot drive, OS, applications, user settings, so that if the boot drive dies and is replaced, you can restore and get up and running again quickly. I'd suggest turning off Time Machine in between OS/Application updates because on a production machine, not much is changing so what are you really backing up? Not much, yet backupd consumes a lot of system resources when it runs each hour which I notice degrading system performance in the most basic use case, not even doing live video production work. So I'd think that Time Machine going off every hour would be a show stopper for video people.
For your important files, those don't even belong on a boot drive. And I wouldn't have Time Machine backing them up. I'd use something like a scheduled rsync script, which Carbon Copy Cloner can do for you. Set it to backup once a day outside production hours, maybe over lunch, or maybe shortly after the end of normal working hours. Or possibly with a logout script.