Re: Xsan: Best practices for admin and backup

Hey Arturo, and welcome to the fun-filled world of Xsan administration!

This is a REALLY broad question and there are a lot of ways to go, so I'm just going to spit out some good ideas and bad ideas that I know about. Hopefully a few of them are new and/or helpful to you.


Good Ideas:

-Rely on a well-maintained DNS. DNS is one key to a happy Xsan. Use host files if you must, but you'll be happier in the long run with DNS. Regardless, ALWAYS use Static IPs.

-Backup everything in the /Library/Filesystems/Xsan/config/ folder on your metadata controllers on a regular basis. These have come in handy for me. Doesn't hurt to backup the same folder on a client every now and then.

-Target important data on your Xsan for regular, scheduled backups using some good software. I like ChronoSync, but there are a lot of decent software options. I backup Final Cut Pro project files and any kind of "Digital Master" file that can't easily be replaced (like files acquired from tapeless cameras like the EX1 or HVX-200). As for a backup location, I like to use disk devices like the DroboPro with some redundancy built in, and plug them into one of my servers using Firewire 800. Lots of good hardware options out there -- doesn't need to be fast.

-Format any Xsan backup destinations as case-sensitive file systems. Xsan is case-sensitive, and it helps for backups to be.

-Make a clone of your metadata controller's system hard drive from time to time. I make a clone before every server upgrade, just in case it doesn't go well. Carbon Copy Cloner is a great software tool.

-Make sure to keep both a primary and backup metadata controller healthy and on identical software versions.

-Keep all edit suites (clients) on the SAN on the exact same versions of software. Using your backup metadata controller as a Software Update server is one way to help manage software deployment.

-Keep your Xsan volume usage at or below 75%. Xsan, like all filesystems, is prone to fragmentation when it gets full. However, Xsan fragmentation is pretty ugly, and the tools to fix it are imperfect.

-Keep your Xsan volumes mounted on your metadata controllers if you want to modify file permissions or attempt to use Spotlight (for Xsan 2).




Bad Ideas:

-Don't mix metadata with user data in the same storage pool.

-Don't try to mix Intel-based and PPC-based metadata controllers (for a primary and backup pair).

-Don't upgrade a client (edit suite) to a newer version of Xsan or Mac OS X before upgrading your servers. Your servers (metadata controllers) always get upgraded first -- even for very minor (dot) releases.

-Don't try to backup your entire SAN onto LTO/DLT tape unless you really need to. (Some might disagree with me here, and it really does depend on your environment and needs. Nonetheless, backing up a whole Xsan can be very expensive and somewhat complicated.) Find your most important data and back it up, instead. There's very little on our Xsan that can't be lost right now -- it's either backed up on disk (DroboPro or older SCSI devices) or on video tape (original field footage and master tapes).

-Don't try to re-share your Xsan volume over AFP (or any other protocol). Apple generally says it's a bad idea (though there is improved AFP support in Xsan 2), but I just don't like the concept. I think of a SAN as a sports car, and file sharing is like pulling a trailer uphill. Don't ask your GTO to pull a trailer uphill -- it's not what it's meant for. Let the SAN go fast and work out other solutions for accessibility. (Someday, I hope we will be in a situation where one centralized repository will serve both the needs of a SAN and file sharing, but I haven't seen a perfect solution yet.)

-Don't mess with the contents of the /Library/Filesystems/Xsan/config/ directory unless you really know what you're doing.

-Don't change root passwords on your metadata controllers if you don't have to. It's a pain.

-Don't change IP addresses on your metadata controllers if you don't have to. Also possible, but a pain.

-Don't take for granted that expanding your Xsan volume in the future will be easy. It can be done, but adding new storage to an existing volume in a way that makes sense can be a challenge.

-Don't plan on Spotlight working well on your Xsan. I have yet to meet anyone who raves about how well Spotlight works in Xsan 2.

-Don't rely on the cvcp command to copy media files. cvcp doesn't copy extended attributes. Copy media files to/from an Xsan using something else -- I use plain old cp a lot and ChronoSync.



I think two things are the most important: Know a talented person who can get you out of a bind when necessary (Jordan comes to mind), and carefully PLAN anything involving Xsan.

You mentioned cvgather, and it is pretty helpful. Running it regularly wouldn't hurt -- and allows you to back up a lot of stuff in addition to your config folder. In general, though, I don't think you need to need to use the terminal as much with Xsan 2 as you used to. cvgather, cvfsck, snmetadump and cvadmin are all pretty helpful every now and then, and snfsdefrag can be useful(ish). But, I think most of your best backup practices involve common sense backups of important data with a GUI. I use these command line tools a lot before and after system changes and updates, but not every day.

Above all, good luck!

Dave