Re: XSAN over 100 TB

Hey Robert, I've worked a little with a university who has a 120TB Xsan 2, divided into two separate volumes for two different purposes (live playback of video footage on-air for television, along with a general video production SAN for editing). It's all built on the previous model of Promise RAID (750GB SATA drives).

Getting it running was a pain -- both from a Fibre Channel management and RAID hardware management perspective. There were quite a few issues with the older Promise stuff (controller and drive failures out of the box), which were all eventually resolved. Since then, the system has been fairly stable. There were also some challenges in Fibre Channel switch management for so many devices and ports.

It also sounds a little touchy -- it's hard to know the right mix of RAID firmware, switch firmware, and software versions for the OS and Xsan to make everything happy. (True for Xsan in general, but sounds like it is amplified with a big system like this.) A RAID firmware update needed to be rolled back after it didn't work in the environment. Propagating updates across all RAIDs was a challenge, partly due to Promise's web-based management tool that left something to be desired. (I hear they have been updated, but the core management tools are still command line and web based.)

Both volumes get really good performance (I believe one is 90TB, the other 30TB). Once running, they're very stable. The smaller volume has a real-time mirror with another Xsan volume in another building (because it needs immediate fail-over redundancy for live playback on-air). The larger Xsan volume has more limited backup.

Jordan brings up a GREAT point about backups -- especially dealing with tapeless files. Also, making changes (firmware and software updates) to a large, mission critical Xsan that is being used in production can be very challenging. Best to have solid redundancy plans that fit your needs -- whatever they are.

Also, considering easier to manage RAID hardware might save you some headaches. I'm not sure how well Active Storage scales to the 100TB+ level, but at the 30TB level, management is really a piece of cake. It helps me sleep better.