Hey Chris, I hate to be a downer, but the most "ideal" setup for two Xserve RAIDs is probably with some sort of pulley system that you could rig up for weight training in a make-shift gym.
I say that because the Xserve RAIDs with 500GB drives are probably from around 2005 or 2006 -- which makes them a really scary thing to be relying on in an Xsan environment as we near 2012. Very difficult to find spare drives and parts and people who will come help you if you have a major issue. Not to mention that drive technology has gotten a lot better since then.
But, if you're just looking to get some experience and these things aren't going to be used in production for important data, there are certainly some setup possibilities for you.
You've hit on one of the classic Xsan build dilemmas: what to do with those two metadata drives that the system requires to work well. I've seen people setup two RAIDs exactly like yours are setup, and there's nothing obviously "wrong" with it -- maybe other than using the "Other" storage pool as heavily as you are. And, ideally you want to keep your video storage pools below 80% capacity so they run faster.
Other people set both RAIDs up like your first box (mixture of RAID 1 and RAID 5) so that they end up with four RAID 5 LUNS of the same size (five disks each), and put those four LUNs into a single storage pool in Xsan. You lose capacity that way, but striping together four RAID 5 LUNs into a single pool might improve performance, and give you a single "Video" storage pool which is easier to manage. Those extra RAID 1 (or some people make them RAID 0) LUNs on each box can be used for another very low-performing storage pool for lightly used data (maybe administrative files or audio in some setups).
The downside of this from a performance standpoint is that a 5-disk RAID-5 won't be quite as fast as a 7-disk RAID-5. But, you'll probably make up for it because your striping across four equal LUNs in a single storage pool in Xsan rather than two, unequal storage pools of two LUns each.
Another way (and one I used for a few years) is to basically give up one half of one RAID for that metadata LUN, and use the other three halves for data. You can also throw "hot spares" in the mix since these things are old -- basically, if you don't include a drive in a RAID set, it becomes available as an extra rebuild drive in case there's a failure. So, you would end up with something like this:
RAID1 left side: 2-disk RAID-1 for metadata, 4-disk RAID-0 or RAID-1 for useless files, 1 hot spare
RAID1 right side: 6-disk RAID-5 for video, 1 hot spare
RAID 2 left side: 6-disk RAID-5 for video, 1 hot spare
RAID 2 right side: 6-disk RAID-5 for video, 1 hot spare
Many people (including Apple) argue that you want an even number of LUNs for Xsan to stripe across, though, and you could certainly see decreased performance with this setup because of only three video LUNs (as opposed to four). But it worked adequately for me back when I only had two RAIDs and needed something small yet reliable. And, if you built it without the extra 4-disk LUN on the first RAID, you could just use those leftover disks as spares for when your RAIDs start to throw up bad drives. They will.
Anyway, the fact that you have to use two of your RAID drives for metadata means there is no "perfect" way to setup a system like yours. This is why Active Storage came out with a product called Innerpool a little while back.
http://activestorage.com/innerpool.php It allows you to put the metadata drives inside the server and use your RAIDs only for what they were meant for (data).
If you can convince people, though, you should really consider some new hardware. Any adjustment to your current setup will get you very minimal performance improvements (if any), and it will continue to be a ticking time bomb. New hardware will give you a huge leap forward and help you sleep better at night.
Good luck with everything, and please post back with any questions.
Dave
