Video SAN for IT department
I'm looking for documents over the web that explain the difference between the need of a video SAN and a IT SAN.
I'd like to be able to explain to IT guy in a technical way what's the video need for storage.
Example of information I'm searching:
File Sharing on file or volume based
Working with Avid there is certain need for a SAN to work in a collaborative environment
I'm pretty sure there is already some documentation about that on the web.
Video SAN for AVID -
AVID ISIS 5000 -
Facilis Terrablock -
A Video SAN has higher performance than a 1Gig network. It is very difficult for an IT guy to comprehend the enormous bandwidth that we require to go to each workstation. Even on a simple 1GbE video san network, we need a full 90MB/sec, to comply with the bandwidth requirement of video streams that at a minimum require between 20 - 40 MB/sec per stream. When you are dealing with accounting data or database info, it is hard to imagine the demand for this type of minimum bandwidth (Fibre and 10gig offer more bandwidth than this per client). It is hard for these IT guys to think that we will actually FILL UP 96 TB of storage in just one year. "How is that possible, what on earth are you guys doing". I laugh when I see consumer reviews of single 2TB drives, and some of the comments are "what are you doing to fill up a 2 terabyte drive".
For AVID, the MetaData server for project manangement is another defining fact, that is hard for a generic IT guy to understand (why does AVID even do that, he might ask).
It's good to sit and explain the details of how you use the storage -- give them "a day in the life" explaining the different stages in the lifecycle of the data (ingest, edit, rendering). Be sure to include typical volumes of data you're talking about, access patterns, etc. This should help educate them some and allow to start asking you relevant questions.
First, get the terminology down. The IT SAN guys are going to be very particular about this.
- SAN: This is the Fibre Channel network connecting the server/workstation and the storage array. SAN very specifically means block storage over FC.
- If you want file based access (AFP, CIFS, NFS) over IP/ethernet, you're talking about a NAS (Network Attached Storage).
- In the IT world, there are huge differences when you start talking SAN and NAS. If you use the wrong term, there will be a lot of confusion and a lot harder to get to the right solution.
Some more specific differences:
- IO Patterns: video work is dominated by extremely large sequential IO (reading those huge video files off disk into your editor) where as typical IT SAN usage is more often dominated by a huge number of very small random IOs.
+ Array cache is very important in the random IO situation (you can keep a lot of important blocks in cache and rarely have to hit disk in many cases), but somewhat less relevant in a video SAN (cache will never hold what you need, so you're always hitting disk).
+ With the huge sequential IOs of video work, parity RAID (RAID 5 and 6) allow you to throw the most spindles at IO for a given number of disks. Since IO speed is going to be dictated by the disk speed and number of spindles.
+ SATA disks are fine. They come very close to FC disks for most sequential IO and give you a lot more volume per disk.
- File System (assuming FC block based SAN): Assuming you have multiple edit stations sharing the same files (the typical situation), you'll need a clustered file system. A shared/clustered file system (many threads here about some of the different ones) that will allow multiple workstations access the same volumes (LUNs). Typically in IT SANs only a single machine can access a given volume (LUN). The array, SAN and hosts have to be configured specifically to allow the sharing and run a shared/clustered file system to allow this to all work.
- Multi-pathing: In IT SANs, you almost always (can't think of a case where you wouldn't) have multiple FC connections from a host to the storage array and then run a multi-pathing driver on the host to allow both failover between paths and aggregation of those paths. In a typical configuration, a host (server) connect to the SAN would have at least 2 FC HBAs in it. These would connect to different switches that comprise the SAN and then to the storage array. This gives you multiple paths between the host and the array. If one path dies, your IO goes down the other path. With some arrays and multi-path drivers, you can aggregate the paths such that if you had 2 4Gbps FC HBAs, you could use them both at the same time and get 8 Gbps of IO. I don't know how common this is in the video editing SAN world, so I'll defer to Bob Zelin or others on that.
Hope that helps some.
The main difference between the two is the "real-time" requirement of video apps.
IT infrastructures tend to focus on redundancy. Then the system is loaded heavily, performance will be slower, but it still "working"
Video servers need to push video streams in realtime. The drives matter, the raid controller matters, how the network is setup matters. If you don't have it all correct, it won't work.
Some key elements:
Drives and raids can't retry too much. If there are tons of retries, the drive needs replacing
Networks have to be lossless. You need flow control so TCP isn't dropping packets (this is often incompatible with larger edge/core network configurations)
Machines need to be tuned for optimal performance. Your desktop/laptop isn't set up to push or pull data over a 10Gb port. They are setup for GUI response (so the mouse is smooth) and having lots of applications open. To really drive a 10Gb port in either direction, there's some kernel tuning required.
I'm not sure this is documented anywhere, but we sure focus on it! :)
CTO, Small Tree Communications