2 Promise E-Class enougth for 6 HD Streams?
by Berenger Capmany
on
Sep 23, 2009 at 5:53:45 pm
Hello,
We have a quote to build a xSAN network for 5 stations.
We have asked the reseller to have 6 streams of Uncompressed HD.
One for the 4 edit stations and 2 for the Grading station.
Most of the work will be HD and 2K in ProRes.
Some HD Uncompressed and from time to time the Grading Station will do 2K DPX.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Philip Garcia on Sep 23, 2009 at 7:01:15 pm
Hello Berenger Capmany,
I think that 2 of those units will be enough storage for what you are proposing to do. However, have you given any thought to Active Storage for your needs?
Re: 2 Promise E-Class enougth for 6 HD Streams? by Berenger Capmany on Sep 23, 2009 at 8:36:43 pm
Hello Phillip,
Yes the amount of storage is enougth since projects will be something around the 4TB and maximum of 4 at same time...
I was worried for the *real* bandwith... honesly I do not trust the figures used by the vendors. So I wanted to know if these read 1400 MB/s and 6 Streams promised by promise :P are real in real life, not just in demos and brochures.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Matt Geier on Sep 23, 2009 at 7:07:48 pm
Berenger,
It could be enough, (Ill let one of these other experts on the Promise VTRAK chime in and let you know their own take), however, in general it's hard to say without being an expert on all the pieces in your configuration. One thing is for sure though, this quote you have has a very high price tag on it for just 5 clients!
***Disclaimer: I think XSAN is a great Apple Product! I do believe though, for some people, perhaps like yourself, forking over the money for the equipment, and then redundancy, and then meta data hardware you need just to make it all work right in a Fibre Channel environment, is not cost effective. Traditionally Fibre Channel SAN's were the answer, and everyone agreed on it, because it was what the answer was. However, nowadays, Gigabit and 10Gb Ethernet are now a viable, and very reliable infrastructure you can also consider, if you know someone who knows how to make it all work correctly. (it's Ethernet, but you still need to know how to install the network, and the pieces to make it all work right..)
Something to think about ---
For your HD Uncompressed editing to work really well, you could simply use locally attached storage and skip doing this with Fiber Channel all together, simply by setting up a Gigabit or 10Gb Ethernet environment for the editors to work on Central Shared Storage, in Real Time.
On a Gigabit Ethernet wire on a Mac (90MB/sec), you can move 2 streams of Pro Res HQ very nicely at one time (Actually around 60-70MB/sec)....
On a 10Gb Ethernet wire on a Mac (200MB/sec +)
This will even improve in the future with Fibre Channel over Ethernet coming in the next few years....
The idea in a nutshell is this -- if you direct attach the right kind of storage (big bandwidth pipes (MB/s), fast enough (how fast can it talk etc..)to your Apple Server, as long as your editors, don't need to access the same project file at the same time, then you won't need any overlying software to play traffic cop (FCP has file level locking built in).
You can implement this all running AFP (Apple File Protocol) out to the Gigabit or 10Gb clients connected to the network to support video editing in real time off the server.
This all equals = Your FCP editors editing together, sharing storage, in real time.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Berenger Capmany on Sep 23, 2009 at 8:51:00 pm
Hello Matt,
All these thoughts I have already told to them (I'm just a freelance FCP Editor /Shake Composer).
At my home studio I have a Firmtek Box with a Rocked Raid, which is performing over 400MB/s (read) with Hitachi disks.
I have just finished the complete post of a feature working in DPX without a single problem. I also have told them about the Terrablock solution a mixed solution for Ethernet and Fiber connections. But the owners want to go with Promise because is "blessed" by Apple and they (and the reseller) feel confident with it.
But anyway, using ethernet..(plain ethernet) the bandwith is pretty fast for file copying... but for editing? everytime there is any TCP/IP interruption you have a drop frame. Or are you talkinh about implenting a special protocol to "tunnel" video avoiding all the TCP/IP, SMB, AFP messing around.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Matt Geier on Sep 24, 2009 at 3:53:51 am
Berenguer,
Thanks for responding. (sorry to all for the long post... I felt the need to be careful in how I discuss this... it is after all an Apple XSAN forum..)
I'm always intrigued by what people are doing with Apple products! Apple hardware is wonderful as are a lot of the hardware and software solutions that play with Apple!
Your hardware configuration that you have sounds interesting. Personally, I'd like you to push it to the limit to find out how far you can go with it, and give some feedback. - For example, how many Pro Res HQ streams can you edit off it at one time?
You should understand that Ethernet is very capable, YES. If you do enough digging around, you can find Ethernet based networks for video editing which work like a charm!
Some of them are Direct attached SAS Storage to a Server, and some are Fiber Channel Storage at the Server. - The trick in either case, is getting the right pieces working and talking together to make it all happy and polite to the editors! (and making sure it lasts in the long term..wont crumble under pressure, or fail you, etc!...)
Ethernet is actually very fast and can handle enough bandwidth to cover the needs of video editing from that perspective to a certain degree of course. An Ethernet wire at raw bandwidth can handle up to 100MB/sec of throughput, this is quite enough for Pro Res HQ and all the lesser. Apple's O/S allows you to use Jumbo Frames on Ethernet and run that gb wire up to 90MB/sec under optimal conditions! - Real Time Video Editing, however, does not rely solely on the amount of bandwidth, but also how "fast" the solution talks back and forth.
(You can comfortably put 2 Pro Res HQ streams running at (60MB/sec) onto a gigabit Ethernet wire safely (with TCP overhead see).
I will tell you some helpful public knowledge --
The real key to getting a system that works, is by purchasing one designed to meet the requirements, exactly. - Then compare the price/performance and other decisions that go into installing the solution or signing the check for one.
If you do enough searching around here on The Cow and on the web, you'll find some references to some Ethernet based Real Time Shared Storage solutions.
There are some which work, some which do not, and some which do it so well, nobody believes it can be done. (there are those skeptics...)
Now that you've been perked by what I said , I certainly don't want to get into a "vendor war" - NO - I'm simply providing some needed information to you, and the people you do business with, as well as anyone in the future that finds this post.
Here's the info: There's a company called Small Tree that works very closely with Intel and Apple on engineering levels. They have some solutions that you would benefit from in more ways then one.
Speaking of blessings...
You can navigate your browser to Intel's website and find Small Tree. Additionally, if you need a further blessing, place a call to Apple Sales, and ask them to find you a Multi Port Gigabit Ethernet or a 10Gb Ethernet Network Adapter! I'm sure they will oblige too!
I'm certainly not here to steer you from Apple XSAN, or any other vendors business. I'm personally partial to these conversations and try to remain as neutral as possible. This allows you, the person (the consumer) inquiring, to make informed decisions, based on truth and knowledge. - Enough Said.
Go do some more research.. here some helpful links ..
Just for the record, Small Tree Mac OSX Drivers only work with Small Tree Cards purchased from Small Tree.
(Unless a vendor list specific Mac OS support for their Intel based cards, it should be assumed that they do not have proper Mac OS support unless they have stated so, installing a driver from Small Tree will not help unless the card is bought from Small Tree...)
With regard to video editing over Ethernet, hopefully this will send you on the right path! I'm certainly looking forward to hearing back from you myself!
Re: 2 Promise E-Class enougth for 6 HD Streams? by Berenger Capmany on Sep 24, 2009 at 6:22:18 pm
Hey Matt,
Thank to you for your answers.. this lead to the real ethernet editing.. it's definetively a path to follow... I will start today to learn more about!
a friend of mine is also freelancing in a TV production company where they are now thinking to move to FC for 5 macpros.. this will be the way to go, because this way they can add the iMacs. It's a kind of News agency..
Re: 2 Promise E-Class enougth for 6 HD Streams? by Jordan Woods on Sep 24, 2009 at 3:10:06 am
You have to take into account what your facility will need as top end. Is your top end 4 bays of uncompressed HD and one doing 2k dpx files? If this is the case you will drop frames for sure. You have to factor in the possibility that those 4 bays may need access to more than one stream at a time. I usually factor 3 streams per bay on any compression, just in case. (minus 2k and above, since the machine won't handle more than one stream)
With that in mind, you are looking at 4x3x160=1920 and one bay running 2k adds another 300MB/s max.
Now, do you really need to put together a SAN capable of putting out 2GB/s? I can't answer that, but that is a possible reality. When I have tested this, 2 raid heads in active/active mode will not give you more than 500MB/s writes and 900MB/s reads, approximately.
4x160=640MB/s Two boxes will absolutely handle that, one box... doubt it. If you add a stream of 2k on to that... good luck, you'll be at your roof.
You might live completely in prores on your main bays with the occasional 2k in the grading bay. In this scenario two boxes will be fine. I like to take into account room to grow and head room... just in case.
1400MB/s is a number all RAID manufactures put out because it comes from IOmeter testing. This is a very nice easy test that shows total output of the unit under nice 1MB file transfers. This absolutely does not relate to number of video streams. Also to be clear, any published numbers of video streams have to take into account XSAN, so be sure to look for XSAN qualifications, not direct attached since they will not relate to your circumstance.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Berenger Capmany on Sep 24, 2009 at 6:29:16 pm
Hello Jason,
Yes I know it's on the roof.. but, after your advices, enougth.
The facility will be amined to independent/student filmakers. So most of the work will be Prores, both Offline editing of a classic offline/DPX workflow or a RED workflow.
In any case, the 4 edit bays, will do basically cut, and disolves as much. And the Post Suite will be keep separated.
Is not going to happen that all the 4 suites will be editing HD uncompressed and the Post/Grading Suite wil be doing DPX...
as from what I learned from the Promise Website, a JBOD is always possible to add to extend bandwith and capacity.
The number of streams are also refered as what they claim in their web, 6 HD Uncompressed with two boxes under xSAN2.
Re: 2 Promise E-Class enougth for 6 HD Streams? by Dave Klee on Sep 24, 2009 at 10:17:11 pm
Hey Berenger, I just wanted to chime in and agree with a lot of the stuff Jordan said. Even for your low-stress edit suites doing ProRes work, I'd budget 3-streams of video per workstation as a minimum.
And, Jordan's dead on about the "theoretical direct attached" versus "real world Xsan" performance. 500MB/s writes and 900MB/s reads is attainable with for a good RAID in an Xsan. 1400MB/s is not.
However, as an Xsan administrator at a small corporate video shop, I just want to encourage you, your resellers and anyone else you deal with on this purchase to consider Active XRAIDs instead of Promise.
When Apple discontinued their old RAIDs and "blessed" Promise as the new Xsan solution, I assumed it was the only way to go -- and for a time it was the only option. Thankfully, though, Apple has wised up and allowed other RAIDs to be a part of Xsan, and some very good options are out there.
We've had Promise RAIDs here in our old Xsan, and I helped out a little with a larger (120TB) Promise RAID (Xsan) deployment down the street. Bad drives and controllers were not uncommon out of the box. Documentation was poor. Administration was difficult. And, performance was not what I hoped for.
We switched our smaller (20TB) Xsan over to Active XRAIDs this summer, and life has been much better. Everything worked -- nothing DOA. Management software is very easy to use. Setup and configuration was a breeze -- no complicated scripts to find and load, and all settings were very straight forward. Performance has been great. We're running eight edit suites -- five are pretty heavily used -- all working in DVCPROHD or ProResHQ. It all works fine with just two Active XRAID boxes.
Now, as an editor, it might not really matter to you that the storage is easier to administer and configure. I can't say Promise equipment would fail quicker than Active -- in fact, once everything got running, the Promise gear has been fairly stable. I can't even say categorically that Active is really faster -- I haven't done a fair head-to-head comparison. (I can say that my new Active-based Xsan is more than twice as fast as my old Promise-based one.)
But, I believe Active storage is better. Easier to use means less down time in the long run. More flexibility means more long term value. Higher quality components make a more reliable Xsan.
I don't really know you and your situation, so I can't say for sure what is best. But, I know that my life has been better since moving from Promise to Active storage, and I won't likely switch back. There are some solid vendors out there who sell Active and know Xsan -- if it was me, I'd find one of them and save some headaches.
Feel free to get in touch if you have any questions or want to chat. I don't work for anyone who sells things, so don't really have a dog in the fight. Just want to make sure people know there are some good Xsan storage options out there. Regardless of what you decide, good luck with the purchase!
