FORUMS: list search recent posts

Fast, reliable & cheap (!!) SAN

COW Forums : SAN - Storage Area Networks

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Chris Maytag
Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:11:48 pm

Hello,

We're beginning the process of specifying a fiber-based (or that's our assumption, based on bandwidth needs) storage system for a small workgroup (< 12 clients). Our capacity needs are moderate in enterprise terms (say 100TB), but speed requirements are steep: we work with 4k & 8k "dome master" frames (which are square 4096x4096 and 8192x8192 formats), sometimes at 60 FPS.

We know we're not going to get all the performance (or bells & whistles) we want with what little money we have (which isn't much, we're a non-profit based at a public university).

That being said, because we're willing to do the grunt work ourselves, we figure we'll save as compared to a large, dedicated enterprise-class commercial system such as we'd get from the major integrators.

My query here isn't asking for specific product solutions (unless you care to offer one), but rather a request for "must-read" material relevant to understanding the choices we're facing.

Any comments appreciated.

Chris Maytag
Fiske Planetarium
University of Colorado, Boulder


Return to posts index

Matt Geier
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:38:48 pm

Hi Chris,

This is a little abnormal from my usual method of approach to engaging on the forums.

I saw your post and knew I wanted to talk to you and learn more about your needs so I can help address them accordingly.

I'm interested in having you work with me here at Small Tree.

Feel free to visit Small Tree's Creative Cow Forum as well as our Facebook and Twitter feeds to review more content on this type of subject. http://forums.creativecow.net/smalltree

I encourage your Interests further.

Regards,


Matt Geier

Direct and Channel Sales Accounts Manager
United States & Canada, Western Regions
Small Tree Communications
866-782-4622


Return to posts index

Bob Zelin
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:39:27 pm

we work with 4k & 8k "dome master" frames (which are square 4096x4096 and 8192x8192 formats), sometimes at 60 FPS.


you want "cheap" at these frame rates. You are out of your mind.
Where do you come up with this crap.

What is your compression ratio that you want to play back with ?
The "rolls royce" of shared storage systems is Facilis -
http://www.facilis.com/products.html

look at the spec on the 24D -
you can do ONE stream of 4K dpx 24p. ONE STREAM - not multiple users in true SAN enviornment. Now, look at Apace, Studio Network Solutions, Cal Digit, Small Tree, Maxxx Digital Final Share, Editshare, and even AVID ISIS 5000 - all the good brands - you can't do it. You can't have what you want. And you CERTAINLY can't have it cheap.

You are "non profit" playing out at 8192x8192 ?
- you are the reason I never give to charity.

Bob Zelin



Return to posts index


Bob Zelin
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:43:31 pm

we work with 4k & 8k "dome master" frames (which are square 4096x4096 and 8192x8192 formats), sometimes at 60 FPS

Matt -
I am sorry to engage you in a negative light. Small Tree is a great company. But Small Tree can't do this and NO ONE can do this. He will be lucky to get a single stream from two 16 bay drive arrays stripped together to get "8192x8192" format at 60fps. This is pushing the envelope of what we can do today - and it CERTAINLY will not happen in a shared storage enviornment, where MULTIPLE users can have access to this. Exactly what the hell are you observing this on - multiple projectors ? because there ain't no LCD or LCD screen that can play this resolution ? This is all a bunch of nonsense for this "non profit" organization.

Bob Zelin



Return to posts index

Alex Gerulaitis
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:43:38 pm

I can't quite explain why but I think this Bob Zelin's post should be in Cow's Hall of Fame.

(Bob: did he say "multiple streams" or "realtime"? I think he didn't which means he just wants it "as fast as possible" - and in that case, he might get away with COTS components including those from Small Tree.)


Return to posts index

Chris Maytag
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:46:16 pm

Bob,

A couple things:

First, you'll note that I said "we work with" those frame sizes, not that I expected to play back streams in realtime. Read more closely.

Second, your response:

> you are the reason I never give to charity.

is precisely the sort of response that gives this forum, and, well, people in general, a bad name. It's rude, it's unnecessary, and accomplishes nothing constructive.

Thanks for nothing, Bob, and have a nice day.

Chris

---
Chris Maytag
Fiske Planetarium, University of Colorado at Boulder
Sci. Viz, Video & Motion Graphics
Cinema 4d / Maya / Adobe / FCS


Return to posts index


Chris Maytag
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:58:39 pm

Bob,

I thank you for your attempt to "help". But given your misreading of my post, and with comments like:

"you want "cheap" at these frame rates. You are out of your mind.
Where do you come up with this crap."

and:

"you are the reason I never give to charity."

I think I'll be speaking to other folks, not you.

Best,
chris


Return to posts index

Alex Gerulaitis
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 10:33:25 pm

Chris,

It's unfortunate private messages don't work here on Cow - what I want to say isn't necessarily for public consumption: Bob is known and is basically an industry celebrity for two things. One, he is a fantastic specialist and integrator, fiercely loyal to his clients and business partners yet never thinking twice before telling it like it is - or like he thinks it is. Which brings forward one other thing: he sometimes says rather dramatic things before he carefully reads the post.

There is history to that, too: it's a recurring trend on this forum with people asking for enterprise grade SANs on the cheap or with DIY COTS components, and Bob has taken upon himself to set them straight - an arduous yet necessary task.

Bottom line, please don't take offense at Bob's words.

Alex Gerulaitis
Systems Engineer
DV411 - Los Angeles, CA


Return to posts index

Bob Zelin
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:48:45 pm

I am looking at Dome Master specs for 8K, which is used for Planetariums, to be used with Projectors. Sorry buddy, but you can't have shared storage for this file format - and not for 12 users. I know nothing about this format (it's a projection format) so unless you are actually working with files that get up-resed, and are LOWER RES files that you actually create your shows with, you can't have what you want. Period.

I am looking at the Evans and Sutherland specs.

Bob Zelin



Return to posts index


Bob Zelin
Re: Fast, reliable & cheap (!!) SAN
on Nov 9, 2012 at 9:52:20 pm

is this what you have at your university ?

http://www.es.com/Products/Production_Systems.html

if you are using an Adobe After Effects based system, and just want a SAN for AE CS6, you can use any of the shared storage systems on the market. They all work. Small Tree, Maxx Digital, Facilis, EditShare, etc. This list of working systems appears on this forum countless times, and in all the banner ads.

Bob Zelin



Return to posts index

Jim Davison
Re: Fast, reliable & cheap (!!) SAN
on Nov 10, 2012 at 10:45:01 am

Hi
We have the technology to make the fastest storage Server available.
We Generaly do not use SANs to the work stations but you were not spacific.
If you are doing most of the work then you can save some $.
But the question you ask is a good one but it makes a differance as to what you are using as clients.
PCs, MACs Linux or what.

AS for the storage we work with a company that has a great reputation for speed and price.
But I need answers to the above and more.
if you want to talk about it call me
Jim 310-770-0099 Anytime as we are available to our customers 24 x 7.
Jim


Return to posts index

Bob Zelin
Re: Fast, reliable & cheap (!!) SAN
on Nov 10, 2012 at 4:14:00 pm

Hi Jim,
I did not realize that Helios shared network systems could support a multiple user 4K workflow.

from the Helios website -
"Be it file server, print server, Web-based file server, or prepress solution "

Since the site doesn't mention hi bandwidth shared video solutions, I am confused as to the solution you have in mind.

You know, Accusys has an Infiniband solution (which is resold as Cal Digit Super Share) -
the new performance of the dual controller looks very impressive -

fasthttp://www.accusys.com.tw/ExaRAID/ExaRAID/Performance.html

faster than I can do with 10gig ethernet -

and I know that Facilis is now starting to offer 16Gig Fibre channel connection - but I don't have the specs. But their current 8gig fibre solution using the ATTO FC 8gig cards won't do a multi user 4K solution.

If Helios has a fast solution that can offer multi user 4K workflow, I would sure love to hear about it -
this is the Helios website -
http://www.helios.de/start.html
and this is Jims Company Packet color that rep's Helios in Reno, Nevada -
http://www.packetcolor.com/Packet/clients.html

looking foward to learning more about Helios from you Jim !

Bob Zelin



Return to posts index


Bob Zelin
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 10, 2012 at 4:25:08 pm

http://www.es.com/products/Production_Systems.html

I would bet a nickel that this is what Chris is using.
According to the Evans and Sutherland website, this is all done
with Adobe After Effects, with Pre-renders, and "virtual projection".

these are the projectors that Evans and Sutherland recommends.
http://www.es.com/products/Configurations.html

from this little bit of web searching, it appears that you only need a simple Adobe After Effects system. Doing 12 clients is really nothing, and the usual list that I specify will work fine for you.
To have 100TB of usable storage after the RAID grouping, will require 3 48TB arrays, or two 64 TB arrays (which can be done today with the new 4TB SATA drives on the market). I strongly suggest RAID 6.

Shared storage solutions that will work with Adobe After Effects for Dome Master include (this is just a sample list - )
Facilis Terrablock
AVID ISIS 5000
EditShare
Small Tree Granite Stor
Cal Digit Super Share (ExaSAN Accusys is the same thing)
Studio Network Solutions
Maxx Digital Final Share
Apace Systems
and of course systems based on software solutions from
Tiger Technology MetaSAN, Command Fibre Jet, and good old XSAN.
Rorke Data is a great integrator for Fibre Jet, if you are so interested.

But I really want to hear about what Helios can do Jim !!!

Bob Zelin



Return to posts index

John Veron
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 12, 2012 at 5:23:04 am

Hi,

I don't post often but I am going to propose something completely different here.

Please contact someone in your area who sells FusionIO cards.
Ideally not an IT vendor (as they have no clue what we guys do).

FusionIO cards are similar to SSDs (well, not really). They sit in the PCIe slot as storage.
You can bundle multiple together to provide even more performance using a FusionIO software called Sphere.

Now, ONE card can easily give you 4K uncompressed! I benchmarked 1.4 GBytes/s .
They are expensive though and can only hold 420 GB (the ioFX card).

While they appear expensive you will save A LOT of money when you try to get that performance through a disk array.

Ask the Fusion Guys for advise on how to serve out the cards over FibreChannel to other computers as you would with normal storage.

You could use SANmp or FibreJet (SANmp is cheaper) for sharing.
Good luck!


Oh, and curious to hear what the others say!


Return to posts index

Chris Maytag
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 12, 2012 at 3:44:15 pm

Thanks for your reply, John.

We're familiar with FusionIO, and they have a place in our workflow on one or two machines. But we're not looking for streaming (as others have inferred incorrectly) or frame buffering. So, to be clear: we need the fastest possible shared production storage for our money. It's clear that DAS and NAS solutions aren't acceptable for various reasons, hence my original post here.

My take is that a fiber switch coupled with SAS arrays & fiber HBAs on the workstations is the way to go; this is why I orignally posted.

---
Chris Maytag
Fiske Planetarium, University of Colorado at Boulder
Sci. Viz, Video & Motion Graphics
Cinema 4d / Maya / Adobe / FCS


Return to posts index


John Veron
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 12, 2012 at 11:37:20 pm

I see, well I personally think that not a single NAS solution will be able to give you the performance.
You need block-level access to the disks, not file level.

A NAS "translates" the computer request for a file to a location in its file system on its storage.
There is quite some overhead which makes it slow. On all of them.

Editshare, ISIS 5000 and others won't cut it. Too expensive to get this massive performance.
Better to look at FC based storage such as Rorke, StorNext, EVO, etc.

But even then it won't be cheap and you need to throw a lot of disks at it.


Return to posts index

Alex Gerulaitis
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 13, 2012 at 1:02:28 am

[Chris Maytag] "It's clear that ... NAS solutions aren't acceptable for various reasons, hence my original post here."

Just because NAS is slower (700MB/s per port is the fastest NAS I heard of), or for other reasons?

Alex Gerulaitis
Systems Engineer
DV411 - Los Angeles, CA


Return to posts index

Chris Maytag
Re: Fast, reliable & cheap (!!) SAN (dome master)
on Nov 13, 2012 at 3:20:06 am

> Just because NAS is slower (700MB/s per port is the fastest NAS I heard of), or for other reasons?

It's an assumption based on speed considerations. While I don't know that we couldn't make do with slower, file-level storage in a pinch, we think this it's the better way to go.

---
Chris Maytag
Fiske Planetarium, University of Colorado at Boulder
Sci. Viz, Video & Motion Graphics
Cinema 4d / Maya / Adobe / FCS


Return to posts index

Walter Soyka
Re: Fast, reliable & cheap (!!) SAN
on Nov 13, 2012 at 5:51:50 pm

I'm not a shared storage expert like the other posters here, but I do specialize in large format animation and design (often at >4K resolutions). I'd love to hear a little more about your workflow, and from that, I may be able to help the others estimate your bandwidth requirements.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Eric Hansen
Re: Fast, reliable & cheap (!!) SAN
on Nov 26, 2012 at 8:44:17 pm

Hey Chris

on what computers do you actually need to playback in realtime? is it only on the computer that plays the actual projection? if so, i would suggest a modest shared storage system, and super fast direct attached storage on the playback system. this is the setup i see at post houses that have an online/theater/master suite. when you're dealing with massive frames like that, but only need them in one place (at the end of the post production process), then just beef up that one place.

FusionIO is a good idea, especially if After Effects is doing the actual playback. you can buffer the entire timeline to the card (assuming it all fits). Or, get an ATTO SAS card, attach a bunch of Enterprise SSDs to it in a RAID-0, and go to town. the latter is the setup I saw being tested at Skywalker Ranch last year in a theater that needed to pump out 4k DPX in 3D. based on what the techs told me at the time, i would only attempt this with the help of the ATTO technicians directly.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]