FORUMS: list search recent posts

this is a GREAT article on Thunderbolt shared networks

COW Forums : SAN - Storage Area Networks

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Bob Zelin
this is a GREAT article on Thunderbolt shared networks
on Oct 31, 2013 at 2:10:48 am

Hi -
read this -
http://arstechnica.com/apple/2013/10/os-x-10-9-brings-fast-but-choppy-thund...

which says in it's title -
"I don't need no stinking 10 Gigabit Ethernet, I have a Thunderbolt cable!"

and if you read the whole thing, you can read the most important part of the article -
"Assuming stability and consistency will be improved, is Thunderbolt networking useful in the first place?

Obviously, outfitting Macs with Thunderbolt 10 Gigabit Ethernet adapters and hooking them up to a 10GE switch is the superior solution. With an equally superior price tag. In situations where a fast network is only needed to copy files between two computers, using a $30 or $40 Thunderbolt cable is much more convenient at a fraction of the price. I can also imagine a Mac Pro being outfitted with a 10GE adapter, and then one or two other Macs connecting to the 10GE network through a Thunderbolt connection to that Mac Pro. And if Thunderbolt networking catches on, we may even see Thunderbolt ports on NAS devices."


So, for you Apple fanboys out there, you ain't doing a shared storage system that actually works with a $30 thunderbolt cable.
So just stop it.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Oct 31, 2013 at 8:03:32 pm

Ok Bob, so that is the first test on a new feature that has been available for ONE WEEK..not on two MAC Pros but one Macbook Pro and one old Mac Air with a 10Gb TB port. So you stop it - you have NO IDEA if this is going to be potential solution or not with Mac Pro's and 20Gb TB ports and software revisions we all know will be coming.

Nobobdy can say hooking up shared storage from one Mac Pro to two or three other Mac Pro's via 20GB TB will work or not until SOMEBODY ACTUALLY TRIES IT.

I'm sympathetic to your situation, a large part of your market may disappear - but that's your problem. Because if it works (and I'll definitely be pushing for it to work) you can bet your ass that is exactly what everybody will be doing. I still haven't heard **anybody** give me a technical reason why doing shared storage over 20Gb TB wires won't work as well as ethernet wires.

The pertinent question Eric posted in the other thread is this "The question is can Thunderbolt in network packet protocol mode function with the latency it has with a 6 device limitation for realtime processes on chain versus a hardware 10Gbe Ethernet switch and also can the controller handle the load a 10Gbe network card can at that latency required for realtime processes. That is what he is talking about and what I questions currently."

I guess we will find out when the Pro's come out - that test on the link you posted is hardly the final word on this. If something like a TB switch comes out with latency management features it could still be something that is far less expensive than 10GB switches and 10GB NICs on every system.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Andrew Richards
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 6:36:05 pm

[Greg Leuenberger] "Because if it works (and I'll definitely be pushing for it to work) you can bet your ass that is exactly what everybody will be doing. I still haven't heard **anybody** give me a technical reason why doing shared storage over 20Gb TB wires won't work as well as ethernet wires."

Everybody? The new Mac Pro has 6 TB2 ports across 3 TB2 controllers. The best you could ever do, assuming everything else works perfectly, is 5 IP-over-TB Mavericks clients of that Mac Pro as a file server. And even then only if it runs headless, which is a huge waste of those twin GPUs that make up so much of the cost of the Mac Pro. Is that enough clients for everybody? Maybe many small shops, but even if this idea does work, it is going to be strictly very limited in its ability to scale. Oh, and everyone has to be really close to the server since the longest available Thunderbolt cable right now is only 10 meters. And it costs $330 per cable, so your 5 seat "cheap" workgroup costs $1,650 just for the cables.

I think you're also seriously underestimating the work that will be necessary getting such a setup to actually work, not to mention the cost of the equipment necessary to test prior to using it in anger. The back of my napkin has it at least over $5,500 before you even price the RAID. Would you spend that kind of money one something just to test it to see if it works?

[Greg Leuenberger] "If something like a TB switch comes out with latency management features it could still be something that is far less expensive than 10GB switches and 10GB NICs on every system."

A TB switch would essentially be a PCIe switch on the inside, a technology that is already on the market. Since I can't find a published price for one, I'm betting they cost something more like a house than a car. Basically, don't hold your breath on a Thunderbolt switch ever coming to market. The tech is just too expensive and the potential market just too small (and notoriously thrifty).

We'd be much better off if someone would just put the guts of one of these into a big brother to one of these. Or better yet, if Apple would put a modern NIC on board in the new Mac Pro.

Best,
Andy


Return to posts index


David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 7:19:20 pm

[Greg Leuenberger] "Nobobdy can say hooking up shared storage from one Mac Pro to two or three other Mac Pro's via 20GB TB will work or not until SOMEBODY ACTUALLY TRIES IT."

Greg, just because the new Mac Pros have not been released yet, does NOT make it impossible for experienced professionals to determine how its various components will perform. It's not rocket science, but it is science nonetheless.

The technology involved is not so radically new that it completely changes ALL bandwidth requirements and ALL protocols necessary for streaming video to multiple connected clients without packet loss. And, while the new Mavericks OS does have some very impressive improvements and performance enhancements per its networking protocols, there's no magic involved that will suddenly enable Thunderbolt connectivity to do everything right out of the box that you're hoping for.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 6:37:10 pm

[Bob Zelin] "So, for you Apple fanboys out there, you ain't doing a shared storage system that actually works with a $30 thunderbolt cable."

Thanks for reiterating the message Bob.

As you and I both know, high-performance does come at a price, albeit incredibly reasonable nowadays. If editors can't afford proper storage sub-systems at current low prices, they just might be in the wrong business.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 7:01:14 pm

@Andrew - not "everybody" (obviously) I thought I was pretty clear in my posts I'm talking about small shops with 2 or 3 stations. Regrading a TB switch - I'm guessing it would look something like a Caldigit Supershare (maybe I'm wrong) - not exactly 'house' expensive. Cables are expensive - but still less than 10Gb Nics, right? Right. The killer is the switch...except for that Netgear one which isn't exactly getting rave reviews a 10GB switch is $10K and up.

Is this a feasible setup? Maybe, I'm just asking the question - and it's a reasonable question to ask. My hardware/software budgets are 20-30K per year. I spend most of that on 3D workstations and software...which I alternate every other year. So I'm not keen on spending 10K on a switch and another 3-5K on Nics. I wired my facility in Sunnyvale with CAT6 3 years ago thinking 10Gb Switches would come down in price, they've hardly budge for what.....5 years now?

I agree with you that it would be nice to have 10Gb ethernet in the new Mac Pro..but we don't we have TB...hence my inquiry.

Bob and David - when you say things to a fellow professional like "fanboy" or "if you can't afford it you're in the wrong business' you sound like a couple of clowns...especially when you have a vested interested in a set up like this NOT working. TB shared storage on the new Mac Pro's is a reasonable inquiry. Odds are it won't be usable - but it's a reasonable question that doesn't deserve derision from people who are obviously worried it's going to impact their business. Just fyi - cheap, fast, networked storage - or cloud storage - that doesn't require a reseller is coming, in fact it's pretty much here. Deal with it.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index


David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 7:45:26 pm

[Greg Leuenberger] " you have a vested interested in a set up like this NOT working."

Greg,

I'm sorry if a dose of shared storage reality may have killed some of your enthusiasm for the new Mac Pro, but please, don't start false rumors. We very much have a vested interest in making certain that all of the systems we sell are reliable, and work as expected or better, because we service and support EVERY system we sell, whether it's on the Mac, Windows, or Linux platform.

We at ProMAX were the very first company to show FCP at NAB as a professional tool, and we have probably sold more Apple workstations for editing purposes than any company on the planet, save Apple themselves. And, we'll probably sell a zillion of the new Mac Pros too.

Again, please let me reiterate, shared storage is science, and with science come certain realities. I'm sorry if setting proper expectations here is objectionable, and I sincerely apologize to you for quoting Bob's instance of the term "fanboy," which I never use personally.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 1, 2013 at 7:54:53 pm

No problem David - I have a pet peeve with fanboy, it's meant to be insulting and dismissive and it shouldn't be tolerated.

I'm not adverse to this not working - I really don't expect it to. I also don't expect a test between an old Mac Air and a Mac Book to be the final answer on the subject. If 10Gb gets some pressure (not sure from where?) and comes down in price then this isn't even a relevant question. As long as 10GB switches are 10K and up it is relevant and small companies like mine are (not always!) going to stick with their Pegasuses and PCI-e SSDs.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 2, 2013 at 7:42:24 pm

Listen Greg -
ProMax, Studio Network Solutions, Small Tree, and Maxx Digital all sell shared storage systems that work perfectly for a small shop for under $10,000. Some are $7000 - $8000 . Is this really TOO EXPENSIVE ? Well, if you have only 40 bucks, I guess it is too expensive.

In these days, where you don't need any VTR's, and cameras are so inexpensive, and you can do everything on a Mac internally, I did not realize that $7000 was so insanely expensive that it's a "deal breaker".

OF course, if you are student, trying to produce a film, and you have zero money, you want your friends to all get together and help, and having a couple of 40 dollar cables would allow several people to try to make their masterpiece film. But most of these starving independent film makers are going to film school - whose parents are spending $40,000 to $60,000 a year for them to go to school - so I can't feel sorry for those families, and say "yes, you deserve a free shared storage system for under 200 bucks".

When you look at products like the Promise Pegasus, you realize that this type of technology is CHEAPER THAN EVER BEFORE IN HISTORY. But I realize (as I realized years ago in linear editing), that to some people, NOTHING IS EVERY CHEAP ENOUGH.

Of course, all of this is moot - because Apple's real goal has nothing to do with thunderbolt shared storage, XSAN, or any kind of local shared storage. Apple's dream is that everyone shares on iCloud, and that is their concern, not getting a 40 dollar cable to work. They too, don't want to kill their billion dollar + investment.

(ps - can't we get those Thunderbolt cables from Monoprice - 40 dollars is SO EXPENSIVE, and I want them for $2.95!!!).

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index


Andrew Richards
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 4, 2013 at 1:08:40 am

[Greg Leuenberger] "I'm guessing it would look something like a Caldigit Supershare (maybe I'm wrong) - not exactly 'house' expensive."

If that CalDigit box really is doing PCIe switching (those look a lot like Infiniband connectors), then from my understanding of Thunderbolt, a switch could be doable. It would probably take Intel developing the ASIC though. That would only happen if there were an enterprise application for Thunderbolt.

[Greg Leuenberger] "Cables are expensive - but still less than 10Gb Nics, right? Right."

The most expensive NICs, maybe. The newer 10GBaseT NICs are regularly selling for mid $3XX range prices and work happily with your existing Cat-6 infrastructure.

[Greg Leuenberger] "The killer is the switch...except for that Netgear one which isn't exactly getting rave reviews a 10GB switch is $10K and up."

The switch is always the killer, be it managed Ethernet (even good gigabit managed switches still cost into the thousands), Fibre Channel, Infiniband, or a theocratical Thunderbolt switch. Then again, for a shop of less than 5 seats, you don't need a switch—just a server with several slots for NICs to connect clients to directly.

[Greg Leuenberger] "I wired my facility in Sunnyvale with CAT6 3 years ago thinking 10Gb Switches would come down in price, they've hardly budge for what.....5 years now?"

This is what happens when enterprise tech does not find its way into the consumer space. Prices stay high.

Best,
Andy


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 4, 2013 at 8:01:28 pm

Hi Greg -
1) Cal Digit SuperShare is Accusys Exasan, which is 100% Infiniband, that requires some software to run - like XSAN, MetaSAN, or CommandSoft Fibre Jet, or SANmp. Cal Digit buys all this stuff from Accusys. Accusys has not done very well in the US (but it is an excellent product), and they have closed their US offices, and are working directly out of Taiwan.

2) I am unaware of any single port 10Gig NIC that costs over $1000.
Small Tree, ATTO, Myricom, Chelsio, Solarflare, LSI Logic are all under $1000 per NIC.

3) who said the Netgear 10G switch is not getting RAVE reviews. This is pretty much all I install these days. The performance is fantastic, and they are cheap. The big model - the M7100-24x is a serious switch, that can support 24 10G users. Most people buy the smaller models. The 12 port XS712T is dirt cheap, and fantastic.

4) your old school Sunnyvale facility might have been done with expensive switches from Arista Networks, Interface Masters, or Fujitsu. This is 2013, and the Netgear 10G switches work perfectly, especially for a small workgroup of 10G users (and it's all backwards compatible to 1GbE for those clients that don't have a 10G NIC).

with that said, no matter how cheap is cheap to one person, the next person will say "40 dollars for a cable ! - I can't afford that". Nothing is ever cheap enough.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 8, 2013 at 9:26:55 pm

So, couple of things. First of which any information on Prosafe switches is far more helpful than 'forget about it fanboys' or 'if you can't afford it you suck'. I'm running a mixed PC/OSX studio..some of the Macs are iMacs (pegasus+ultrastudio on thunderbolt) and I plan on getting the new Pro's minus some unforeseen suckiness. So yes, 1-port 10GB copper NICs go from about 350-750 (so less than $1K) but if I want the iMacs (and future Mac Pro's) online then that's another 500 or so more for the TB expansion box. So maybe not quite $1K - but not exactly built in like TB. I upgrade PC workstations every 2 years and TB2 is a fraction of that cost on a MOBO. All the switches I've seen (Arista, Small Tree) are 12-15K and have been for YEARS now. SAS storage is also more expensive than TB storage. So yes, cost is an issue. $10K can go to replacing the Gigabit network I have or provide 2 new 3D workstations (PC or Mac Pro)...so that's why I've been waiting around (like a lot of people) for 10GB switches to come down. Why is that such a hard issue to empathize with?

Regarding NetGear, that's encouraging to hear. I have SCOURED the internet looking for any decent review of the Prosafe line and other than a few 'meh...the fan's so loud I sent it back' and a lot of 'scoff - it's Netgear' by IT pros..I haven't seen anything. So if you're installing them, and they're 'great'..then I may as well get one since they're less than $2K.

So Bob, if you want to put up a helpful review of the Prosafe...I'm sure people would very much appreciate it.

When people see Apple promoting fast data transfers over TB2 of course they get excited...because if it works that's $7-10K that can be invested someplace else. Seems pretty straightforward to me. So, I'm going to hold off and see - if TB2 shared storage is a Red Herring (which seems to be the consensus here at least) then it's looking like Netgear and SAS storage into one of my Xserves and some external TB boxes with 10GB nics in them (bleach)

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index


Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 8, 2013 at 9:57:13 pm

[Greg Leuenberger] "SAS storage is also more expensive than TB storage."

How so, Greg? The few TB storage choices (Promise, CineRAID, etc.) - are roughly in the same ballpark with similar SAS solutions. Once you add drives, the difference is negligible if any. Once you look at expandability, features, speed - SAS wins by a large margin. TB wins only by not having to install an HBA or a RAID controller, i.e. when working in the field and/or on laptops, needing something that's portable between multiple systems, etc.

An 8-bay JBOD SAS tower can be had for under $600 (sans HBA or controller), a decent LSI-based 8-bay SAS expander - for under $1K.


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 8, 2013 at 10:27:46 pm

That's $1,600 (plus a slot) with 0 drives...BYO. An off the shelf plug and play 12TB Pegasus is $2K....so yeah, it's cheaper..I have a 27" client monitor plugged into the back of the Pegasus...another bonus..it's also portable between any TB Mac (which is pretty much any Mac).

Maxx Digital 8-Bay 8-TB (less storage) MiniSAS is $5K and I don't think that even includes the HBA...add another $500 at least. That's 3X as expensive with LESS storage. I'm not adverse to building things myself, but you can see the appeal of TB...especially TB2.

-G

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 8, 2013 at 10:44:49 pm

[Greg Leuenberger] "That's $1,600 (plus a slot) with 0 drives...build it yourself"

I don't understand where you're getting these numbers. A 16TB SAS box starts at $1.4K - yes, plus a slot. SAS RAID controllers start at $600, HBAs - less than that.

And no, you don't have to "build it yourself" outside of popping an HBA or a RAID controller into the tower.

And if the attraction of TB for shared storage is the ability to plug a monitor into the back of it... I digress.

TB isn't some magic bullet - its hardware and connectivity costs money. Perhaps in some configurations it's a little less than SAS - but nothing extravagant. It's also a consumer tech not fit or designed for shared storage applications.


Return to posts index


Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 12:33:13 am

We seem to be mis-understanding each other -- you said

"An 8-bay JBOD SAS tower can be had for under $600 (sans HBA or controller), a decent LSI-based 8-bay SAS expander - for under $1K."

You can get an 8-Bay SAS tower WITH drives for $600? I assumed you meant empty tower and $1K SAS HBA (from ATTO, or LSI or Areca...one of the good ones). I see you meant expander enclosure...So I thought you meant $1.6K add your own drives. If you can get a 16TB SAS box with HBA for ~$2K please send me a link because that's *nowhere* close to what I've seen from any vendor around here.

"And if the attraction of TB for shared storage is the ability to plug a monitor into the back of it... I digress."

TB isn't some magic bullet - its hardware and connectivity costs money. Perhaps in some configurations it's a little less than SAS - but nothing extravagant. It's also a consumer tech not fit or designed for shared storage applications."


I know exactly what TB is and it's working just wonderfully - my client's projects (and I have big clients) are all running off TB or Accelsior local storage...it works just fine. TB is not 'consumer tech'...any more than my GForce Titans running Maya and NUKE are 'consumer tech'. Did I say hooking up a monitor was the deciding factor or something? I don't see that I did. Hooking up displays to TB is a big plus - it's nice to have a 27" client display with a thin little cable off your rig...it's nice having a portable little box on the desk with 3 cables and not cards inside my computer..my BM and AJA cards are on a shelf.

My Gigabit switch and Xserve-Raid5 can't do anything more than ProRes (which I'm starting to hate with it's gamma shift issues)...I want to use uncompressed 8-bit YUV as my main codec. I can't do that over Gigabit hence the TB and Accelsior local storage....been waiting for cheaper 10GB switches and cheaper NICs (and cheaper SAS storage).

It's looking like SAS plus Netgear switches may be a lot more cost effective than what I've been looking at (like the Maxx Digital box noted above)...3rd party SAS cards hooked into 3rd party storage running across a 3rd party switch into 3rd part NICs sitting in 3rd party TB expansion boxes isn't exactly my idea of elegant - hence the (looking more like a wish) that TB2 shared storage will work.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 12:57:46 am

[Greg Leuenberger] "I know exactly what TB is and it's working just wonderfully - my *clients* are Adobe and Avid and Cisco"

That self-professed knowledge and high profile clients change everything! /Furrowing brows/ What took you so long to mention that you know what Thunderbolt is?

Don't know what explains your missing the link in my post, to a $1.4K 16TB SAS box, and why you choose to ignore various other points.

Good luck with your quest to advance your knowledge. :)


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 2:44:49 am

You do know that link is for a 4TB box with no HBA? Right? The equivalent TB Pegasus with NO HBA needed is $1,100 - and that's from the Apple site. Plus yes...you get the other benefits of TB. So yeah...I don't think we're speaking the same language here. What's your point? Here's a smaller, slower, less expandable and more expensive SAS box? Go buy it?

I mention my clients to give some legitimacy to what I'm saying - I do that when some guy tries to tell me an edit system being used for Adobe's latest Photoshop launch videos is a 'consumer' product. Feel free to do the same.

I'm sorry I came here - this forum is just......yikes. Serious conflicts of interest going on here. For the two or three intelligent comments on this topic - thanks. I'm outta here - have fun people.

-Greg

[Alex Gerulaitis] "Don't know what explains your missing the link in my post, to a $1.4K 16TB SAS box, and why you choose to ignore various other points."

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index


Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 3:58:26 am

[Greg Leuenberger] "You do know that link is for a 4TB box with no HBA?"

My bad! Creeped into a 16TB search, I goofed. That said: a 12-bay SAS JBOD - $800 (no drives, no HBA), 6x 2TB WD SE Enterprise class drives - $930 ($155 ea.). LSI SAS HBA - $350. That's $2,080 for the same capacity, expandable to 12 drives. 8-bay SAS JBOD towers start at below $400. Need RAID? About $400 more.

Alternately, Areca 4036 is $200 more, with 24Gbs pipeline (vs. 12Gbs on the other one). You probably know what SAS expansion means, and what it brings to the table in terms for storage applications.

Finally, here is to hoping you don't plan to have 3-5 editors all connecting to a 6-bay Pegasus box as their main shared storage. Once we start looking at 8-, 12- and 16-bay boxes, feel free to prove TB is still cheaper.

We were talking about shared storage, weren't we? You still think SAS is more expensive?


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 5:00:10 am

Hi Alex - I think we're arguing about small things here. My initial thought (hope, prayer...whatever) was to have a single bus - Thunderbolt 2 - for storage and sharing. Probably (at least for now) a huge stretch. I do think TB is cheaper than SAS...yes, but it's not that significant. The cost of the switch and the required NICs (and required expansion chassis if you have iMacs or intend on getting new Mac Pro's) is the issue I was harping on. Anyway, you seem like you're trying to be helpful so there's really nothing to argue about. I've looked into SAS fairly extensively because I'm thinking about upgrading my current setup (Apple Raid card in an XServe) and I'd rather just streamline everything with TB (the storage is portable...fewer points of failure...supports more peripherals...ect...I really don't need to cheerlead it...it's good tech).

10GB is basically a hack...10GB cards sitting in a Thunderbolt external chassis hooked up to an expensive switch..why? why would anybody want that if you could go thunderbolt only? Unfortunately for everybody this hack of a setup is probably all we can do.

This forum really isn't the place to ask these questions - it's basically a shop front being capitalized on by a few small time vendors. Oh well, I'll go elsewhere - it's too bad this forum can't support fellow professionals by sharing knowledge without it becoming a sales pitch and a clown show.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

EricBowen
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 13, 2013 at 12:17:26 am

I see allot of very experienced editors and technology industry people in the Cow forums who give very valuable info including How to's and problem/workflow resolution information every day. I understand there is allot of excitement about TB2 and the new Mac Pro's. For many, both seem to answer allot of requirements they have needed for some time while remaining in the Mac Domain. I and others are simply pointing out that this excitement can turn into problems when purchased. These problems can turn into no end of failure to work with applications or workflows required and these over all are not inexpensive investments. I am simply pointing out that Apple has some really strong marketing behind TB and there are pitfalls here with regards to technological limitations and the current editing applications/workflows. Whether that is enough to give some one pause to consider this further or not is their choice and not mine. However I will present the info or even the questions I have about a new technology whether good or bad.

Eric-ADK
Tech Manager
support@adkvideoediting.com


Return to posts index


EricBowen
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 7, 2013 at 6:07:07 pm

You have to be careful with reviews. There are allot of variables here such as network adapter configuration settings that impact latency and performance. There is also the plug n play mentality that clients expect to just plug it in and it works. That does not mean it works at peak or ideal performance. The Intel 10Gbe adapters alone have profile settings in the advance settings that set the adapters advance settings to specific workflows including low latency. If these are not set then some workflows will be negatively impacted. I have yet to have a client come back and say the Netgear switches do not work as advertised and as you state there is nothing else in that price range right now. These are a small business solution and for what they do, there is nothing better right now.

Eric-ADK
Tech Manager
support@adkvideoediting.com


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 12:24:48 am

Greg -
at the risk of offending you (hey, I love offending people), I risk making this statement, only because the esteemed Eric Bowen has replied.

From my observation, you would RATHER GO OUT OF BUSINESS than to hire Alex Gerulatis, Eric Bowen, or myself to make all of this work for you. You want to do it yourself, because you know "just as much as all these other big shot a##holes", and so you make these posts. People like Alex, Eric, and myself make our living installing hi end systems like these, and we are not buying $15,000 Arista Network switches to make them work. So it's your decision to sit there and suffer, instead of making a living producing video's, which is how you should generate your income. Not sitting there and figuring out how to do all the techno mumbo jumbo yourself.

As for the "noisy switch" - I read that review on Newegg as well. What the hell does that mean. Do you thing that because a 10G switch has a hi power fan in it, which belongs in a MACHINE ROOM, along with its noisy SERVER CHASSIS, that I think that "oh this switch is crap, because the fan is too noisy" ? Professional equipment has ALWAYS been noisy, that's why they have temperature controlled machine rooms - and for the poor companies that can't afford a temperature controlled machine room, THATS WHY THEY INVENTED CLOSETS.

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 12:51:39 am

(meanness deleted) I pointed out the newegg review as being INADEQUATE and asked you to do a review since you seem to have experience with these switches (seriously, wtf is wrong with you?)

I'm not going out of business pal, I've been in business for almost 15 years and have made millions. My clients are companies like Adobe, Cisco, Oracle, Symantec, Dassault...the idea that I'm going to go out of business because some clown on the internet thinks I should hire him instead of continuing to do things myself is....I don't even know what it is.

I come here now and again to get some advice and thoughts on storage...what I get is a bunch of people trying to sell things. You're a forum leader and you're the source of almost ALL hostility I see on several COW forums. What the hell is wrong with you? It brings the professionalism of COW down several notches...go over to the Foundry or Autodesk forums and you never see crap like this. Look at your first post...are you 15?

Ask yourself this - why am I doing this stuff myself? Because *I* don't feel you have enough value...it's not that hard..I don't have to pay some guy to do it. I was under the impression that you could come to this forum and ask questions about SAN so you COULD BUILD IT YOURSELF. Instead you get assaulted by internet ass clowns who try to sell you stuff and are completely unhelpful and condescending. I don't think you're a big-shot a-hole...just the a-hole part. Installing network storage ISN'T ALL THAT HARD, DUDE. I've been building PCs and installing NAS's and Xserve-based storage in a mixed OS environment since 1998. I'm a master at 3D applications that make installing network storage look like pre-school. Seriously....get over yourself.

This isn't a forum for you or other vendors to SELL YOUR SERVICES. It's to discuss storage solutions. Take out a freaking advertisement. People should be able to come here and ask DIY questions. Clown.

FFS.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 4:55:25 am

For the record, I don't sell anything. I am not a dealer.
Since April 2013, I use netgear 10 g switches, because they offer incredible performance at rock bottom prices.
I have totally loser clients like amazon, Disney and electronic arts, because they prefer to hire a loser clown like me, than someone really qualified. I am just a lucky 15 year old.

As I study new products like Fcp x, and 10g, and mavericks, and tbolt2, I feel I deserve the right to make my smart a$$ observations, based on my experiences. And even when I am wrong, like I was recently about NFS, I correct my errors publically.

Just like my condescending comments about the red camera, and Fcp x, unlike most people, I will continue to research them, and find solutions to make them work for my clients. And with subjects like
thunderbolt networking, and I will ultimately figure out how to get this to work in real world environments, and post them on forums like creative cow, as I have done for years.

In the mean time, 10 g Ethernet is the most cost effective was to network a bunch of macs. Not gen 1 thunderbolt..

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Greg Leuenberger
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 9, 2013 at 5:31:50 am

I've done plenty of game openers for EA and been in their edit suites - and yes, they are a loser company : )

All joking aside Bob - your online manner sucks. What's more, you know it sucks. You wouldn't speak to me like that in person (because I'd knock your teeth in) but you can hide behind the internet and act like an ass to everybody. You have no business being a 'forum leader' around here and surely you know that.

This forum sucks and it sucks because of you....and yes, you have attempted to turn this into your personal store front. It's BAD FORM to try and sell your services on these forums. Have you been on the internet before? You never see this crap on more respectable forums...you get run out.

You want to compare experience? Do you really? My experience with production almost certainly dwarfs yours. You know, ACTUAL production experience....3D expert...Compositing expert....edit systems like FCP and Premier are things you're expected to know in passing. Not plugging in network cables. Putting together networked storage is easy. Putting together PCs is easy. If I were to head over to a PC building site and ask what kind of RAM to put in a certain MOBO I'd get a straight up response - not the smart-a$$ crap you write up here.

Why do production companies go out of business? Because they spend money on sh#t they don't need..like HD decks that never pay off..and obscenely over-priced network storage (the HD deck of the current era). Like a team of people to install things any reasonably intelligent person could do over the weekend with some research.

[Bob Zelin] "As I study new products like Fcp x, and 10g, and mavericks, and tbolt2, I feel I deserve the right to make my smart a$$ observations, based on my experiences. And even when I am wrong, like I was recently about NFS, I correct my errors publically. "

No, you don't. Let me repeat that just so I know that it's sinks into your head. No...researching new products doesn't allow you to act like an a$$ to people. I'm an expert in 3D animation (you know, the hard stuff) - I provide HELPFUL NON-SMART-A$$ assistance to people all the time on the Foundry forums, on CGtalk and other places. That's why I demo at Siggraph for The Foundry...for Dassault..for Adobe. Because I have a good name in the community and am respected. Who in the world would want you to represent them? If you want to be respected act respectable...you do act like a 15 year old, it's embarrassing.

Again, READ YOUR FIRST POST on this thread. Do you think this forum can be called professional with a stupid post like that from it's "leader". Seriously? Now read the last sentence of your last post....was that so hard?

Grow up dude.

-Greg

Greg Leuenberger
CEO
Sabertooth Productions, Inc.
http://www.sabpro.com


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 10, 2013 at 4:23:04 pm

Hi Greg -
so let's talk about Thunderbolt network distribution. From what I can tell, at the moment, there really are no Thunderbolt hubs or switches. So we will have to wait for the new Mac Pro, which will be $3000, or $4000. This will become our "server", and provide us with 6 Thunderbolt ports. Now, one of these ports has to go to our large shared storage volume (Promise, Areca, Netstor, other yet to be released products). So that leaves us 5 computers that we can share over Thunderbolt. Which makes this potentially a comparable solution to an 8 port Netgear XS708E switch, which you can buy from Newegg for $922. But of course, you need 10G NICS for all those damn computers, and Thunderbolt is free on iMac's and Mac Mini's. Of course, I assume that you don't have any Mac Pro's at your company, because as you know, you can't get a 12 Core Mac Pro with an NVidia GTX-680 to work with that Thunderbolt stuff. So I assume that you have all iMac's and Mac Mini's currently.

So, you have your new $2999 or $3999 new cylinder Mac Pro, with your new TBolt 2 RAID array from Promise, Areca, Netstor, etc. You now need LONG Thunderbolt cables, because I am assuming that your staff is not all sitting next to each other on the same desk. Sumitomo
http://global-sei.com/ewp/E/thunderbolt/
has 20 and 30 meter optical Thunderbolt cables, and so does Corning, but I am having great trouble finding them. I wonder how much they are going to cost. I wonder if they come with a Plenum jacket, when we run them thru your ceiling tiles, so they meet firecode.

So now we have a maximum of 5 client Mac computers, all running Cinema 4D and Adobe After Effects at your facility, with our new Mac Pro, and our long optical Thunderbolt cables. I sure hope that Thunderbolt 2 interfaces can allow for at least 1500 MB/sec on the 16 bay Netstor drive array (I certainly have not tested this), because this is what is currently available from countless RAID manufacturers that you see here on Creative Cow, and we all know that you need this kind of bandwidth for a shared environment.

With so many facilities, such as yours, that use existing Mac Pro's with NVidia cards, it seems that getting five $600 dollar 10G nic cards, and that $924 8 port Netgear switch (it's so damn noisy !), is a pretty cost effective solution, considering I can use generic Cat6 cable for up to 55 meters for this, as opposed to who knows how expensive for those new optical Thunderbolt cables from Corning and
Sumitomo.

And yes, I know, if you have a bunch of Mac Mini's, we can certainly keep them all on the same desk with the new Mac Pro tower, all connected with 1 meter copper Thunderbolt cables, and use Gefen or Smart AVI DVI/USB extenders to get back to our client desks, where their monitors and keyboards are (of course, we wil need Cat6 cables to accomplish this, for the extenders).

So Greg, when you need a render farm setup for your Cinema 4D, or Adobe After Effects workstations for your facility, and if you need other things, like large storage solutions, LTO archive solutions, Asset Management from companies like CatDV or Axle Video - or even some video tie in stuff for those dreadfully expensive HD Decks, that some of your clients might demand that you rent, and you need them tied in with some hardware from AJA, Blackmagic or Matrox - well, you feel free to call me any time, and I will be more than happy to assist you with all of this, and your growing company.

And as soon as I am lucky enough to get my hands on a new cylinder Mac Pro (hopefully next month), I will be sure to tell you my test results with Thunderbolt networking, and the performance over a new Mac Pro.

Your picture on LinkedIn looks real cute, by the way !

Talk to you soon -
Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Andrew Richards
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 11, 2013 at 5:18:54 pm

FYI, the price for the optical Thunderbolt cables is $330 for the 10m Corning and about $850 for the 30m Sumimoto (I got that by converting from Yen since I could only find them for sale on Amazon's Japanese storefront).

Best,
Andy


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 11, 2013 at 4:27:34 pm

[Greg Leuenberger] "Why do production companies go out of business? Because they spend money on sh#t they don't need..like HD decks that never pay off..and obscenely over-priced network storage (the HD deck of the current era). Like a team of people to install things any reasonably intelligent person could do over the weekend with some research."

The definition of "over-priced" is as they say, "all relative," and I can assure you, the most over-priced gear anyone can buy is hardware that doesn't adequately do the job for which it was purchased.

And, buying gear from a manufacturer or reseller, and paying to have it configured, deployed, and supported by experts, can be the very best investment of capital there is. There are many thousands of casualties in this industry who fall by the wayside every year because they fail to recognize this.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

marcus lyall
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 18, 2013 at 10:48:21 am

Quite a thread.

The frustration of this SAN forum is that you've got people with very little technical knowledge and those with expert knowledge. And a lot of vendors. It doesn't feel very 'hobbyist' which is the charm of the Cow. In the other forums, someone who has figured out something on a short film can help someone who is working on a Hollywood film.
There isn't quite the critical mass here to make it work like the others.
SAN forum doesn't feel very peer-to-peer.

Bob at least showed people how to build their own SAN. Which was really useful for me.
Because I couldn't afford a turnkey system. I get why he's cranky. SAN stuff is the backbone of your business if you do post. Bad things happen when it goes wrong.

But there's surely room on this forum for those people geeky enough to want to try out unorthodox networking shizzle and report back to other people on how it worked.
Isn't that the point of this forum?
It's great that we get warned by people that it won't work or that we'll lose all our data.
But sometimes, in between jobs, it's fun to just give things a go and see what happens.

I am not a network engineer but I manged to build a SAN from scratch, which is still working 4 years later.

Just to restate that for those people who are on a budget, Ebay is a very viable option for 10gb equipment. We've bought all our 10gb from ebay. All of it has worked.

There are very few moving parts in networking equipment.
It is sold at knock-down prices by people stripping out data centres.
My 20 port switch cost less than the EdgeCore switch that does the 1gb ethernet.

You can probably get 4 or 5 machines linked to a 10gb switch for under $4k total if you search hard enough.



Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 19, 2013 at 7:03:44 pm

Hi Marcus -
I have had this type of fight with other people like Greg, in the past. I am active on many "professional" forums, and like to express my opinion. When I talk to people in person, many people feel that I am "yelling" at them. I am just passionate about this stuff (like I might be passionate about talking about cars, guitars, women, and alcohol !). And I raise my voice.

Greg does not like this type of behavior, and would rather "knock my teeth down my throat".

At CCW in NY City, I saw the Corning Thunderbolt 2 cables - both 10 meter and 30 meter. I saw the speed tests with Thunderbolt 2, showing between 700 - 800 MB/sec. And I am fully aware that a new Mac Pro (cylinder) with it's 6 Thunderbolt ports, may make a wonderful server, if you purchase these Corning Optical Thunderbolt 2
cables. These cables cost the following -
10 meter cable - $295
30 meter cable - $599

So if our friend Greg is reading this, once again, YOU CANT DO THIS FOR $30 DOLLARS ! (this seemed to piss him off when I said this, and I kind of like that). The idea of using Thunderbolt as a replacement for a small workgroup instead of gigabit, 10G, or Fibre Channel is a wonderful idea, and in may, in fact work wonderfully.
Obviously, there are no Thunderbolt 2 ports or cards for existing Mac Pro's. But a new young company that wants to put together a small workgroup, with a new Mac Pro, will have wonderful opportunities.

So when new products that are a challenge to our industry - like RED, FCP-X, and Thunderbolt networking - I pay very close attention to all of this, research it, and report it, on forums like Creative Cow. I feel passionate about this. If some people don't like that - well, it's a good thing those people are not in the same bar with me, while I am expressing my opinions on different types of lager !

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Sergei Yakovlev
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 26, 2013 at 6:36:19 pm

Bob,

The Ars Technica article is flawed. Mr. van Beijnum measured disk performance instead of networking performance. See this thread for accurate measurements (Thunderbolt 1): http://forums.macrumors.com/showthread.php?t=1657957 . Thunderbolt 2 may be even faster—it would be nice if somebody measured it using two new MacBook Pros.

So, the breakdown is that IPoTB is really fast, but its current implementation does all routing in software, which brings performance down if more than two computers are involved. I believe it is actually possible to hardware-accelerate IPoTB by using native PCIe packet switching, but we don’t know if Apple will implement that.


Return to posts index

Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 26, 2013 at 8:05:25 pm
Last Edited By Alex Gerulaitis on Nov 26, 2013 at 8:20:16 pm

Do these 10-second-long iPerf transfer tests really negate AT review? Just curious.

On a side note: reminds of the excitement of using FireWire for "high speed" networking back in 1997. Unlike TB, FireWire supports tree-like topology, isochronous transfers, can connect up to 63 devices... Future held promises of 3.2 and 6.4Gbs speeds with full backward compatibility. Cheap cables! FAAAAAAST!!!

Cross-OS support: even MS jumped on the bandwagon.

Didn't pan out: security (same DMA vulnerabilities as TB) and other issues, and of course, Ethernet was already there, and wasn't proprietary.

If, say, IpoTB absolutely kills IP over 10GbE on costs, deployment ease and speeds in some scenarios - isn't it still a huge uphill battle to get it supported on multiple OSs and hardware platforms?

Until then, isn't IPoTB just a cute little niche thingie, much like what TB is today?


Return to posts index

Sergei Yakovlev
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 26, 2013 at 11:54:53 pm

Alex,

The AT review is incorrect by itself. Iljitsch has measured the speed of MBA’s internal drive, which is slower than TB1 speed. Of course, the iperf tests do negate the review, but you could see that it was wrong even without the tests. You can read the comment section for additional information. I am surprised that this review has not been retracted. The mistake is quite ironic, as Iljitsch “contributes articles about network protocols as well as Apple topics”. I mean, of all AT contributors, he should have known better.

I agree with you that the whole TB story reminds a lot of the FW story. However, I think that this time it might pan out differently. Mostly because TB is backed by Intel. But there are other differences.

TB2 (20Gb/s) is faster and cheaper than 10GbE (2-port TB2 controller costs just $10), FW800 (0.8Gb/s) was slower and cost about the same as 1GbE. TB’s max cable length is 100m (10m cables available now), FW’s was 4.5m. Finally, and this is the killer, 1GbE was built into every single Apple’s computer, while 10GbE is built into none. Since you’d have a better chance of finding an Ethernet cable lying around, you’d be using 1GbE to transfer files. Now, to use 10GbE networking you’ll have to use 2 TB cables, 2 external 10GbE TB interfaces, and a Cat 6 cable. Which is quite more complex and much more expensive than using a single TB cable.

Cross-platform support and dedicated switches will only happen if Apple publishes IPoTB as a standard. Being Apple, they might well prefer to keep IPoTB to themselves as a “competitive advantage”; in that case, it will certainly remain a niche technology for direct connections or very small workgroups.

P.S. Oh, and FW did find use as a networking technology in F-22/F-35 :-)


Return to posts index

Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 3:10:15 am

Sergei,

Appreciate the response. You're probably right about the AT review. I am too lazy to re-read it, but remember the impression of a quick-n-dirty test that wasn't to be taken too seriously.

I do have a few petty issues with some of your points though. :)

[Sergei Yakovlev] "Mostly because TB is backed by Intel."

Not just "backed" but "developed", and then given up for exclusive use to Apple. Didn't make sense to me then, still doesn't now. Awesome tech with a fantastic potential usurped by a single company hell-bent on squeezing every penny out of it - and now several years later there is a big "FAILURE" stamp all over it: pathetic adoption.

OTOH FW was developed and "backed" by far more companies than TB was/ is, notably Sony and TI besides others. Marrying it to DV and putting it as an interface of choice on first digital camcorders quickly drove up its adoption - which I don't see happening with TB. Years after its introduction, its adoption is still close to nil.

[Sergei Yakovlev] "TB2 (20Gb/s) is faster and cheaper than 10GbE (2-port TB2 controller costs just $10)"

Perhaps it's cheaper between two MBPs. Or even in a five-seat workgroup with TB cables running wild - perhaps. Anything beyond that, with wiring, switching and routing included - still cheaper? How about scalability, does it look as good as with Ethernet?

Don't get me wrong, Ethernet over TB is way cool for quick-n-dirty shoestring shared storage setups. Maybe even fast and maybe even reliable. We won't know for a while. It just doesn't seem to have much future.


Return to posts index

Sergei Yakovlev
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 7:43:06 pm

[Alex Gerulaitis] "Not just "backed" but "developed", and then given up for exclusive use to Apple."

Sorry, I’ve used the wrong word. What I meant to say was since Intel has developed TB (née Light Peak), they must be interested in it succeeding in the market. The exclusive use was just for one year. I think Apple’s involvement actually furthered TB adoption rather than impeded it. The slow adoption is due to Intel’s tight control of TB licensing—they have even forced some TB products off the market.

[Alex Gerulaitis] "Years after its introduction, its adoption is still close to nil."

Well, the adoption could be better, but it’s not all doom and gloom. Take a look at these products:

http://www.blackmagicdesign.com/products/blackmagicproductioncamera4k
http://www.blackmagicdesign.com/products/ultrastudiothunderbolt (TB2)
http://www.avid.com/US/products/Pro-Tools-HD-Native
http://apogeedigital.com/products/symphony-64-thunderbridge.php
http://www.promise.com/promotion_page/promotion_page.aspx?region=en-global&... (TB2)
http://www.promise.com/promotion_page/promotion_page.aspx?region=en-global&... (TB2)

It looks like the professional market is supporting TB alright.

The problem is, the consumers don’t need TB that much, as USB3 will do for most situations. So the GPU card makers and motherboard vendors have been reluctant to add TB. Intel must have realized that this can eventually lead to TB dying, so now they are pushing TB add-ons with their “Thunderbolt Ready” program:

http://blogs.intel.com/technology/2013/11/intel-announces-thunderbolt-ready...

Hopefully, the GPU card makers start to integrate TB directly on their cards, or at least provide internal DP connectors to eliminate unsightly loopback cables.

[Alex Gerulaitis] "Perhaps it's cheaper between two MBPs. Or even in a five-seat workgroup with TB cables running wild - perhaps. Anything beyond that, with wiring, switching and routing included - still cheaper? How about scalability, does it look as good as with Ethernet?"

I agree that right now IPoTB is only useful for simple scenarios.

Technically, it is possible to build larger networks by connecting multiple Mac Pros together, with each Mac Pro fanning out to iMacs and MacBook Pros (the six-device limitation only applies to a single TB chain). And it would be cheaper than 10GbE: $330 10m (or even $40 2m) TB cable per host vs. $30 0.5m TB cable + $1000 10GbE TB interface per host, plus the cost of 10GbE switch.

However: 1) the performance would be really bad, as the current IPoTB implementation is software-based (the performance degrades with each additional routing hop—you can see that in the tests I’ve linked to above); 2) the total network throughput would be limited by a single 20Gb/s connection.

The first problem can be solved if Apple hardware-accelerates IPoTB by using native PCIe packet switching. I believe this is possible on current hardware. The second problem can only be solved if Apple publishes the IPoTB standard so that other companies can produce an IPoTB switch with full bandwidth.


Return to posts index

Alex Gerulaitis
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 3:57:47 am

[Sergei Yakovlev] "FW800 (0.8Gb/s) was slower and cost about the same as 1GbE."

I don't think 1GbE was nearly as ubiquitous or cheap in 1997 as it is now - part of the reason IPoFW made sense.


Return to posts index

Sergei Yakovlev
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 8:02:04 pm

[Alex Gerulaitis] "I don't think 1GbE was nearly as ubiquitous or cheap in 1997 as it is now - part of the reason IPoFW made sense."

Right, it made sense for FW400. However, when FW800 was introduced in 2003, 1GbE was already shipping in PowerMacs and PowerBooks, so it didn’t make that much sense anymore.

Now, the situation is different. There are no Apple computers with 10GbE, but all of them have TB. While FW400 provided a 4x theoretical speed-up over Fast Ethernet, TB provides a 10x theoretical speed-up over Gigabit Ethernet. And 10GbE are not coming down in price anytime soon—the market adoption has been slower than everybody expected. So, IPoTB makes even more sense than IPoFW had back in the day.

But, of course, IPoTB is not directly comparable to Ethernet due to the issues I’ve mentioned above.


Return to posts index

EricBowen
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 6:01:19 pm

What that review did show was the potential latency issues with the network packet mode. That is the hurdle TB must climb versus standard network topologies. That is also the main issue I have with the current expectations of TB2. The bandwidth is great however the latency can really limit the results and cause potential troubleshooting nightmares. This would make the TB2 setup far more complicated than a standard network. All of this really reminds me of the Token Rings and the arguments back then and that of course didn't last.

Eric-ADK
Tech Manager
support@adkvideoediting.com


Return to posts index

Sergei Yakovlev
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 8:51:12 pm

[EricBowen] "What that review did show was the potential latency issues with the network packet mode."

Eric,

The review did not show the latency was in the IPoTB. It could be in the SMB protocol or in the file system (and it probably is, with buffers overflowing because of the slow disks). TB has better latency than 10GbE because it is simpler and the packets (frames) are smaller.

If people really insist on testing IPoTB by copying files, they should copy between two SSD RAIDs with average write performance exceeding 1250 MB/s. Of course, in that case, 10GbE should also be tested by copying files.


Return to posts index

EricBowen
Re: this is a GREAT article on Thunderbolt shared networks
on Nov 27, 2013 at 9:22:21 pm
Last Edited By EricBowen on Nov 27, 2013 at 9:22:54 pm

With a 2 device direct connect only without any other device chaining yes. Add other devices and other clients to the TB network chain, then no way it's lower with the latency issues I am seeing. Also keep in mind 10Gbe has low latency profiles that 1Gbe does not. Many may not even be aware or using those. Oh and 10Gbe has definitely come down in price in the last year. The adapters alone are $350 to $550 and the switches are now under $1K.

Eric-ADK
Tech Manager
support@adkvideoediting.com


Return to posts index

Bob Zelin
Re: this is a GREAT article on Thunderbolt shared networks
on Dec 1, 2013 at 8:14:17 pm

I am surprised that this thread is still going on !
At CCW, I saw the 10 meter and 30 meter Corning Thunderbolt 2 cables, being demonstrated like the AT test, showing about 800 MB/sec from computer to computer (with Tbolt 1 10Gb/sec). But of course, no drive arrays, and just a demo - not real life. And as we have discussed, the cost of building a new system that has a "hub" would at the moment require a new Mac Pro Cylinder ($2999 starter price), with 6 Tbolt 2 ports (and who knows that the performance will be on all 6, because there are only 3 Falcon Ridge chip sets on the new Mac Pro), a fast TBolt 2 Chassis (Promise 8 bay - no price yet), plus 5 30 meter Corning cables at $599 each (add $3000 for cables), plus your computers must all have Thunderbolt on them - so retire that 12 core Mac Pro with the NVidia GTX-680 ! :)

But of course, no matter how it works, and no matter what the reality of this system working properly is - the MORAL of this LONG thread is that users like Greg want what Steve Jobs vision was - to be CHEAP AND EASY, so it can "one day" only cost 30 bucks, and not require an expert to make all this stuff work. And his hope was that for 30 bucks, he could do shared storage over Thunderbolt, and tell "all of us" to go "screw off", because he can do it himself. Lets face it - there are plenty of clients out there that can't wait to tell us to GET LOST, so they don't have to hire us anymore. And don't have to buy any more expensive equipment.

But as Eric points out (and as Alex knows), the cost of the 10gig NIC cards and switches have become SO inexpensive, you have to weigh the value in buying a new Mac Pro as a server, along with the $3000 cables, and TBolt 2 drive chassis, when many companies already have lots of wonderful fast Mac Pro computers, big drive arrays, and just need everything interconnected. And by the time that all this stuff wears out, perhaps there will be a Tbolt 2 switch, or even newer technology that will obsolete all of this. Bottom line - some people will just wait until it DOES cost 30 bucks !

Bob Zelin

Bob Zelin
Rescue 1, Inc.
maxavid@cfl.rr.com


Return to posts index

Jayson Packett
Re: this is a GREAT article on Thunderbolt shared networks
on Jul 18, 2014 at 11:35:39 am

Just taken an interest in this subject recently. I am looking into shared storage over Thunderbolt 2. Maybe this is where you get told it can't happen but I would be interested to hear the communities thoughts on this since Apple / Intel have just released drivers to do 10gb over TB2. So current state of affairs for 2014. How is shared storage over TB2 looking now?
And a supplemental question, what would be the best way to connect 3 work stations to the same shared storage?


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Jul 18, 2014 at 3:27:20 pm

[Jayson Packett] "Apple / Intel have just released drivers to do 10gb over TB2."

Where did you read/hear that that?

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Jul 18, 2014 at 6:18:34 pm

[Jayson Packett] " I am looking into shared storage over Thunderbolt 2. Maybe this is where you get told it can't happen but I would be interested to hear the communities thoughts on this since Apple / Intel have just released drivers to do 10gb over TB2."

Hi Jayson,

Honestly, I think this is wishful thinking on your part. I've scoured Apple Support for new drivers and I just don't see anything new that offers any changes to the issues we've written about here pertaining to shared storage NOT working over T-bolt or even T-bolt-2.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Jayson Packett
Re: this is a GREAT article on Thunderbolt shared networks
on Jul 18, 2014 at 10:13:58 pm

To clarify, I think it was Intel making the announcement they were releasing drivers for Windows for 10GBE over TB as Mavericks now has this.


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Jul 18, 2014 at 10:21:20 pm

[Jayson Packett] "To clarify, I think it was Intel making the announcement they were releasing drivers for Windows for 10GBE over TB as Mavericks now has this."

Well, unfortunately the networking protocol necessary for effective shared storage is still not built-in to Thunderbolt, so don't rush out to buy the parts, because you'll be disappointed.

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

Jay Bowman
Re: this is a GREAT article on Thunderbolt shared networks
on Aug 26, 2014 at 4:59:46 pm

David,

Two years ago we replaced our Apple XSAN with an EditShare Energy 48 TB shared storage solution. We integrated a Blade G8124 24-port 10GbE switch for connectivity to seven 2010/2011 Mac Pro towers via Myricom 10GbE cards. We recently replaced our old Mac Pros with 2014 Mac Pros and a few iMacs. We currently use the mLogic mLink thunderbolt expansion chassis to house the Myricom cards. EditShare, Mac Pro and Myricom are all optimized. I have been running speed tests and have found that write speeds are well over 550 MB/sec, but read speeds suffer at times. Meanwhile, the Myricom driver says it is experiencing no issues when it runs its test at boot-up. All systems seem to report optimal functionality but we continue to have latency issues when working with just a few streams of FCPX optimized DVCPro HD 100 Mb/sec video. Have you or anyone you know of bridged the gab between TB or TB2 to 10GbE successfully?

Jay Bowman
Post Production Manager
Abernethy Media Professionals


Return to posts index

David Roth Weiss
Re: this is a GREAT article on Thunderbolt shared networks
on Aug 26, 2014 at 7:51:58 pm

Hi Jay,

The issues I've mentioned previously were all about using Thunderbolt storage, not about using T-bolt to 1Gb Ethernet or 10Gb Ethernet to connect to T-bolt enabled workstations - we routinely do that and have no issues do so.

However, I know for a fact that the 10GbE cards we use do have new specialized drivers that enable Thunderbolt connectivity and performance, but many cards on the market do not yet have such drivers. You should check with Myricom to see if they can provide you with updated T-Bolt drivers.

Hope this helps...

David

David Roth Weiss
ProMax Systems
Burbank
DRW@ProMax.com

Sales | Integration | Support

David is a Creative COW contributing editor and a forum host of the Apple Final Cut Pro forum.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2019 CreativeCOW.net All Rights Reserved
[TOP]