FORUMS: list search recent posts

Slightly OT - Curious about effective link aggregation (trunking) in OSX

COW Forums : SAN - Storage Area Networks

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Ed Murphy
Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 6:10:51 pm

My apologies for being slightly off topic, but this seems to be the most appropriate forum for shared storage and networking discussions.
I wonder if anyone can speek to my recent experience with link aggregation and managed switches in a Mac environment? With some simple tests, I found what I think is strange behavior...

For a few years our setup has consisted of two Netgear GS108T "smart" switches, two Qnap NAS RAIDs for our media library, four MacPros, and two iMacs. The Qnap NASs are each connected to the switches with LACP link aggregation over two ports each.

Now, I've recently found poor network performance on those computers connected via the second switch which I assume is due to the bottleneck of only being uplinked to the first switch via one GbE port. I would like to resolve that, expand network capacity, and setup link aggregation on the four MacPros for better throughput, by upgrading our switches. I'm looking at the HP ProCurve line as well as other Netgear and Cisco options.

However, I wanted to determine what kind of performance boost I could expect before purchasing, so I set up my MacPro with link aggregation and just did a few simple file transfer tests.
Both tests involved copying a 7.14GB file from the Qnap RAID to a local drive and back again. The results were the same in each direction.

Test 1: MacPro connected via single GbE port 5, Qnap connected via aggregated ports 1 and 2. (Current setup.)
Transfer rate in activity monitor peaked and sustained approx 80MB/sec, transfer took 1:41 in time.

Test 2: MacPro connected via aggregated ports 3 and 4, Qnap connected via aggregated ports 1 and 2.
Transfer rate in activity monitor peaked around 200MB/sec and sustained approx 160MB/sec, transfer took 1:41 in time.

So, while there appeared to be about double the network traffic from one case to the other, the transfer took just as long in both setups! I repeated this several times. How could that be?
Anyone have any idea what might be going on?
Should I expect different results with a newer switch, like the HP for example?

Any insights are appreciated.

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television


Return to posts index

Eric Hansen
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 6:59:28 pm

not off-topic at all.

Steve Modica can chime in with the technical explanations, but basically, OS X can't benefit from link aggregation on the client side. it can only support a single connection to a server, in this case, your NAS.

doing a link agg from your NAS to your switch has benefits however, as it doubles the number of links available to outside clients using the same IP address (it doesn't go faster, it just makes more connections available)

as far as seeing double the speed, and high bursts, I believe this is the OS sensing more than there really is. If you setup a RAID-1 in disk utility on 2 internal drives, and then copy a file to it and watch it in Activity Monitor, you will see a read speed of 80MB/s (for example) and 160MB/s write because the system is writing to the 2 drives in the RAID-1 and counting that as 2 streams. i think if you link agg on the client side, you will see a variation of this, and that's why it still takes 1:41 to copy the file.

hopefully steve chimes in with a more technical answer

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

Ed Murphy
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 8:18:05 pm

Thank you, Eric.

I suspected it was something like this, and Petros Kolyvas just gave me the "one train can't ride on two tracks but two trains can" analogy in another forum's old thread I had revived asking the same question.

So my follow up question is: If I easily have the means to do so since I'll be upgrading switches anyways, is it totally pointless to set up link agg on the four MacPro clients I have? Or is it possible that would be helpful in scenarios other that a single file transfer?

It sounds like it would possibly help in situations where one workstation was sending or receiving data to multiple NASs or other workstations in terms of a file transfer or similar. Is it also possible though, that the OS/network/NLE (PPro in this case) would be smart enough to use those multiple links when working with multiple sets of media? Perhaps only when that media is in different locations?

Thank you again for any thoughts.

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television


Return to posts index


Bob Zelin
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 10:28:45 pm

http://forums.creativecow.net/thread/197/858151

Bob Zelin



Return to posts index

Bob Zelin
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 10:33:03 pm

from the Small Tree site

Q: CAN YOU TRUNK WITH LINK AGGREGATION FROM THE SERVER TO THE CLIENT AND GO TWICE AS FAST?

A: No. This is a common misconception. Many inexperienced Apple resellers will make this mistake, which is why you want to talk to a networking expert before installing any serious infrastructure.

Link aggregation works by load-balancing “sockets” (referred to in the IEEE 802.3ad spec as “conversations”) over all of the available ports. On a server, this works wonderfully since a server might have 10 clients connecting. Each of these 10 sockets can be routed over a different port, providing an evenly-loaded network with a higher aggregate bandwidth.

Clients generally connect to one server. Hence, if you run link aggregation on a client, this one socket (conversation) will run over one of the ports. You might see some traffic on the second port if you’re also checking e-mail or viewing web pages, but you won’t see 2X the bandwidth to the server. Some people implement link aggregation on clients, but it’s primarily for redundancy (fail-over capability).



Return to posts index

Bob Zelin
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 10:36:12 pm

so, with all that said -
I have actually casually discussed this with Steve Modica at one point, but never got into it, or understood it, BUT -
I am right this second working with an AVID ISIS 5000 at Disney, and I can take two ethernet cables from the two ports of the Mac Pro, assign static IP addresses to them, and shove them into the Dell Force 10 S25 switch, and get about 170MB/sec bandwidth from a Mac Pro WITHOUT using LACP. I have NO IDEA of how this works, but AVID said it would work, and they were right. I wish I knew how they did this.

Bob Zelin



Return to posts index


Eric Hansen
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 6, 2012 at 10:56:54 pm

Bob wrote: "I wish I knew how they did this."

by charging a lot of money :)

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

James McKenna
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 7, 2012 at 1:59:57 pm

"by charging a lot of money :)"

That's right. The client software you're paying for has the ability to break up the requests along a dual paths. Each port has it's own IP address. I was out of Avid way before ISIS became a shipping product, but I've heard that this is the way it's done, and that makes sense to me. Personally I think 10GB-baseT over copper 6a is the more elegant solution, and optical is even better (then FC becomes an option).

In the end it doesn't really matter for that product, even if you can push 190MB/sec on a single client, you can do that on all of TWO clients before your ISIS 5000 drops to it's knees.

Jim McKenna
Facilis Technology
978-562-7022 x101


Return to posts index

Ed Murphy
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Dec 7, 2012 at 5:24:34 pm

Thank you for the great information, Bob. I read the thread you linked and appreciate having a better understanding.

It seems most points suggesting it wont help performance are citing the scenario of many clients talking to one server, and you said:
"Clients generally connect to one server. Hence, if you run link aggregation on a client, this one socket (conversation) will run over one of the ports. You might see some traffic on the second port if you’re also checking e-mail or viewing web pages, but you won’t see 2X the bandwidth to the server. Some people implement link aggregation on clients, but it’s primarily for redundancy (fail-over capability)."

However, in our environment we have several clients connecting to 2 or 3 servers, (NASs) and sometimes to each other. If my clients were link aggregated, would that offer the occasional performance increase when dealing with multiple connections, just as it does on the NAS side with multiple clients wanting to connect?
I totally understand now that I'm not doubling my throughput over any one connection, but it seems like this would still be a minor performance boost in our environment.

If nothing else, there's really no down side to doing it if I have the ports and the coper anyways, right? I mean aside from my time spent of course, but then, heck we'd have fail-over too...

Thanks again for everyone's input!

I also have a related question.
Could I uplink via 10GbE to one of my NASs to at least improve performance to/from that node? I originally thought uplinks were only meant for other switches.
What I imagine in asking this question:
Our largest NAS is the Qnap 879 Pro, which has the option to install a 10GbE NIC for about $300. I was previously looking at 24 port HP 1810 series switches, however those don't appear to have 10GbE uplink options. If I were to instead go with one that does appear to, such as the Cisco 2960-S line, or something from Small Tree, could I theoretically uplink via 10GbE to our 879 Pro for greater performance of at least that media storage to the rest of the network?
Forgive me if this is an ignorant question.

Thank you again for your thoughts.

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television


Return to posts index


Matt Geier
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Jan 9, 2013 at 4:17:43 pm

Hi Ed,

Link Aggregation on the client won't work out of the operating system the way you'd expect.
For you to link aggregate those client ports and pass data down both at the same time, requires the software application to support this, much like the reference to AVID Isis.

If you set this up out of the os without something special from the software on the client side, you'll see data move over 1 of the links. The other will be connected and appear to be doing something at the same time, but if you watch the io you'll see what I mean.

Link Aggregation on the server will be what you look to do, even if you have more then one server, you just set them all up with link aggregation to your managed switch.
This is where most folks will implement link aggregation, even for six ports of 10GbE in some cases, not just 1GbE networking.

Smart Switches are a one step up from Unmanaged Switches. They have some things in common. Most of them cannot be set up to properly support Dynamic Link Aggregation which is what Apple O/S uses by default. Dynamic is the best for cases of shared storage most of the time. These kind of switches also don't have high packet forwarding rates and lack in overall processing power to manage the data properly, leaving you at the mercy of the switch performance and what it does do.

They should not be used for video or traffic/data intensive networks. It's not the right choice.

Perhaps calling some of the companies and speaking to the vendors will help you better understand this and get some clear cut answers to your questions.

Feel free to call Small Tree or shoot over to our creative cow forum for more consult on the subject. I represent their West Sales region of the US and Canada. Maybe connecting to get some user references and other resources is what you need too.

Let us all know what else we can do to help further.

Best regards, and good luck,

Matt Geier
(Video Networking Solutions Expert)
(Creative Design Workflow Consultant)
(Social Media Networks Consultant)
(Technical Video Industry Sales Consultant)


Return to posts index

Jon Thomasberg
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Feb 12, 2013 at 4:10:25 am

Channel bonding is certainly possible to use more than 1 link between: host and NAS (which is technically just another host acting as a iSCSI target, CIFS, AFP, or NFS server); host and a capable enterprise-class switch (we are talking Arista, Cisco Nexus, etc. not some cheap thing from your local big box retailer). This is accomplished either via specialized software (as already cited) or by configuring Channel Bonding in 'balance-rr' mode (a.k.a mode 0). Be aware that TCP sessions will have packets arrive out of order and require significant retransmits, reducing the aggregate bandwidth to less than the sum of the links, but still significantly more than a single link or attempting to combine the throughput of two links using LACP, as many of you all have tried and realized that it is no faster than single link, this is due to the fact that LACP uses a per-destination flow over only 1 link, regardless how many links you aggregate in the LACP. This is done because traffic is segmented in chunks (size defined by maximum transmission unit, or MTU) and each segment is given a sequence number so that the segments arrive in order for reassembly on the other end.

So back to 'balance-rr' (load balancing, round-robin) mode: All the links in the bonded virtual interface must be of the same speed, duplex, MTU, buffer size, etc. or it won't work. This works in Linux and in Mac. I am not certain how to accomplish this in Windows, or if it is even possible.

If you have linux, you can read up on the topic here:

/usr/src/linux/Documentation/networking/bonding.txt

...pay special attention to section 12.


This is the beginning piece. However, if you are savvy in networking and you feel comfortable editing .conf files in linux CLI, you should be able to accomplish this quite simply. On the other hand, if what you have read above sounds like Greek to you, you can: A) start reading up on it, learn it and test (on a sandbox, non-production system); or, B) hire someone knowledgeable to do this.


Return to posts index

Nick Vasilopoulos
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Mar 9, 2013 at 3:53:22 am

Hi Ed,

Yes you can uplink from your switch to a NAS, but it depends on your switch. I've got similar hardware to you (QNAP 879) and have been beating a similar path as you - to get more bandwidth - via link aggregation, balance -rr, etc - I even went as far as purchasing the 10G card for the QNAP and have been testing it.
But back to the switch. I've recently upgraded after a long search to Brocade ICX 6450-48 switches which have 2 10GbE SFP ports and 2 1GbE SFP ports. (the 2 1G ports can become 10G ports with a license upgrade - i.e, more money). You can use the 10G for uplinking to another exact model switch (and then manage the switches as 1 device via IP/GUI) or you can turn off trunking for this port and simply use it to connect to a 10G NIC - like the one in my QNAP.

I will update this thread with my findings.

And BTW, I have no affiliation with any company mentioned, I'm just a paying customer!

Best,

Nick


Return to posts index


Ed Murphy
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Apr 1, 2013 at 4:17:53 pm

Hi Nick,

I would be interested in your findings. We elected to shelve the idea of getting a more expensive switch to try and achieve a 10G uplink to the QNAP - mostly because I couldn't find anyone to definitively confirm that would work. But, we are still within our 30-day return window!

Have you successfully connected your switch to your 879 via 10G and with multiple clients on the 1G side of the switch? What kind of speed enhancements have there been, if any?
I'm very interested to hear about your experiences.

Thank you,

Ed Murphy
Senior Editor / Technical Director
David Lynch Foundation Television


Return to posts index

Christoph Tilley
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Apr 14, 2013 at 12:07:52 am

Hi, I just ordered a Synology DS3612xs and I'm curious for the best setup for us.
The NAS will be connected via single Gbit link to the network. Which is fine for all workstations just doing backups. But for one workstation (MacPro) I would need 10GbitE for very fast file transfers. This workstation has to be connected to the regular GbitE LAN too.

I was thinking of buying two 10GbitE-PCIe Cards, one for the NAS, one for the MacPro and connecting them together.

But how does the OS know to go the 10GigE route instead of the regular Gbit route?

Thanks in advance! Chris



Return to posts index

Eric Hansen
Re: Slightly OT - Curious about effective link aggregation (trunking) in OSX
on Apr 14, 2013 at 7:13:03 pm

- direct connect the 10GbE card on the Mac Pro to the 10GbE card on the NAS. you can't go through a switch.

- put them both on the same subnet (ie, NAS on 192.168.150.10, MP on 192.168.150.20). it has to be something different than the office network subnet, otherwise the Mac Pro will try to connect over the 1GbE port.

- bring up "Connect to Server" with Command-K on Mac Pro. Enter 192.168.150.10

- enjoy 10GbE goodness

of course, i skipped a lot of configuration steps, which I won't go into here. I've never used this NAS. It looks interesting. But these steps should work for most any NAS.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
podcasts
© 2019 CreativeCOW.net All Rights Reserved
[TOP]