link aggregating the client (AVID ISIS 5000)

Hi -
I have been told by people on this forum, and by two different reliable people at AVID that they can achieve about 150MB/sec over DUAL ethernet on a client, if they link aggregate from two etherent ports on the client back to the ISIS 5000 server.

Now, I have been told in the past that this will not work -
from the Small Tree site, for example -

"Q: Can you trunk with link aggregation from the server to the client and go twice as fast?
A: No. This is a common misconception. Many inexperienced Apple resellers will make this mistake, which is why you want to talk to a networking expert before installing any serious infrastructure.

Link aggregation works by load-balancing “sockets” (referred to in the IEEE 802.3ad spec as “conversations”) over all of the available ports. On a server, this works wonderfully since a server might have 10 clients connecting. Each of these 10 sockets can be routed over a different port, providing an evenly-loaded network with a higher aggregate bandwidth.

Clients generally connect to one server. Hence, if you run link aggregation on a client, this one socket (conversation) will run over one of the ports. You might see some traffic on the second port if you’re also checking e-mail or viewing web pages, but you won’t see 2X the bandwidth to the server. Some people implement link aggregation on clients, but it’s primarily for redundancy (fail-over capability)."

So I tried this today, extensively, in every combination I could think of, using the smae hardware that AVID uses (but not an ISIS 5000) and I while I could successfully link aggregaate to the client, I could NEVER achieve the 150MB/sec speeds from the client over dual ethernet that AVID claims they can achieve. Has anyone here ever seen this actually work ?

Bob Zelin

Hi Bob,
I use an ISIS 5000 and get 175MB/s writes and 145 MB/s reads on a single chassis, and could potentially achieve even more with multiple chassis. My client system is an 8-core Nehalem Mac with the built-in NIC. I tested this with Avid's disk testing tool PathDiag and verified it looking at Activity Monitor and iStat Menus. I get about 95 MB/s on one adapter and 50 MB/s on the other for reads. The system is NOT link-aggregated, instead the two adapters have IPs on the same subnet and traffic is distributed between adapters connecting to the server.

The ISIS has its own (2.2 Megabyte) kernel extension, uses its own communication protocol over IP and even has its own file system. They seem to have developed a lot of "secret sauce" to make it function more like a SAN than a NAS. This is different than other products that that are basically made by VARs utilizing SMB/AFP/NFS (editshare comes to mind here) or more complex systems on the sever side (like Isilon) that still utilize standard client software. ISIS (7000) up to at least version 2 used OSR's File System Development Kit in part for their file system.

There is very little public information on how things are supposed to be set up in a dual environment, but it does seem to work as advertised.

David Laster
NYC Post Production Consultant

Honestly I don't think it is too complex to split data up over multiple connections but most protocol stacks don't support it. Have you played around with NFS? It creates more packets than SMB/AFP that can potentially be load balanced. You can run the daemon on the mac and it is built into Mac OS X Server. This may work well with standard link aggregation if you can find out how to do that on a mac. Another thing with ISIS is that

Do you have access to an ISIS 5000? I'm sure you could learn a lot about how things work by running Wireshark on a client and try copying some text files. Do a netstat -all in the terminal as well.

I forgot to mention too that my machines are using regular 1500 MTU packets not "Jumbo Frames"

The theoretical limit for Gig E is 125 MB/sec, so there is still a bit of room beyond the ISIS 5000's performance.

David Laster
NYC Post Production Consultant

Hi,
which protocol they use on isis over cooper 1Gbit or dual 1Gbit connection (TCP) or others??

On DDP - DynamicDrivePool we can use MCS - Multiple Connections per session with up to 4 x 1GbE or 10GbE. And we have been doing this for years. We run our own iSCSI/AVFS protocol and our own file system.

As we are IP SAN we can get 200MB/s Read and Write with 2 cables

I think Avid `borrowed technology`from Maximum Throughput (Sledgehammer) but they are still behind us ;)