ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

Mixed platform render farm

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Max Palmer
Mixed platform render farm
on Jun 3, 2011 at 5:07:42 pm

If I'm on a mac currently, and I want to add a couple of intel boxes to use as render nodes, is this easily done, or should I simply get two more macs? I'm trying to be price conscious, but I'm not sure how easy it would be trying to get AE render engine on the PC's if I don't have the windows version of CS5.


Return to posts index

Walter Soyka
Re: Mixed platform render farm
on Jun 3, 2011 at 5:30:35 pm

[Max Palmer] "If I'm on a mac currently, and I want to add a couple of intel boxes to use as render nodes, is this easily done, or should I simply get two more macs? I'm trying to be price conscious, but I'm not sure how easy it would be trying to get AE render engine on the PC's if I don't have the windows version of CS5."

Todd mentioned in your thread a few days ago [link] that you'd need a Windows license. The Setup and installation page [link] has this to say:
In addition to the full version of Adobe After Effects, you can also install additional copies on additional computers to use as After Effects render engines to assist with network rendering. You install render engines in the same manner as the full version of the application, but you do not activate them. You run the render engine using the Adobe After Effects Render Engine shortcut in the Adobe After Effects CS5 folder.

You may also need separate licenses for your plugins. Licensing varies by developer; some licenses are platform-specific; some allow render nodes; some require separate render-only licenses. Fonts, too, are often licensed by the number of machines they'll be installed on.

You will absolutely need fast storage and a fast network for AE's network render to be worthwhile.

You'll want to disable all GPU-acceleration within effects that support it, unless all the render nodes are totally homogenous.

Cross-platform rendering works, but you need to be mindful of filename and path conventions.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Max Palmer
Re: Mixed platform render farm
on Jun 3, 2011 at 5:35:48 pm

Ah sorry, I didn't know if I was being clear in my last post. I will definetly have some sort of network access storage, and a gigabit connection to said device. What sort of drives should I be looking for in a network device? Can I just do standard SATA, or would I need to have a raid setup? I can't have something crazy expensive. I was currently only planning on spending about $300 or so for a basic dedicated network server box. Right now, I'm only looking at doing maybe 2 render nodes, and it would be shared at most between two people. Projects would be small, not hi-def as of right now.

Right now it seems like I'm either going to ahve to decide between building cheaper intel boxes, and then getting a windows lisence for AECS5, or just buying some mac boxes and using the mac software I already have.

Thanks for the help!


Return to posts index


Walter Soyka
Re: Mixed platform render farm
on Jun 3, 2011 at 5:55:56 pm

[Max Palmer] "I was currently only planning on spending about $300 or so for a basic dedicated network server box."

The speed on this will probably hurt you. Here's my stock answer to network rendering questions -- maybe this will help. Let me know if you have any questions that aren't addressed here.



Let me first outline how network rendering works with After Effects, and then I will list a couple of bullet point notes from my experiences with network rendering. Much of this information is found in the manual [link], but some of is not. All of it was learned the hard way.


How After Effects Network Rendering Works

You use Collect Files to get your project, its render queue, and all required footage together into a single folder. You configure the render nodes to watch that folder; once Collect Files is complete, it creates a render control file. On the render nodes, you select the folder to watch. When the render control file appears in it, the render nodes begin work according to the options it specifies. All watch folder network renders are done as image sequences, with each instance of AE on each render node working on one frame at a time.

There is no active communication among the render nodes, and the machine hosting the watch folder does not actively control them. Control is completely passive -- each machine looks at the shared storage to see if "Frame 0000" has been rendered yet. The first one to see that there is no image for "Frame 0000" will create that first file in the image sequence to claim responsibility for the frame, then begin rendering it. Finally, when the frame is complete, it overwrites the placeholder "Frame 0000" file.

As other render nodes join the party, they will see that "Frame 0000" already exists, so they will skip it and look for "Frame 0001." If "Frame 0001" already exists, they'll continue sequentially until they find the first frame that doesn't exist, create a file for it to claim responsibility for it, and begin rendering.

All the render nodes are also accessing footage from the shared storage, so that storage system is seeing many constant small reads and writes.

As you can imagine, this places some serious strain on the network and disk subsystem on the server.

Aharon Rabinowitz outlines a method for using multiple computers to render, but without watch folders (and without the coordination that watch folders allow). Depending on your needs, it might be worth trying:
http://library.creativecow.net/articles/rabinowitz_aharon/multi_machine_ren...


Notes on After Effects Network Rendering
  • Each of AE's render nodes is self-directed; there is no supervisor or controller for distributing the workload. (See above for details.)
  • You need a very fast network and very fast storage on your server. (Again, see above for details.)
  • Network rendering will create an image sequence on your shared storage. You can add a post-render action to re-import the image sequence and output a movie, but this will only be handled by one computer. If there is sound in your project, don't forget to output it separately if necessary.
  • You must specify the maximum number of render nodes when you set up the watch folder render. Multiprocessing increases your node count. For example, if you have four quad-core machines, you will have 16 render nodes.
  • All fonts and plugins used in your project must be installed on all render nodes.
  • Some plugins are licensed for unlimited use in render farms; others are limited to a specific number of render nodes; still others must be licensed for render nodes separately. Many plugins activate through their installers, but others activate with the AE environment itself. In this case, you will need a full license of AE in order to activate them. Most font licensing also limits the number of workstations they may be installed on.
  • Some plugins allow GPU acceleration. I highly recommend turning this off unless your render farm is completely homogenous.
  • Cross-platform file naming and path conventions can be tricky.


Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Matt Dobbins
Re: Mixed platform render farm
on Nov 17, 2016 at 8:11:14 pm

Hi Walter,

I was wondering; have the last 5 years significantly changed the information you've laid out in this thread? I work with a small but powerful editing studio and, as it is the end of the year, they are entering the buying season. We are looking to streamline and improve our systems so that we can offer more to the clients and offer it more efficiently, specifically with our AE and C4D rendering.

We've got two 4,1 two 5,1 and one 6,1 mac pros. We are looking at 3 different options; 1) buying another 6,1, 2) upgrading the 5,1s, or 3) building some PCs to incorporate into our mac workflow as render workhorses.

Are you still having success with your PC/mac integration and, if so, what would you recommend to update the information you posted in 2011?

Best,
Matt Dobbins


Return to posts index

Walter Soyka
Re: Mixed platform render farm
on Nov 18, 2016 at 11:57:10 am

My team is cross-platform; some of us prefer Windows, some of us prefer Macs. Adobe apps and C4D still provide a seamless cross-platform experience.

It's hard for me to give good forward-looking advice on Ae rendering right now. The After Effects team is in the middle of a major overhaul of the Ae architecture, so there's currently no multiprocessing feature, and they have started introducing GPU-based rendering for certain effects. Hopefully, the best path forward here will become clearer soon

I will say that while we've been working with the new versions of Ae for the day-to-day workflow improvements, we've been doing our heavy renders on Ae CC 2014, with multiprocessing, on beefy dual-CPU PC workstations with lots of RAM. If I were building a render farm for Ae (which I keep threatening to do), I'd look at a render farm application like Deadline or Tractor to control it.

For C4D, Team Render is very easy to use and can take advantage of whatever computers you have, but I'm most excited right now about ZYNC Render, running the C4D renderer on Google's cloud infrastructure:
https://zyncrender.com/

To answer your question generally, though, my perspective is that Apple is not currently focused on building performance computers. I would not choose another nMP (6,1) as dedicated render machine today -- that machine is already three years old and difficult to substantively upgrade. Upgrading your 5,1 systems might make sense (depending on what configurations you have now and what upgrades you're considering).

Integrating PCs has worked for me. I recommend buying systems from a content industry-focused integrator (unless you really know what you are doing). I'd also suggest that any Mac shop looking to integrate PCs in their workflow should exercise a little patience getting going. There's a little bit of a learning curve, and it's easy to take for granted how much Mac-specific knowledge you have picked up that you may not have right away on Windows. For me, though, the switch has certainly been worthwhile.

Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive [twitter]   |   RenderBreak [blog]   |   Profile [LinkedIn]


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]