ADOBE AFTER EFFECTS: Forum Expressions Tutorials Podcasts Creative Cloud

AE Render Engine not maximizing CPU usage

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Kyle TraynorAE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 4:32:50 pm

So I am currently trying to set up a mini render farm using AE's Render Engine and some spare Mac Pros (specs for all machines are listed below).

So far the engine is working but when viewing the activity monitor I can see that the CPS are only running at about 30%. The workaround so far has been to open multiple instances of the Render Engine on each machine. While this appears to be working it is not the most user friendly solution and sometimes these instances have trouble recognizing the watch folder.

The Render Engine seems to be much more stable with only one instance open on each machine. Does anyone know of a way to get peak efficiency out of these machines without having to open several versions of the render engine?

Here are all my specs:

Rendering out 1080 TIFF Sequences

HOST Render Machine: 
2x 2.66 GHx 6-Core Intel Xeon
32gb RAM
Graphics Card - NVIDEA Quadro 4000 w/ 2048mb-VRAM

Render Mac 1:
2.93 Quad Core Intel Xeon
8gb RAM
Graphics Card - NVIDEA GeForce GT w/ 512mb-VRAM

Render Mac 2:
2x 2.8 GHx Quad Core Intel Xeon
8gb RAM
Graphics Card - NVIDIA GeForce 7300 GTw/ 256MB-VRAM

Render Mac 3:
2x 3.2 GHx Quad Core Intel Xeon
16gb RAM - Graphics Card
Graphics Card - ATI Radeon HD2600 XT w/256mb-VRAM

The images I uploaded show the machines with multiple instances open. As you can see the CPUs are working at full capacity, I want to see if it is possible to do this without the multiple versions open.

The goal is to get our AE render time down to 1:1. I think we are currently around 10:1.

Thank you for your help.

- Kyle

Kyle Traynor
Post Production Artist

Return to posts index

Kevin CampRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 5:55:16 pm

do you have 'render multiple frames simultaneously' enabled in the multiprocessing preferences of the render engine software?

Kevin Camp
Senior Designer

Return to posts index

Kyle TraynorRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 5:58:06 pm

Yes I do, I should have listed that in the original post.

Thanks for responding so quickly Kevin!

Kyle Traynor
Post Production Artist

Return to posts index

Kyle TraynorRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 7:02:54 pm

Sorry I also meant to mention that all machines are running CS5 and that all plugins/text have been matched on each machine.

Kyle Traynor
Post Production Artist

Return to posts index

Kevin CampRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 8:03:17 pm

i was hoping somebody who uses network render would chime in (i don't, so i really don't know all the ins and outs, just the basics)....

but since the render engines should be working performance-wise pretty much the same as ae would normally for a render, i would think the typical performance guidelines would apply...

if you haven't looked at them, todd's 'optimal performance settings' guide is here:

one other issue that could arise with a network render would be network bandwidth... trying to push and pull a lot of large files through a network does require a lot of bandwidth. even if the systems can process the frames quickly, they still need to pull source footage and push rendered frames though the network.

you can monitor network activity in the activity monitor. you might compare that activity with just one render farm mac running compared to all of them just to see if that effects the network speed.

Kevin Camp
Senior Designer

Return to posts index

Walter SoykaRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 8:43:31 pm

Do you have any effects in this project that prevent multiprocessing from occurring?

My (long) stock answer on network rendering follows:

Let me first outline how network rendering works with After Effects, and then I will list a couple of bullet point notes from my experiences with network rendering. Much of this information is found in the manual [link], but some of is not. All of it was learned the hard way.

How After Effects Network Rendering Works

You use Collect Files to get your project, its render queue, and all required footage together into a single folder. You configure the render nodes to watch that folder; once Collect Files is complete, it creates a render control file. On the render nodes, you select the folder to watch. When the render control file appears in it, the render nodes begin work according to the options it specifies. All watch folder network renders are done as image sequences, with each instance of AE on each render node working on one frame at a time.

There is no active communication among the render nodes, and the machine hosting the watch folder does not actively control them. Control is completely passive -- each machine looks at the shared storage to see if "Frame 0000" has been rendered yet. The first one to see that there is no image for "Frame 0000" will create that first file in the image sequence to claim responsibility for the frame, then begin rendering it. Finally, when the frame is complete, it overwrites the placeholder "Frame 0000" file.

As other render nodes join the party, they will see that "Frame 0000" already exists, so they will skip it and look for "Frame 0001." If "Frame 0001" already exists, they'll continue sequentially until they find the first frame that doesn't exist, create a file for it to claim responsibility for it, and begin rendering.

All the render nodes are also accessing footage from the shared storage, so that storage system is seeing many constant small reads and writes.

As you can imagine, this places some serious strain on the network and disk subsystem on the server.

Aharon Rabinowitz outlines a method for using multiple computers to render, but without watch folders (and without the coordination that watch folders allow). Depending on your needs, it might be worth trying:

Notes on After Effects Network Rendering
  • Each of AE's render nodes is self-directed; there is no supervisor or controller for distributing the workload. (See this thread [link] where I explain how the AE network renderer works in some detail.)
  • You need a very fast network and very fast storage on your server. (Again, see the link above for my explanation of how the AE network renderer works and why this is necessary.)
  • Network rendering will create an image sequence on your shared storage. You can add a post-render action to re-import the image sequence and output a movie, but this will only be handled by one computer. If there is sound in your project, don't forget to output it separately if necessary.
  • You must specify the maximum number of render nodes when you set up the watch folder render. Multiprocessing increases your node count. For example, if you have four quad-core machines, you will have 16 render nodes.
  • All fonts and plugins used in your project must be installed on all render nodes.
  • Some plugins are licensed for unlimited use in render farms; others are limited to a specific number of render nodes; still others must be licensed for render nodes separately. Many plugins activate through their installers, but others activate with the AE environment itself. In this case, you will need a full license of AE in order to activate them. Most font licensing also limits the number of workstations they may be installed on.
  • Some plugins allow GPU acceleration. I highly recommend turning this off unless your render farm is completely homogenous.
  • Cross-platform file naming and path conventions can be tricky.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events

Return to posts index

Todd KoprivaRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 8:50:55 pm

aerender doesn't take advantage of multiprocessing in After Effects CS5. It's a bug. The bug is fixed in After Effects CS5.5.

Todd Kopriva, Adobe Systems Incorporated
Technical Support for professional video software
After Effects Help & Support
Premiere Pro Help & Support

Return to posts index

Kyle TraynorRe: AE Render Engine not maximizing CPU usage
by on Jul 29, 2011 at 11:55:44 pm

We are running AE 5.5 (another thing I should have listed in the initial post). I will look into the optimal settings for AE based on each machine. Thank you guys for your help. I will let you know how it is going and if anyone has anymore ideas I would greatly appreciate it.

Thank you,

Kyle Traynor
Post Production Artist

Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2016 All Rights Reserved