ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

first foray into networked rendering...any advice appreciated.

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Joe Procopio
first foray into networked rendering...any advice appreciated.
on Nov 26, 2008 at 1:37:44 pm

we have a big job coming up to do the open and bumpers for SNL...we did the same thing for the Thursday Night Update for the elections, and since it was our first time doing something for them with such a SHORT deadline...we scrambled to get them out with our 4 AE systems and 3 Final Cuts running CS3 sneaker netting things, faster then our then network would allow...but now we have a common server connecting these systems with a GigE connection, so I'm wanting to try out this network rendering...now these are all 1080i exports coming out of AE, that we need to create DVCPro QTs to work in FCPs...we had a hard time getting things correct the last time, and ended up importing them into AVID, and spitting them to tape for the broadcast...i want to eliminate that...(the issue was a field dominance thing that we couldn't get right in FCP)

now to do the network rendering, I read that it actually creates a series of png files, that is what allows multiple systems to render the same project...then we could bring them into QT Pro and export a QT...does this sound correct?

I'm pretty sure we won't have plugin issues, but I'm checking on that...we didn't last time. And we have CS3 on all the systems, only 1 isn't licensed, but we don't need it for the rendering part.

Any issues I should be aware of when we do this?

the 4 AE systems are Quad 2.5GB Power PC G5s with 4Gb of ram, and the 3 FCPs are Dual Quad 3.0 Mac Pro Intel machines with 8GB of ram, 1 running Leopard 10.5.5, everything else is running Tiger 10.4.11. We are not utilizing any of our PC based CS3 machines since I'm too concerned with cross platform issues.

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Walter Soyka
Re: first foray into networked rendering...any advice appreciated.
on Nov 26, 2008 at 3:25:14 pm

Hi Joe,

In addition to the plugins, you should make sure you have all the necessary fonts installed on all the necessary render nodes.

When collecting the files for the watch folder, make sure that your number of machines allowed accounts for your multiprocessing settings (i.e., 4 quad machines would be 16).

If a render fails, check the last few frames committed to disk to make sure they are properly rendered (non-blank). Delete any bad frames. Then you just re-add the comps to the queue, re-collect to the network, and it will skip all the previously rendered frames

Your performance will probably depend on how beefy your server is, because AE's network renderer is unsophisticated. It's pretty easy to tie up the disks or saturate your network when you've got multiple render nodes each looking for the next frame to render, then pulling the footage it needs for the frame, then writing it back out.

If you have several sequences to render, it might be faster for you to collect the files onto your render nodes directly, with each node watching a folder on a directly-attached drive, and have each node work on one sequence at a time, locally. If it's just one sequence, you could split it up manually by setting different start and end frames, then manually collect them all into a single folder to create your Quicktimes. It's extra manual to set up and finish, but you will save time by not creating a bottleneck at the server or network.

I always use image sequences for my renders, whether they are network renders or not, because if you have to make a tweak, you only need to re-render the affected frames and recreate the QT movie from the image sequence. Again, it's a couple extra steps, but it really saves time on lengthy renders.

If you've got time, I'd suggest you take an old project and run a test of both the above scenarios to see which is really faster with your setup.

Good luck,

Walter Soyka, Principal
Keen Live, Inc.
Presentation, Motion Graphics & Widescreen Design
RenderBreak: A Blog on Innovation in Production



Return to posts index

Jan Sherlink
Re: first foray into networked rendering...any advice appreciated.
on Nov 26, 2008 at 4:50:22 pm

You can automate the Quicktime-making

Drag your comp to the Render Queue;
in the Output Module (image sequence!) select "Post-Render Action = Set Proxy"

Drag The comp to the Render-Queue again;
in Render settings select "Us All Proxies" and choose the codec you want !

The first render, all Workstations will render the Image Sequence.
with the second render, 1 system will collect all Images and render a Quicktime.


that is ...
... if you have a decent and speedy network !

cya,

Jan


Return to posts index


Joe Procopio
Re: first foray into networked rendering...any advice appreciated.
on Nov 26, 2008 at 4:57:19 pm

so am i correct in thinking that i could set one of the 8 core machines to render the QT automatically even though the original project was done on one of the older quads?

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Walter Soyka
Re: first foray into networked rendering...any advice appreciated.
on Nov 26, 2008 at 5:09:03 pm

Stitching the image sequence together into a movie will be disk- or network-bound, not CPU-bound, so the speed of the workstation won't make a big difference.



Walter Soyka, Principal
Keen Live, Inc.
Presentation, Motion Graphics & Widescreen Design
RenderBreak: A Blog on Innovation in Production



Return to posts index

Jan Sherlink
Re: first foray into networked rendering...any advice appreciated.
on Nov 27, 2008 at 7:25:09 am

As Walter said;
this is more a throughput-thing and not CPU intensive.
The first file in the queue will be picked-up by all computers to render the image-files,
the second file in the queue will be picked up by "one of the computers" and rendered again but since the proxy is now enabled it's just a conversion from image-sequence to the codec of your choice.
Keywords there are GByte network/ fibre / Raid ;-)



cya,

Jan


Return to posts index


Rhett Robinson
How Many machines?
on Nov 27, 2008 at 6:26:05 pm

This caught my attention
"I'm pretty sure we won't have plugin issues, but I'm checking on that...we didn't last time. And we have CS3 on all the systems, only 1 isn't licensed, but we don't need it for the rendering part. "

You sound very knowledgeable, but wanted to be sure you knew that you don't have to have a full AE install to be a net render client, although all machines will have to have the same plugins -



Return to posts index

Joe Procopio
Re: How Many machines?
on Dec 1, 2008 at 1:28:15 pm

right...that's why i mentioned we don't need the license for the render...maybe i wasn't clear, but I know that, thanks!

i'm going to do my initial test today, and hopefully all will go well...

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Joe Procopio
Re: first foray into networked rendering...any advice appreciated.
on Dec 8, 2008 at 4:52:49 pm

so now that I have a chance to test this....can someone give me explicit directions? I would research all this myself, but I am under the gun now, and only have until wednesday to do make it work.

I have a composition ready to go...it is loaded up...what steps from here? i'm not an AE guy...I'm an Avid/FCP editor and don't use AE, so I apologize...our designers are too busy at the moment to work with me, so I'm on my own...

i have created a watch folder...i was about to collect files...am i on the right path? do I need to send the sequence to my render queue at all on the main system? or is that what sending it to the watch folder does?



Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index


Walter Soyka
Re: first foray into networked rendering...any advice appreciated.
on Dec 8, 2008 at 6:26:58 pm

Hi Joe,

You're on the right track.

Add your comps to the render queue. Use "Multi-Machine Settings" for the Render Settings and "Multi-Machine Sequence" for the Output Module.

Next, Collect Files. You can collect source files for queued comps. Check "Change render output to" and give it a meaningful name (defaults should be ok). Check "Enable Watch Folder render," and choose a suitably high number of machines to allow. When you click the collect button, it will copy all the necessary source files to the folder of your choosing.

Make sure do you put everything into a folder, because you cannot use Watch Folders on the root of a volume. I'd advise creating a Render folder on your shared storage, and then collecting files for sequences into their own sub-folders of the Render folder.

Next, Watch Folder on each render machine, and point it to the Render folder (which contains the sub-folders for each comp).

More information here:

http://livedocs.adobe.com/en_US/AfterEffects/8.0/help.html?content=WS387852...


Walter Soyka, Principal
Keen Live, Inc.
Presentation, Motion Graphics & Widescreen Design
RenderBreak: A Blog on Innovation in Production



Return to posts index

Joe Procopio
Re: first foray into networked rendering...any advice appreciated.
on Dec 8, 2008 at 7:53:35 pm

thank you...that sounds logical enough....

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Joe Procopio
Re: first foray into networked rendering...any advice appreciated.
on Dec 9, 2008 at 3:39:52 pm

last question I hope...

when I collect files, I choose all, can i just copy the files for that particular comp, or if there are many comps using the same media, can I just point them all to the same folder?

in other words, what is the best way to handle working with multiple comps and the same media?

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Joe Procopio
automating QT creation
on Dec 9, 2008 at 3:32:28 pm

so i got the multimachine render going...now trying to get the automated QT creation happening...when i drag the comp a second time, go to render settings: custom: use all proxies... and then in output moule, i select my particular codec, yes? DVCPro 1080i as this would have it.


what system actually creates the QT?

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Joe Procopio
Re: automating QT creation
on Dec 9, 2008 at 3:53:23 pm

one more last question...how does it deal with audio?

Broadway Video, NYC
AVID/FCP editor/engineer


Return to posts index

Jan Sherlink
Re: automating QT creation
on Dec 9, 2008 at 5:10:06 pm

There's no "linking all comps" to the same media.

Queue the comps that need to be rendered (the automated QT renders also)
When collecting select "Queued comps" and reduce project;
If you have 20 comps and only 10 queued, AE will only collect the 10 queued comps and the media it needs. (you'll see the amount of Megabytes changing when you do this)

After rendering the Image-Seq, the first computer that's free will start rendering the Quicktime.
You can't select wich computer it will be !
You can select whatever codec you want, but all Computers in the network should have that codec installed.

Since the Image-Seq is used as a proxy, and can't contain audio, your final QT won't contain audio either.
2 solutions:
- Use QT-Pro to "add" audio
- Queue Comp1
Set Comp1 to render as Image-Seq + Set Proxy post render
Drag Comp1 into a new comp (Comp2) and add the audio as a second layer.
Queue Comp2 with Use all proxys




cya,

Jan


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2020 CreativeCOW.net All Rights Reserved
[TOP]