ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

Massive Render Strain and a Need to Fix That NOW.

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Jim Zarbaugh
Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 8:49:06 am

Gentlemen- First time poster but desperate enough to come out of the cracks for any help I can get.

Unfortunately, there are so many variables here and I am under such a crunch that I'm not even sure if anyone can help without being here.

I am running After Effects CC on a brand new iMac. I am not a techie by a long shot, but I did know enough to click on my apple for these specs:
OS 10.8.5
34 GHz Intel Core 17
32 GB 1600 Mhz DDR3

I am working on an enormous composition of 9800x2280 in a disastrous idea I came up with to realistically tie four screens together so that something can appear to move from one to the other in real time and space. I am designing virtual scenery for a large Christmas music program. The room has two HD IMAG screens that will be supplemented with two LED screens brought in for the event. The long stage screen is about sixty feet wide. The other is less narrow and hangs closer to the ceiling. Every year the story is the same- even if the project is finished, the long renders kill me and catch me off guard.

I haven't slept in days and I'm sure that is not helping my productivity. I want to be able to deliver the video tomorrow for their dress rehearsal tomorrow night. I'm afraid to even let you in on the details, as I am sure I'm at high risk of being laughed out of this forum for being so careless.

Using particular,I have designed a series of colorful dancing water fountains similar to ones you might see at Disney's California Adventure "World of Color". I have about 225 fountains to control, so I quickly came to the conclusion that I would be better off rendering out individual fountains in Animation codec with alpha channels. I figured I could mix it up like a puzzle and add variety by changing colors. Unfortunately, in an effort to make sure I had fountains that could shoot from one screen to another, I ended up with some pretty huge files. Animation is already clunky, but all of them are at least 2280 high. Their widths vary from 2280 to maybe 6000.

I find that rendering out these huge files with color and glow added may take me another week, and that isn't an exaggeration. On one particularly bad clip where almost all the fountains came on, three seconds took seventeen hours on our render computers. Sure, they are older computers- but not THAT old. One lesson learned- try not to use all the fountains at once!

All that said, is anyone willing to even touch this and throw out a suggestion to get the render time down and make this possible? I have seen this train wreck coming, but for some reason I kept telling myself that it was going to be different this time. Stubborn artist and eternal optimist.

Here is what I tried- ProRes green screen clip to substitute the larger Animation file. No difference. I also used Compressor to convert a file to ProRes with Alpha- which I had never even heard of before tonight. On a twenty frame render test it saved me only twenty seconds (13 min, 29 seconds vs 12 min, 41 seconds). I have even tried exporting single rows of alpha fountains at a time with hopes of reassembling the piece with nine large videos instead of hundreds of individual ones. Right now that still seems to be taking a long time. A one-minute segment of just the bottom screen (where all the action is) has been rendering for 45 hours and still claims to have sixteen hours left.

I WANT to try and scale my video clips in half, but that will mean having to re-size and distribute all 225 fountains and their effects. Not good during a crunch. I've also considered changing the size of the ginormous master composition, but that would most likely hurt the quality of the end product.

Thanks for anyone willing to throw out a suggestion!

Jim
Sleepless in Plano, Texas


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 2:13:07 pm

UPDATE: After a couple hours of sleeping at my desk, I am looking at this with new bloodshot eyes. I think the true culprit here is the comp size. Well, there are many culprits, but the comp size may be the first to go. I think I can copy my master comp four times and trim each screen down to the area of importance.

The question is, how important is it to delete any files that are not seen on screen? Is it enough to turn them off? Or is it so bad that I even need to trim down my used layers so that they are only as long as necessary?

Oh- And I'm pretty sure they used the old "Let's tell Jim that dress rehearsal is two days earlier so that maybe we can get the project finished on time" trick. So while sad, as least I have that.

Thanks again for anyone who is spending any brain power on this!

Jim


Return to posts index

Cassius Marques
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 4:27:27 pm

[Jim Zarbaugh] "how important is it to delete any files that are not seen on screen"

If they aren't present in your composition or if they are, but their video switchs are off you can leave them there.

[Jim Zarbaugh] "Is it enough to turn them off?"

yes.

[Jim Zarbaugh] " Or is it so bad that I even need to trim down my used layers so that they are only as long as necessary?"

This helps a lot. Sometimes AE can't see that some layers are no longer visible and keeps processing on them.

And I'll suggest that you're probably low on RAM and you should DEFINATELY turn multiprocessing off. On a last case, you can save your project and trim it to half/quarter your size and render it in parts (compound effects, cameras and such won't work though, So you should check it if this is doable).


Return to posts index


Dean Pickersgill
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 5:37:04 pm

... And (depending on the resolution or your source) can I suggest setting proxies up to low-res versions of your footage? Hasn't "always" been a solution for me in the past as I've found I often need to go back on a greenscreened element to correct a key after switching from proxy back to the original source, but it has been a time-saving tool I've employed from time to time ...

Rgds Dean
IndiVid Videography


Return to posts index

Dean Pickersgill
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 3:07:47 pm

Jim, you haven't mentioned your disk configuration, what kind of graphics card you're using, and what your AE cache/multiprocessing settings are - might help some of the more experienced guys here to advise you best ;-)

Keep in mind that every effect you add, to every layer you add it, also adds a strain on your memory, disks and CPU's. Certain effects add more strain than others, so the use of a CUDA equipped card will help massively with those that can take advantage of GPU.

It might help also to keep your preview-renders at a quarter resolution - you can always put them back to full before you render the comp, and also to use the RAM preview option rather than just hitting the spacebar.

Rgds Dean
IndiVid Videography


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 3:42:03 pm

Dean,

I'm not at the computer right now, and I'm not even sure what disc configuration even is. However, I CAN find out for you. I'm pretty sure we all have Aja Kona cards at work. I don't know my cache/multiprocessing setting are off hand, but I will say that my limited knowledge of such things makes me believe that I probably set the number high thinking erroneously that I would get more out of it. I know I have it rendering more than one frame at a time, though I have not seen the benefits of such a thing. Of you have any suggested settings, that might really help.

The previews are at the point where I am watching at 1/4 or even custom (1/6 or 1/10). It is pretty rough. I purge the cache regularly, as it obviously fills up quickly.

I'll try and get the rest of that info you as soon as I can. Thanks for the help.


Return to posts index


Dean Pickersgill
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 5:46:36 pm

More info will definately help advise you best (although I'm sure there are more experienced guys here better placed to help than I!). There's plenty of help already in these forums (search 'em) with tips for achieving a great configuration, but more detail about your kit will help these clever bods help you ;-)

Rgds Dean
IndiVid Videography


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 5, 2013 at 2:33:18 am

Dean,

I posted memory and multiprocessing info, but still not sure what the disc configuration is. If you get a chance, can you take a look and give me your opinion? It would really help.

I am currently re-compressing all my animation files to about one sixth the data size. I can hardly tell the difference visually- I am really quite shocked. However, how this will help the final product render is still left to be seen.

Thanks again


Return to posts index

Walter Soyka
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 3:46:19 pm

Jim, as Dean says, your memory & multiprocessing settings are very important. Dean's also right on that disk speed is the driving factor for pre-rendered items. If we know more about your technical setup, we can better identify the bottlenecks.

If you are really in a pickle, I might suggest rendering at half-resolution and scaling that up afterwards for compression and delivery. Soft video is better than no video!

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index


Dave LaRonde
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 4:34:22 pm

There are another things I don't think I've seen in this thread: the delivery specifications and the playback device(s).

They make a big difference in how the project is laid out.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 10:02:38 pm

Dave,

The delivery is two 1080i videos, one 2952 x 360 progressive and one 3456 x 720 progressive- all at 29.97 fps and ProRes 422. I'm not sure of the playback device as we hire that out for this event, but it is pretty high tech stuff. They requested the size and codec and will split the videos into a coule different pieces in order to distribute them across the large LED walls. I can send the spec sheet if it would really help.

As far as my AE settings go, I have enabled disk cache with a max size of 100 GB. the output is AJA KONA3 1080psf 29.97 10 bit RGB. Here is a screenshot of the RAM data.


Not all the computers I am using have the same specs, but getting this one right will help me with all of them.

Thanks again, everyone!


Return to posts index

Dave LaRonde
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 10:53:22 pm

[Jim Zarbaugh] "The delivery is two 1080i videos, one 2952 x 360 progressive and one 3456 x 720 progressive- all at 29.97 fps and ProRes 422."

There is contradictory information here.

"1080i video" implies a comp size of 1920x1080 pixels. Then you say one should be 2952 x 360 and the other should be 3456 x 720.

The "i" in "1080i" says that the video is interlaced. Then you say that both of these really long comps should be progressive.

If this is what you were told, you need to get clarification.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA


Return to posts index


Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 11:02:28 pm

The two 1080x1920 videos will be shown on the IMAG screens and will be interlaced. The videos prepared for the LED screens are progressive. That is a total of four videos. Sorry for the mix up.


Return to posts index

Dave LaRonde
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 11:37:03 pm

I'd nest that one big, huge comp in the smaller comps, taking great care to position it properly for each movie you have to make.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 11:59:28 pm

So the settings look OK?

Thanks- I think breaking it into smaller pieces for render is a good idea.


Return to posts index


Dave LaRonde
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 5, 2013 at 12:31:49 am

Well, I wouldn't pull the trigger on EVERYTHING and expect perfection. I'd set up the renders to do only a second or so of each comp and scrutinize the results. If they look good, THEN I would pull the trigger on everything.

Dave LaRonde
Promotion Producer
KGAN (CBS) & KFXA (Fox) Cedar Rapids, IA


Return to posts index

Walter Soyka
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 6, 2013 at 2:03:07 am

[Dave LaRonde] "I'd nest that one big, huge comp in the smaller comps, taking great care to position it properly for each movie you have to make."

This actually quadruples the rendering required for anything that's not cached, because the entire comp will be rendered anew and then cropped for each of the four little "window" renders.

Better to cache, or to use multiple output modules for the same render queue item (which will render the large image once and then crop it for each output module), or to render a single large movie/image sequence and use that as the source for these cut-ups.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Jim Zarbaugh
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 5, 2013 at 2:29:47 am

Walter- I posted my multiprocessing and memory info. If you get a chance, let me know what you think. I really just now enough about the technical side to do more harm than good.


Return to posts index

Dean Pickersgill
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 5, 2013 at 8:17:58 am

Jim, the other fella's you're talking to on this thread are much more enlightened than I on these matters so I'll bow out for the moment to reduce the noise :-) I'll chip in should I see something they miss (although I doubt that) but for now wish you success.


Return to posts index

Walter Soyka
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 6, 2013 at 1:59:16 am

[Dean Pickersgill] "Jim, the other fella's you're talking to on this thread are much more enlightened than I on these matters so I'll bow out for the moment to reduce the noise :-) I'll chip in should I see something they miss (although I doubt that) but for now wish you success."

Dean, the more the merrier! Please do feel free to contribute.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Walter Soyka
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 6, 2013 at 1:57:56 am

[Jim Zarbaugh] "Walter- I posted my multiprocessing and memory info. If you get a chance, let me know what you think. I really just now enough about the technical side to do more harm than good."

Jim, I generally recommend leaving one quarter of your total RAM and half your total CPUs for other applications. You should also consider lowering your RAM allocation per background CPU. Your comp is large, but you might be able to get away with 3 GB per background CPU.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Roland R. Kahlenberg
Re: Massive Render Strain and a Need to Fix That NOW.
on Dec 4, 2013 at 5:21:57 pm

There are AE-centric render farms for hire if you want to go that route. For your rehearsal, look into delivering a low rez version coupled with duplicates instead of real multiple instances.

HTH
RoRK

Intensive mocha & AE Training in Singapore and Other Dangerous Locations

Imagineer Systems (mocha) Certified Instructor
& Adobe After Effects CS6 ACE/ACI


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]