ADOBE AFTER EFFECTS: Forum Expressions Tutorials Creative Cloud

TARGA in AE equals excessive render time

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Paul Emous
TARGA in AE equals excessive render time
on Nov 26, 2013 at 11:50:51 am

Let me state first that I am in this business for quite some time. I don't want to make it sound like this is my first rodeo.. :)

A familiar topic but I was hoping to get some help / advise on how to handle this.

Basically I have a big local client that is part of a big global company. Normally I provide the needed videos for this, but something global sends something - for branding purposes - that need to be adapted locally.

About 2 weeks ago I received a 200GB After Effects project. Now big projects don't scare me but as soon as I opened it and my iMac (i7 3.4GHz 16GB) started chocking on the project I got a heart attack. I don't even know where to start explaining....

Basically there are a couple of comps that need to be rendered and in these comps there are some pre-comps. In the precomps there are about 15 layers of TARGA sequences. I can't RAM preview it.

When I started digging I found a couple of comps that where using proxy files. Now those files wheren't delivered in the project (First question: If you collect footage, does it include the proxies and pre renders?)

Since I couldn't find the pre-rendered files I decided to render them myself. -> I got another heart attack. Currently I am pre-rendering a 750frame comp and after 5 hours i managed to render 180 frames. After Effects is struggling to respond (but its processing so that makes sense). In activity monitor I see that my CPU usage is 120-220% (AE), yet only one core is being fully used.

Question 2:
Is there a way to optimise the performance?

And lastly. Obviously I contacted the client, telling them there are items missing in the project (Pre-renders) and that I want to get them from the creator. They contacted the agency and said its not available and if they have to re-render that will ask a lot of money

How would any one handle this?

I really appreciate your inputs.

Regards,
Paul


Return to posts index

Walter Soyka
Re: TARGA in AE equals excessive render time
on Nov 26, 2013 at 6:48:54 pm

First, a general point: Targa files can be pretty big, especially if RLE compression was turned off. Without compression A 32 bpp (that is, 8bpc RGBA) TGA for a 1920x1080 frame is almost 8 MB. If you have to read 15 of those, that's 120 MB per frame. If you have, say, 4 precomps and each precomp has 15 layers, that's 480 MB per frame -- or 14 GB/s.

That means that your storage can very quickly become the bottleneck -- the computer simply can't read from the disk fast enough. What disk are you storing these TGAs on, and how fast is it?

If your storage is a bottleneck, you will likely find that you can improve performance by turning multiprocessing off. It's a little counter-intuitive, but allowing multiprocessing means multiplying the amount of storage I/O, and once you saturate your storage, multiprocessing can't help anyway.


[Paul Emous] "Basically there are a couple of comps that need to be rendered and in these comps there are some pre-comps. In the precomps there are about 15 layers of TARGA sequences. I can't RAM preview it."

Do you get an error message when you try?


[Paul Emous] "When I started digging I found a couple of comps that where using proxy files. Now those files wheren't delivered in the project (First question: If you collect footage, does it include the proxies and pre renders?)"

Pre-renders are always included with collect files. Pre-renders are regular footage items; they are not special. When you pre-render something, the rendered file is imported back into the project, and all usages of the pre-rendered comp are replaced with the file.

Proxies may be optionally included or excluded from collect files via the "Obey proxy settings" checkbox.


[Paul Emous] "Question 2: Is there a way to optimise the performance?"

There are numerous suggestions on the Improve performance [link] page, but I think the first thing we should address in troubleshooting this is your storage configuration.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Paul Emous
Re: TARGA in AE equals excessive render time
on Nov 26, 2013 at 7:54:57 pm

Hi Walter,

Thank you for your reply. Very much appreciated!

I very much agree with you that storage looks like the potential problem here. I was running this from a FW800 hard disk and then store the output on a different location. Unfortunately due to other projects I had to shift this nightmare over too a dedicated Mac Mini i5 with FW800 hard disks.

If I look at the hard disk I/O its not under very high load. Also it currently uses (only) 70~80% RAM

13h25m on the clock and I rendered 193 frame (out of 750).... What a nightmare.


Return to posts index


Walter Soyka
Re: TARGA in AE equals excessive render time
on Nov 26, 2013 at 8:40:37 pm

[Paul Emous] "If I look at the hard disk I/O its not under very high load."

How are you measuring load? It's not enough to consider bandwidth -- the disk queue length (number of unfulfilled requests to the disk) is important. I am not in front of a Mac now, but I think running "su iopending" in the Terminal will tell you the length of the disk queue.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Paul Emous
Re: TARGA in AE equals excessive render time
on Nov 26, 2013 at 10:54:22 pm

I have no idea how to read this...

2013 Nov 27 02:52:14, load: 1.95, disk_r: 680 KB, disk_w: 387048 KB

value ------------- Distribution ------------- count
< 0 | 0
0 |@@@@@@@ 3648
1 |@ 590
2 | 236
3 | 118
4 | 6
5 | 6
6 | 10
7 | 6
8 | 14
9 | 14
10 | 18
11 | 79
12 | 71
13 | 60
14 | 125
15 |@@ 756
16 |@@@@@@@@@@@@@@ 6987
17 |@@@@@@@@@@ 4952
18 |@@@@ 1860
19 | 142
20 | 26
21 | 19
22 | 61
23 | 20
24 | 84
25 | 12
26 | 10
27 | 12
28 | 18
29 | 12
30 | 26
31 | 0


Return to posts index

Walter Soyka
Re: TARGA in AE equals excessive render time
on Nov 27, 2013 at 3:16:52 am

iopending shows a histogram of the number of pending disk transactions over the period of time it samples. A pending disk transaction is any read/write request that has been made but not yet fulfilled by the disk.

Persistently high queue lengths over 1 indicate disk saturation. Your queue length is very often between 15-18.

Your render cannot go any faster because the system can't read frames from the disk any faster than it is. This can be improved by placing the footage on a faster disk -- internal SSD, miniSAS RAID, Thunderbolt SSD/RAID, etc.

Again, if multiprocessing is on, you should try turning it off as that will lower the number of simultaneous disk requests.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]