AE & RAM, the definitive guide
I have seen a recurring theme on the COW about confusion in terms of AE's use of RAM. I will attempt to explain a couple of issues regarding how much RAM, 32 bit vs. 64 bit, ECC vs. Non-ECC, DDR vs. DDR2, and OS constraints. (will also try and make this as plain language as possible, leaving out as much geek speak as possible)
Question 1. How Much RAM can AE use?
Short Answer: ALOT! AE is what I would call a RAM beast. But there are limitations....
Detailed Answer: Win XP - 32 bit:
AE is a 32 bit application. A 32 bit application can only address a maximum of 4GB of RAM. Hard Stop. No 32 bit application can address more than this.
That said, Win XP 32 bit (Pro or Home) doesn't allow 32 bit applications to access 4GB of RAM. The reason for this is, Win XP itself is a 32 bit application. It reserves a quantity of RAM just for itself, and one of the ways it does this is to limit ALL processes to fight over the same maximum amount of 2GB of RAM. What this means is, if you have five 32 bit applications open at the same time, they will all fight over the same 2GB's of RAM irrespective of how much RAM you have.
This is also why Win XP 32bit will also not see any more than 4GB of RAM, no matter how much RAM is "really" there. In fact, adding more than 4GB of RAM on Win XP 32bit can cause the OS to get a little lost in the extra RAM space, and become unstable.
Therefore - Do not add more than 4GB of Physical RAM if you are running Win Xp 32bit (Pro or Home)! Just becuase you have the RAM slots, does not mean that Win 32bit will play nice with it!
AE will attempt to use all 2GB available to it if possible. This is why AE allows you to set RAM usage in the preferences. If you leave the default preferences the way they are, it will max your system. I would recommend not running ANYTHING other than AE when using the defaults. Lower the Max RAM and Image Cache sizes if you want to use other apps at the same time as AE.
One last note - some users will notice that not all physical RAM installed on a workstation is "seen" by windows. Right click on "My Computer" and show properties. It is common to see say only 2.75GB even though 4GB is installed. This is called "The Black Hole". The reason for this is, PCI-Express. The BIOS of your workstation will take physical RAM away from windows to use as a buffer zone for the data transfer to PCI-Express devices, regardless of whether you have PCI-Express devices or not. The amount the "black hole" takes is relevant to the amount of RAM installed - the more RAM, the bigger the black hole.
Therefore, if you have PCI-Express on your workstation motherboard, you are very likely going to see the "Black Hole".
Detailed Answer: Mac OS X
Mac OS X 10.4 is technically both a 32 and 64 bit OS, allowing 64bit applications direct access to the additional memory. This means that if AE were 64 bit (which it is not) it would not be limited to the 4GB of RAM a 32 bit application can address. For the true geek - a 64 bit application is able to address 18 Exobytes of memory....(kilo, mega, Giga, Tera, Peta, Exo...)
Mac OS X is not constrained by memory limits in the same way that Win XP 32bit is. You can add say 16GB of RAM and OS X will see all this RAM quite nicely.
The benefit to AE users is that although AE cannot see more than 2GB of RAM on OS X, each application process is not fighting over the same 2GB. This means OS X can handle multiple applications that use extensive amounts of RAM more cleanly, as they all get there own memory space (why alot of RAM on a Mac is a good thing!).
The caveat here is, AE 6.5 is what's called a CFM application. This just means its heritage (and much of its core code) is still in "Classic". A CFM applications cannot address more than 2GB of RAM due to its dependence on running through emulation (via a thing in OS X called "Carbon"). A Mach-O 32 bit application is a true native OS X application, and can address more than 2GB of RAM in a single process. A Mach-O 32 bit application (which uses large addressing) would be able to address a maximum of 3GB of RAM in its own private memory segment.
Detailed Answer: Large Addressing...
AE is what is called a "Large Address Aware" 32 bit application. This means that the gifted folks at Adobe realized AE's hunger for RAM, and put in a little piece of code asking the OS for special permission to get more RAM. However, you have to make a system modification (on Win XP 32bit) for AE to "ask permission".
I won't go into details about this (as we don't recommend it), but for the adventurer, simply do a google search on..."/3gb switch".
With the "3GB switch" turned on, AE would therefore access a maximum of 3GB of RAM due to being "Large Address Aware".
Large Addressing does come into play when you use....
Detailed Answer: Win XP 64 bit
Windows XP 64bit Edition has none of the RAM constraints (with the exception of the PCI-E Black Hole) of Windows XP 32bit. This means all 32 bit application processes get their own memory segments (they won't fight over the same 2GB). This means that Win XP 64 bit can see all RAM installed (minus the black hole). There is also a really nice little widget for "Large Address Aware" Applications. AE 6.5 can see all 4GB!
The only problem we have found with XP 64 bit so far is finding drivers at installation time. Once over that hurdle, stack the system with as much RAM as you can muster, and this thing is a powerhouse. I would say the same on the Mac if AE were not a CFM application (see above), but being CFM is a big limitation. The Win version of AE is limited by this problem.
Also the number of chips from Intel and AMD that support 64 bit is growing daily. You may be surprised to find out that the dual-core Pentium D you have is capable of running the Win XP 64 bit OS.
Question #2: What is the difference between ECC and NON-ECC RAM?
Short Answer: AE users do not need ECC RAM - its not worth the price.
Detailed Answer: ECC RAM simply means Error Checking and Correcting. This means that all data going into RAM is double checked to be valid and not "lost". This is really important when you have millions of quick calculations putting small chunks of data in and out of RAM, like say running a webserver providing an E-Commerce system.
When trying to Render and work with AE, the Image Cache tends to be rather large and contiguous. This means it is less likely to get "Lost". The analogy I like is...when you have to try and find a needle in a haystack, ECC is great. When you need to find say country within a continent, thats not so hard and ECC is not necessary. AE is more like the latter.
In fact, ECC's continuous organization or memory can impact performance, as it is checking everything. I just don't feel that this overhead in terms of using AE is worth it.
Question #3: What is the difference between DDR and DDR2?
Detailed Answer: DDR2 uses more bus pathways to pass data back and forth to the CPU. Think of this analogy...DDR is like a 2 lane highway, whereas DDR2 is more like a 4 lane highway. A 4 lane highway can accomodate alot more traffic! There several technical differences between the two - but for simplicity, I will just stick to this description.
There you have it - RAM, fun for the whole family! (I think I will attempt war and peace next...;P
GridIron Software Inc.
Thanks for that post, Steve. Very informative!
Steve, thank you for this review. Really helpful.
I think there's a typo in "The Win version of AE is limited by this problem", right...? I suppose it's NOT limited.
OOPs - good catch. Wrote the whole thing here - and didn't catch that one. Yes AE is NOT limited by the CFM problem in windows...
GridIron Software Inc.
Great Post!! Thanks a million!!
Really good post, very good.
Great stuff Steve, thanks for being an active member on this forum. It does add a very valuable dimension to this community to have developers like you on board :-)
Oh... and it would be great to wrap this text into an article and put it in the COW database so we could easily refer to it.
Forum COWmunity leader for:
ADOBE AFTER EFFECTS
MAGIC BULLET SUITE
INDIE FILM & DOCUMENTARY
Thanks - always happy to help.
In terms of adding to the database - could you point me toward a "howto" for that? Or is that something the site moderators do?
GridIron Software Inc.
Hi Barend and Steve,
We've been following this thread and definitely wanted to turn this into an article for the Cow Library. We will ask that you email us, Steve, at email@example.com so that we follow-up on this when we get back from a road trip we are on today in Hollywood. (We're staying at the Grafton Hotel on Sunset and are feeling very Bogart and Bacall this morning, as this hotel has such a wonderful sense of film history about it. Very historic and filled with remembrances of Hollywood's early film history.)
Thanks to you both. Barend it was nice to hear you on the recent podcast you recorded with Aharon -- and which will premiere shortly. Thank you for being our guest and for being a part of the Cow. Hugs to the family.
And thank you, Steve, for bringing an insightful dimension to the board here. Your posts and contributions are most appreciated.
The best always,
Ron & Kathlyn Lindeboom
who are feeling very Bogie and Bacallish today ;o)
What a timely article. I just purchased a new Dual Opteron dual core system with 4 gig for ram and had been wondering why it was showing only 3 gig in the system properties. Now I know. I had noticed, in the BIOS, during start-up that it lists 4 gig of ram, but right below that it says something about 1 gig or ram. That must be the 1 gig I'm losing to the Black Hole you mentioned. Thanks again for a great article.
I have read this thread, and am still confused about AE RAM usage on OSX.
I have a Quad G5 with 2 gigs of RAM, OSX 10.4.8, AE Pro 7.01. I also use a little utility called "Menu Meters" that displays various things in the top menu bar (like CPU and RAM usage).
When I render in AE, Menu Meters usually tells me that I'm using about 500-600 megs of RAM, and that about 1500 megs of RAM are in the "Free" column. My processors are all at about 20-45% usage.
If AE can use 2 gigs of RAM, why is it only using 500 megs? Does this mean that adding more RAM won't change anything?
If I understand it correctly, AE only uses the RAM it needs. If you're concerned about processor usage to speed up renders, look into Nucleo Pro by Gridiron software.
Will AfterFX CS3 (or some future) have a 64-bit option, like 3dsMax9 does?
This is becoming a huge problem for us as we work with 5,000x1,800 pixel frames pretty often these days.
Can anyone recommend another compositing app which has a similar level of scriptablilty?
Art Director, Animation
What kind of scripting are you doing? We've had some killer problems with large frames and running out of RAM, taken to rendering image sequences rather than our usual quicktimes and using the background render script as it seems to get further without crashing, still it does crash, sequences are great though as all I have to do is delete any corrupt frames and AE will automatically fill in the blanks, no wasted time setting the render area or patching movies together.
I'm trying NUKE right now, can't say yet how the scripting is but the projects themselves are just scripts that seem easy enough to pick apart and edit (I'm not saying I can do that yet but looking at them in a text editor they seem pretty straightforward), there's no way I'm switching to Motion but supposedly it's projects are similarly editable, it would be cool if Adobe went that way too, the idea of being able to open a buggy project in a text editor and copy-pasting into a new file something I want to use seems really nice. Shake is officially dead and Combustion seems to be going nowhere fast so it's AE, Nuke and Fusion that I'm keeping an eye on right now, Nuke and Fusion now run on Mac, PC and Linux too which is great too.
Don't know if this has been any use to you, seems like a lot of people must be hitting the ram barrier too, interested in what scripting you're doing too.
We do quite a bit of expressions driven comps, and layer based pseudo particle systems. Quicky Custom tools running jsx - a la Aenhancers.com. 3dsMax scene imports. Some C# AeRender.exe farm automation (unfortunately the datagridview control sucks on XP 64-bit).
Since most of what we do is more "motion graphics" with compositing rather than strict compositing, Combustion seemed like our only alternative. We just junked our Smoke for a DS nitrous(bad choice IHMO) so Flame is not an option. Nuke looks pretty cool, but seems like it is more of a strictly compositing tool.
Running AeRender.exe for large frames seems to crash a lot less often. Also if you have lots of RAM you can start multiple versions and peg all your cores.
Art Director, Animation
Thanks a lot,man.I got 2*Xeon5160 and 4GB FBD-ECC RAM.And a damn Raid0.It really Brought a lot of problem to me.I rather to buy something else like DDR2-800,but it's too late......
Thak you anyway.
Thats a fine article.. Which has lead me to a question.
I use AE in windows and Mac.
The Mac workstation is equiped with 5GB RAM. When I set RAM to under 50% usage from the preferences i get the maximum duration in the ram preview. If I set it to 70 or 80 or 90 % its rendering less than it had rendered with 40 or 30 % of usage. After a lot of tweaking around i came to the conclusion that it can handle just fine only the 2,5gb of ram, and when i try to make it use more it is just hopeless.
I would really like to hear what it is you think goes wrong in the process.
thanks a lot,
and well done for the article.
Hello....I need help
I have a quadcore with 8 gigabytes of RAM w Windows 7 to 64 bits.
I can use to After Effects CS4 and Premiere Pro CS4 all RAM available or do I have accontettarmi of 4 giga.
Is RAM the reason only 60% of my video was saved to an avi file with Ram Preview?
I worked VERY hard on this video, do you know how I can save the whole video?
My operating system is Windows XP Professional 2002, Service Pack 3.