FORUMS: list search recent posts

Disk Cache & Separate Drives

COW Forums : Adobe After Effects

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Kevin Han
Disk Cache & Separate Drives
on Jan 10, 2019 at 9:52:40 pm
Last Edited By Kevin Han on Jan 10, 2019 at 9:57:55 pm

"For improved performance, choose a disk cache folder on a fast hard drive or SSD separate from your footage, and allocate as much space as possible."



Hi,



I'm currently running a RAID 0 with 2, Samsung 970 PRO M.2 drives (full desc. below). Now I'm a bit worried that my workstation is not optimal for After Effects, considering all my video files, my OS + AE application + Disk Cache are all going to this one drive (since RAID 0 combines both of my M.2 into one drive making it a lot faster, but less stable). Also running 64GB DDR4 RAM @ 3400 MHz.



(SAMSUNG 970 PRO M.2 2280 512GB PCIe Gen3. X4, NVMe 1.3 64L V-NAND 2-bit MLC Internal Solid State Drive (SSD) MZ-V7P512BW)



Here's a comment by 2 different individuals regarding this:



The rules are different with SSD drives versus mechanical HDD's. SSD drives have a latency in microseconds where as mechanical drives have a latency in milliseconds. This means SSD drives can read and write far more files in a given time segment than mechanical drives. If you have an SSD then cache and media files can go there without issue. You can also include cache files easily with an OS drive on an SSD. Mechanical drives are where the separate drive philosophy really developed from. Although you can run media and cache from the same drive provided the drives and interface ie e-sata, USB3 or TB are fast enough. Firewire and USb2 would not be

Eric-ADK

Tech Manager





You don't want the same drive/bus being used for varying read and write operations at the same time. That is a useful principle. But having cache files on the same drive as the OS and applications is in alignment with that principle; the OS is mostly read from the disk at boot, the applications are mostly read from the disk at application start, and the cache files are read and written during program run time. There is very little overlap. What you do _not_ want to do is have cache files on the same disk/bus as footage files or exported files, since those assets are being used at the same time.



Todd Kopriva, Adobe Systems Incorporated

After Effects quality engineering





This comment however was back in 2013. I'm currently running a high end M.2 SSD that reads at 3550MB/s and writes at 3450MB/s which is incredibly fast.



How much performance am I losing by having a setup where I'm running everyone on 1, RAID 0 drive using 2 M.2 SSD's? I also have a NAS HDD in order to store/archive all of my completed videos, etcetcra.



Thank you.


Return to posts index

Walter Soyka
Re: Disk Cache & Separate Drives
on Jan 10, 2019 at 10:35:18 pm

[Kevin Han] "How much performance am I losing by having a setup where I'm running everyone on 1, RAID 0 drive using 2 M.2 SSD's?"

It's almost certainly negligible (unless you are doing very little computation with a whole lot of footage involved).

You can watch a performance monitor to see if the disk utilization runs extremely high or the I/O queue gets long -- either would indicate a disk bottleneck.

Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive [twitter]   |   RenderBreak [blog]   |   Profile [LinkedIn]


Return to posts index

Kevin Han
Re: Disk Cache & Separate Drives
on Jan 10, 2019 at 11:42:32 pm

Thank you for the assurance! I was beginning to worry that I might have to format and re setup my computer from scratch...

I'm working on 5-10 second compositions but a lot of motion graphics, layers, effects, and AI files. I'm pretty new to AE and so I don't get what you mean by doing "very little computation with a whole lot of footage involved." Could you give me a scenario as to what this may look like? Not sure if I fall into this category lol.

Also, could you recommend me a performance monitor software/app where I am able to monitor my disk usage?

Thank you Walter.


Return to posts index


Walter Soyka
Re: Disk Cache & Separate Drives
on Jan 11, 2019 at 3:30:47 pm

[Kevin Han] "I'm working on 5-10 second compositions but a lot of motion graphics, layers, effects, and AI files. I'm pretty new to AE and so I don't get what you mean by doing "very little computation with a whole lot of footage involved." Could you give me a scenario as to what this may look like? Not sure if I fall into this category lol.
"


You are most likely to be CPU-bound. You don't have a lot of footage you need to read off disk. If you were doing the Brady Bunch opening with 9 streams of 4K ProRes footage, you'd be hitting the disk hard every single frame.


[Kevin Han] "Also, could you recommend me a performance monitor software/app where I am able to monitor my disk usage?"

On Windows, check out the Performance tab in the Task Manager (Ctrl+Shift+Esc). From there, you can open Resource Monitor for more details.

On macOS, try Activity Monitor.

Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive [twitter]   |   RenderBreak [blog]   |   Profile [LinkedIn]


Return to posts index

Kevin Han
Re: Disk Cache & Separate Drives
on Jan 11, 2019 at 6:19:47 pm

Thank you,

I did a "test" to see how my disk usage was doing. Upon starting AE, I get about 95% usage, then after loading the program it goes back down into the 0-3%. Then I'll get about a 15-20% spike when opening the AEP file (quite large). During all this my CPU spikes to 10-15%. and Memory is at a steady 15% total or 5.5GB RAM usage just for AE (I'm guessing the big AEP file and all of the Plugins I use for AE causes this). While rendering out a scene, my disk usage might hang around 7 to 15% for disk usage. CPU is still at around 15-17% while memory has gone up slightly to 18-20% or 6.5-8.5 GB for AE.

But I guess what I don't understand is MOST of the time, my Disk Usage is at 0-1% when previewing a scene (which is contrary to what I said above)... and then it'll all of a sudden spike to 50-60% for a few seconds not before going back down to 0-1%... Or sometimes even a disk usage of about 95-100%..? Might happen for about 5 or more seconds. It seems the Disk Usage happens in spikes and only for about a few seconds all while previewing out a scene.

More than anything, I feel like my RAM is at the highest %, then CPU, then Disk Usage, when it comes to previewing scenes during editing workflow. But I can't shake my head around this inconsistency while previewing a scene.. Sometimes it's at a steady 7-15%. Most times it's chilling at 0-1%. Then at random times it'll go anywhere from 50-90%.

Could you summarize this situation? I feel like my disk usage isn't the problem here but I'm a bit worried when it spikes that high...

Thank you.


Return to posts index

Walter Soyka
Re: Disk Cache & Separate Drives
on Jan 11, 2019 at 7:29:41 pm

[Kevin Han] "But I guess what I don't understand is MOST of the time, my Disk Usage is at 0-1% when previewing a scene (which is contrary to what I said above)... and then it'll all of a sudden spike to 50-60% for a few seconds not before going back down to 0-1%... Or sometimes even a disk usage of about 95-100%..? Might happen for about 5 or more seconds. It seems the Disk Usage happens in spikes and only for about a few seconds all while previewing out a scene."

I suspect that Ae is moving cached frames from disk to RAM in bursts for playback.

Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive [twitter]   |   RenderBreak [blog]   |   Profile [LinkedIn]


Return to posts index


Kevin Han
Re: Disk Cache & Separate Drives
on Jan 11, 2019 at 7:30:52 pm

I see. Thanks for all the help!!!


Return to posts index

Max Haller
Re: Disk Cache & Separate Drives
on Jan 23, 2019 at 5:55:54 pm

Hey I agree with everything Walter said. However, one factor that could be interesting to test is how much the speed of your ram would affect preview performance. I don't think it would make a huge difference but I really don't know and haven't come across any benchmarks for vfx work. I just thought of this because I recently bought new ram and I had to manually set it to it's advertised speed to get the full benefit.

You can change your Ram Clock speed in your bios menu. A fair amount of motherboards default your speed to like 2133mhz. If you paid the premium for the 3400mhz you should check to see if it's running properly. There have been benchmarks showing that in RAM intensive games, clock speed can make a considerable difference in performance, (10-20%). After effects is certainly ram heavy so it might have similar benefits.

It's kind of complicated to change, but checking it is easy enough. When your computer is booting go to the bios menu usually pressing f2 or delete and you will have to navigate the menus to a tab that says something like overclocking or performance. There you can check your cpu clock speed and it should have info about your ram as well. Don't change anything without researching it first. These instructions will vary based on the parts in the computer so sorry they're a little vague. I don't think it will make a huge difference but I'd be interested in your results if you do test it!


Return to posts index

Kevin Han
Re: Disk Cache & Separate Drives
on Jan 24, 2019 at 4:18:10 pm

It’s already over locked to the mentioned amount, my ram goes up to 3733 but I can’t achieve those speeds without my computer not booting up lol :(


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2019 CreativeCOW.net All Rights Reserved
[TOP]