FORUMS: list search recent posts

Upper Limit Library File Size?

COW Forums : Apple Final Cut Pro X

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Grant Peacock
Upper Limit Library File Size?
on Oct 11, 2016 at 12:47:31 pm

My late 2011 MacBook Pro (8,3 updated with internal 1TB SSD & 16GB RAM) is choking every time I launch a library that contains just over 200GB in total material. I had split the library out in 2 approximate halves when I suspected that the size of this library might be the issue, but the issue persists each time I open again in FCP X.

This is the first time I have dealt with material that is beyond 1080p (the files were all exported out of Keynote prior to being handed off to me, and are @ 2560 x 1440 image specs). Has anyone experienced this issue before, and should I be focused on the total library size as the issue - or perhaps looking at converting the files to 1080p spec? (this would be a fine choice in this case, since delivery is to the web only).

I suspect that trimming the library will not help of itself, since I've already taken it down by 50% and not seen any improvement. Whatever the demand on the system resources, it's apparently hidden from the view of Activity Monitor. Looking there reveals that CPU activity is basically at idle (FCP X being the top user @ aprox 10%). The load is bad enough that editing is impossible and the cursor lags while typing.

I have all resources resident on the internal 1TB SSD, and it is currently 70% full (300GB free). Scanning the prior threads pertaining to library sizes shows me that some of us are dealing with libraries that are many times larger.

Grant.

Grant Peacock
Washington DC
GPI TV LLC

http://www.gpi.tv


Return to posts index

Noah Kadner
Re: Upper Limit Library File Size?
on Oct 11, 2016 at 1:33:40 pm

Hace you switched to proxy mode?

Noah

FCPWORKS - FCPX Workflow
FCP Exchange - FCPX Workshops
XinTwo - FCPX Training


Return to posts index

Grant Peacock
Library File Size - proxy file versions
on Oct 11, 2016 at 2:14:12 pm

Thanks Noah,

I appreciate the input on this.

I had not thought of doing that. I'm assuming that at this point (not ever having used proxy file versions before) I would need to go back to the ingest process, re-ingest source materials as proxy clips, delete the originally imported (large) files from my library, then relink existing edits to the newly created proxy files?

I've completed about 3 out of 20 projects here (each runs aprox 3 mins) so it might in fact be worth doing this proxy procedure, at this early stage. I have also been wondering if the other approach here is to first downconvert the large Keynote-originated files into 1080p versions, and THEN bring those into a fresh library, thereby bypassing anything that has a large file spec associated with it.

Grant.

Grant Peacock
Washington DC
GPI TV LLC

http://www.gpi.tv


Return to posts index


Bret Williams
Re: Library File Size - proxy file versions
on Oct 11, 2016 at 5:16:47 pm

Just select all your media in the browser, right click and choose to generate proxy media. Then you can swap between proxy and originals on the fly whenever you like.

_______________________________________________________________________
http://BretFX.com Plugins & Templates for FCP X Editors & Motion Graphics Artists


Return to posts index

Joe Marler
Re: Upper Limit Library File Size?
on Oct 11, 2016 at 5:08:10 pm

[Grant Peacock] "choking every time I launch a library that contains just over 200GB in total material. I had split the library out in 2 approximate halves when I suspected that the size of this library might be the issue, but the issue persists each time I open again in FCP X...CPU activity is basically at idle...editing is impossible and the cursor lags while typing."

By "choking", what is the exact behavior? Is it slow to open the library or slow to scroll down through the Event Browser in filmstrip mode, or won't scrub through the timeline quickly or the skimmer itself is halting and slow, or what?

Create a new project within that library and add a few small clips to it. Is that slow? If not then maybe some timeline effect in your main project is slowing things down. However this would normally manifest as high CPU, which you've already ruled out.

If you have background rendering enabled, try disabling that.

During these slowdowns, check your I/O using Activity Monitor. If bytes/sec or IO per sec are high, this could imply something is doing lots of background I/O. If they are both low and CPU is also low that's not a good sign. It could imply some interrupt or hardware problem. I would suggest unplugging all external devices and running Apple Hardware Test: https://support.apple.com/en-us/HT201257

Also try running Disk Utility repair option on your main drive.

If no errors, try to boot FCPX but keep all external devices disconnected -- USB hubs, Firewire devices, external hard drives, etc. Evaluate the performance like that and see if it changes. If it does, the slowdown might be caused by a malfunctioning peripheral device flooding the system with interrupts.

Yet another item I've sometimes seen is runaway or malfunctioning Spotlight index. I suggest try turning this off and/or rebuilding spotlight indexes: https://support.apple.com/en-us/HT201716

If you have any backup task running such as Time Machine, try temporarily disabling that.

My largest library is 6,718 clips comprising 138 hr of material, about half H264 1080p and half UHD 4k, 3.21 terabytes. That is on a 2015 top-spec Mac Pro with the media on a 16TB OWC Thunderbay 4. All content was imported with "leave files in place" with no proxy or optimized media generated. Performance is normally OK but sometimes the Event Browser can get bogged down generating the thumbnails.

After doing the selects and proceeding to the editing phase I often (but not always) create proxies for 4k. I could never create proxies (much less optimized media) for all the content -- that would take forever and would be too big.

If your SSD is 70% full you likely don't have much space to create proxy files for everything, since they are (very roughly) 60%-80% the size of the original media files due to the lower compression.


Return to posts index

Noah Kadner
Re: Upper Limit Library File Size?
on Oct 11, 2016 at 7:21:34 pm

I would just go proxies- it's very easy to toggle and you won't be overpowering your setup so much with all of that oversized resolution media.

Noah

FCPWORKS - FCPX Workflow
FCP Exchange - FCPX Workshops
XinTwo - FCPX Training


Return to posts index


Grant Peacock
Continued issue with project (orig 'library size' post)
on Oct 25, 2016 at 6:28:58 pm

Bret, Noah, Joe,

Thanks all for the input. I have been hesitant to add more to this post, until I could get a clearer sense of what was going on.

I don't know if I'll ever get to the bottom of this (system) behaviour. On the choking and machine slowdowns - with further digging, I realized that all the project materials, being sent over to me (from Keynote), were in aprox 2K. I had allowed the timeline to conform to that originally.

I've gone back and moved each project into a 1080p timeline, and also opened a new library for each individual edit. The machine is no longer choking, but it's unclear to me which of my simplifying actions would have had a significant impact. (The 2K resolution was completely over the top, as I am editing projects for web delivery).

Another strange issue - on these new 1080p timelines, I'm getting 2 odd things happening. Random audio drop outs in voice track, either with waveform representations still painted in, or not (both scenarios are happening). I'm also getting functioning audio playback at points on the timeline where the waveform depiction has vanished.

What is that all about? To make this even stranger, I can scrub back through a section of timeline where the audio just failed to playback, and on the next go, it may actually play. The dropouts are brief - just 1 - 2 seconds. I am starting to wonder if my 2011 MBP is just getting too old!

Grant.

Grant Peacock
Washington DC
GPI TV LLC

http://www.gpi.tv


Return to posts index

Greg Fulcher
Re: Continued issue with project (orig 'library size' post)
on Oct 28, 2016 at 12:38:34 am

Migrate your media to a fast external drive (Thunderbolt, at minimum). Only in an emergency would I ever consider storing footage on the same drive that runs the OS and applications.

To do so, copy the Library in question over to your external HDD, and then open it either by double-clicking on the copied library, or from within FCPX. Once you confirm everything is working fine, it's OK to delete the original Library from the internal drive. Personally, I'd also back it up to another drive, and use an app to continue your backup on a daily basis (I am a huge fan of ChronoSync).

I suspect you'll see great improvement in performance.

Good luck,
Greg

1013media | produce. edit. enable.
San Francisco, CA | http://www.1013media.com


Return to posts index

Joe Marler
Re: Continued issue with project (orig 'library size' post)
on Oct 28, 2016 at 1:59:36 pm

[Grant Peacock] "On the choking and machine slowdowns - with further digging, I realized that all the project materials, being sent over to me (from Keynote), were in aprox 2K. I had allowed the timeline to conform to that originally...I've gone back and moved each project into a 1080p timeline, and also opened a new library for each individual edit. The machine is no longer choking, but it's unclear to me which of my simplifying actions would have had a significant impact....Another strange issue - on these new 1080p timelines, I'm getting...Random audio drop outs in voice track, either with waveform representations still painted in, or not (both scenarios are happening). I'm also getting functioning audio playback at points on the timeline where the waveform depiction has vanished...I am starting to wonder if my 2011 MBP is just getting too old!"

I don't know the answer. My library is now up to 8,085 clips (181 hr of 1080p and 4K material), comprising 4.75 TB total size -- most in a single event. On my top-spec iMac 27 and using several Thunderbolt arrays, performance is mostly OK. This is on 10.2.3; I'd like to upgrade to 10.3 and test it but can't afford the risk.

The easiest step would be transcoding your content to proxy (not optimized) then flipping the Viewer mode to proxy. This can be done after import and it usually makes performance must faster. Just remember to change it back to optimized/original before the final export.

I looked up your 2011 MBP in the MacTracker app. If you have the top-spec 15" version it is not that slow. Of course new machines are faster -- GeekBench 4 numbers for the 2.4Ghz 2011 MBP are 3015 single-core and 8800 multi-core. This compares to my 2015 MBP which is 4367 single-core and 14469 multi-core. There might be a bigger difference in the GPUs.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]