FORUMS: list search recent posts

Copying Media and File Storage

COW Forums : Apple Final Cut Pro X

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Kasey Gay
Copying Media and File Storage
on Aug 2, 2017 at 1:09:18 am

Hi team,

I apologize for posting something so obvious, but I ran a few searches and feel like my search terms are so vague that I can't find answers to the basic questions.

1. Can you shine some light on where to store media versus storing my libraries?

I'm currently working off of a 500 GB internal SSD. I shoot a lot of real estate videos in 4k, and my understanding is that if I want to edit fluidly, I need to have my projects on my SSD. But do I need to keep my media on the SSD as well? Currently, moving whole projects+media off a USB 3 drive, onto the SSD takes hours of transfer time.

Does the media need to come with the library to retain a good editing experience?
Or am I just needing to invest in a large brick of USBc / Thunderbolt memory to make sense of all this.

2. Can anyone help me understand where my space on the SSD is slowly disappearing to?
I use my iMac solely as a work station, but my SSD is slowly filling up even though I thought I was being hygienic about moving things on and off in their entirety and I've apparently still got some learning to do about MAC file structures and what is lingering on my SSD.

Again, sorry for asking basic questions but I feel like I should understand this by now and I can't find the simple answers.

Thanks for the time,
Kasey


Return to posts index

Jeff Kirkland
Re: Copying Media and File Storage
on Aug 2, 2017 at 1:58:05 am

You can choose where the media is stored under the project properties pane. Click on a project and look at the info panel - the first option should be "Storage locations".

if you have final cut libraries on your internal drive and you're storing everything inside the libraries then it's possible you're losing space to render files, optimised media, etc. Use the storage locations option to move those you your external drive as well.

----
Jeff Kirkland | Video Producer & Cinematographer
Hobart, Tasmania | Twitter: @jeffkirkland


Return to posts index

Kasey Gay
Re: Copying Media and File Storage
on Aug 3, 2017 at 2:32:23 am

Thanks for that Jeff. But could I dig a little deeper?

Is it possible to have a fluid editing experience when you get your media files on an external USB3 drive? What is the deciding factor in editing fluidity? Does only your library need to be on your internal SSD, or does the library and media files both need to be ported in and out.

Thanks!


Return to posts index


Jeff Kirkland
Re: Copying Media and File Storage
on Aug 3, 2017 at 3:23:01 am

How fluid your experience will be depends on 1) your hard drive bandwidth and 2) your GPU. A USB3 drive will be ok for small projects in HD but I'd expect to to struggle for anything with multiple storylines, high quality audio, etc - USB as a technology doesn't tend to be great at the type of sustained throughput that video projects need.

I personally keep the library and the media on my external drive (a two drive Thunderbolt RAID 0 enclosure) and don't generally have any issues with bandwidth. I have started keeping my cache and render files on a second external drive and to me FCPX seems to have fewer slow downs but I don't have any real evidence it helps but logic says that adds another spindle to the mix so must be doing something.

The GPU is going to be key to fast renders and smooth application of effects as most are GPU accelerated. The GPU is probably more important than the CPU and RAM in terms of speed - although RAM plays a part too - the more the better. I'd say 8gb of RAM is really the minimum for FCPX and 16gb would be a preference. My iMac with 32gb of RAM always seems a little smoother to use than my 2013 Mac Pro with only 8gb RAM.

Well that's my 2c worth anyway but I'm not really a tech guy.

----
Jeff Kirkland | Video Producer & Cinematographer
Hobart, Tasmania | Twitter: @jeffkirkland


Return to posts index

Joe Marler
Re: Copying Media and File Storage
on Aug 3, 2017 at 3:34:45 am

Your 4k material is probably H264 which is compressed and requires a fairly low IO rate, say around 12 megabytes per sec for 1x playback. In general with 4k H264 the main factor in editing smoothness (excluding effects) is CPU not IO. If you want smoother editing you can transcode to proxy.

How much IO goes to the library vs media files depends on whether you are using internal managed media, unmanaged media or combination, and whether you're using proxy. The proxy files by default are inside the library.

There are many types and speeds of USB hard drives ranging from very fast AC-powered RAID arrays to much slower 5400 rpm bus-powered portable drives.

A good tool to find and safely delete non-essential FCPX files is Final Cut Library Manager: https://www.arcticwhiteness.com/finalcutlibrarymanager/

At the file level OmniDiskSweeper is good: https://www.omnigroup.com/more


Return to posts index

Craig Alan
Re: Copying Media and File Storage
on Aug 6, 2017 at 4:43:45 am

Try to keep your system drive as clean as possible.

Raid 0s with thunderbolt connections are the fastest.
I use 7200rpm sata III drives
Raid 5 gives you less storage, slightly slower speeds but if one drive fails you will be backed up within the raid.
And then be able to swap out the dead drive. A good sign also that it might be time to buy fresh drives.
The drives are usually good for 3-4 years. wouldn't trust it too much longer.
You need to back up all drives in your system or sooner or later you will loose your media.
softraid is a program that monitors all drives and keeps an eye on them in the background.
It also sets up soft raids which Apple used to do but not now. And softraid's soft raid is competitive with hard raids.
What I do is collect individual drives and a drive dock for backups.
B&H # OWTB2U3DKR2
thunderbolt dock
B&H # HG0S04007
6TB drives
label them and store them
B&H # NEHDSTORA35K (B&H Kit)
hard to see but thats 4 x 2cases (8 cases) in stackable cases.
I use CCC cloner for backups
https://sites.fastspring.com/bombich/product/ccc
I create folders on my raids holding my projects.
And on the raid I create master folders for collections of projects.
I keep these master folders under 80% of 6TB.
I keep the raid under 80% of its capacity. I use a 4 drive raids with 6Tb drives.
Then if I work on a new project or update one, I back up the folder it's in onto one of the
6TB backup drives.
So for example Raid: Video 4-6> (the forth drive is for redundancy cause I use Raid 5)> folder Video 4.
So the project is in folder Vid 4 which is no bigger than (80% 6TB = 4.8 terabytes) is backed up to individual drive
VID 4 backup.
When you are done with a project you can delete it from the raid after you have yet another back up.
Now ion you never plan to edit it again then erase it all except the exported video.
In Final CUT
select the library the project is in.
In the Inspector>storage locations>modify settings.
You'll see locations for media, cache/render, FC backups.
FC backups are very small files which can be stored in a folder on your system drive or anywhere.
Some folks keep all the others in the library, some don't.
I create a new folder on my raid and in that have a folders for media card backups, and the three folders media, cache/render, FC backups. I leave motion content alone cause I don't work in motion and if I were to I'd look into to what motion people recommend.
In the modify settings I select the folders above to the correct folder.
Now this is for setting ups new project.
If everything is in the library then use the consolidate if you want to move things around.
To make it simple just leave everything in the library on the raid except I would have the FC back up somewhere else
and media card copies somewhere else.
Without the original media and your backup which is automatic backups of the database portion of Final Cut Pro libraries you would be out of luck. The library can get corrupted. rare but it happens.
with those two things you can save your project.
You know your needs based on how you use your media and how much room these projects are taking.
So choose your drives and raid size accordingly. regardless back up all including your system drive.
Use CCC. way better than Apple's auto back up though you might want to do both. If your system drive dies or gets corrupted then a CCC copy of system drive with have you not miss a beat. exactly as you left off. same with FCP X projects.

Now all that said, there are editors here that deal with way more media than I do and everyone must address the needs they have. But having backups and media being stored somewhere other than your system drive is pretty universal.

Imacs (i7); Canon 5D Mark III/70D, Panasonic HPX250P, FCP X 10.3, teach video production in L.A.


Return to posts index


Kasey Gay
Re: Copying Media and File Storage
on Sep 1, 2017 at 12:49:14 am

Hey Craig,

I really appreciate all the effort you put into that response, but my head is a little crushed right now. I respect your knowledge base but am a little overwhelmed.

Can you help me decipher this? As much as I would like to be someday, I don't think I'm at your level. Most of my income right now is coming from real estate videos and small projects. I'd like to be getting this stuff out quicker. I'm losing a huge amount of time just from slow responsiveness from my computer.

What I'd really like to understand is these 2 things.

1. What $500 or less purchase can I make to get better performance from my shiny new high powered Mac.
2. What are the basic do's-and-don't of media and library storage if I want FCPX to sing for me?
As in, can I 'LEAVE FILES IN PLACE' and just move the LIBRARY onto my hard disk and expect decent performance? Or do I need to move EVERYTHING into one location.

I know I'm super low level. But I'm not at a place where I need to plan for 10 years of storage and huge piles of backups. (Trust me, I'd like to be in that place, if you know anyone hiring *wink wink*). But I'm just a guy right now trying to figure out what steps to take where FCPX will stop bogging down so I can free up more of my time and have more time to learn and advance.

Thanks guys!

Omnisweeper and everything has been incredibly helpful.


Return to posts index

Craig Alan
Re: Copying Media and File Storage
on Sep 2, 2017 at 5:32:06 am

Two basic things that pretty much is universally agreed upon in the production community.
One: do not keep your media on your system drive.
The system drive is used for OS and Apps and small files.
External MEDIA DRIVES are used for media and most all FCP files except maybe the small FCP backup files
that can reproduce the edits of a project but contains none of the media.
Second basic universal understanding: hard drives fail, have backups of them.
Given that, below are one way, of complying:
Get one of these
https://eshop.macsales.com/item/OWC/TB2U3DKR2/
$260

Get two of these:
https://www.bhphotovideo.com/c/product/1322879-REG/hgst_0s04005_4tb_7200_de...
$268 for two 4TB drives.

Get something like one of these
https://www.bhphotovideo.com/c/product/1129120-REG/seagate_stea1000400_1tb_...
$55

get one of these: https://sites.fastspring.com/bombich/product/ccc5
$40.

Format the two 4 TB drives and the 1TB portable drive in the OWC dock using disc utility.
Google it if you are not sure how. I would call one of the 4TB drives MEDIA 1 and the other one MEDIA 1 BU

CLONE your system drive (Macintosh HD) to the portable 1TB drive. You can now boot any Mac from that drive
including a loaner if you need to leave your Mac for repair.

The thing about CCC clones is you can update the back up and it will only take as long as it needs for changes since the last back up - not start all over again. And if your Mac or app goes off line or crashes, CCC will pick up where it left off.
You can even use it to back up camera cards to your Media Drive.

CREATE a master folder called (for example) "FCP PROJECTS" on your system drive and move all your FCP X folders into that folder. (Each project should have their own subfolder in the "FCP PROJECTS" master folder.

Create a folder on MEDIA 1 called "VIDEO 1"
CLONE your "FCP PROJECTS" folder on your system drive to the "VIDEO 1" folder on the MEDIA 1 drive..

(my names are just an option call them what makes sense to you but the "1" is so if you ever have the need for additional drives then you can create a MEDIA 2 drive with a VIDEO 2 folder.

Create a folder on MEDIA 1 BU drive called VIDEO 1 BU

Clone the MEDIA 1 folder "VIDEO 1" to the "VIDEO 1 BU" folder on the MEDIA 1 BU drive.
If you can't figure out how to select a folder to be cloned to a folder - it will be cloned to- use CCC help.

Now that you have your MEDIA/FCP X files backed up - delete your "FCP projects" folder from your system drive.
Then empty the trash on your system drive.

Highlight your MAC HD under Devices in the finder.
Then click command I (thats capitol i not 1) and see how full your system drive is.
It should be no more than 80% full.
Otherwise it slows down.
Same with your MEDIA drives.

SO no more than 3.2 TB of used space on any of the 4TB drives.

(if 4TB is bigger than your needs than obviously you can get 1TB or 2TB drives but do not fill more than 80%)

As far as where to store the footage. If you are a beginner and want less work then just keep it all in the library
But copy your media drives to a different folder within the Project's folder as well as importing them into FCP X as "optimized". It pops up when you go to import just click to optimize. Let final cut do its thing.

As far as not needing to back up - not true. Sooner or later you will wish you did.

If after getting your system drive with plenty of free space and if things are still sluggish:
You have now have a cloned portable backup.
First clone your BU system drive HD
Boot from it and wipe your system drive clean and do a clean install of your system drive by downloading the latest OS 10 from the app store.
Call Apple care and have one of the care takers talk you through it.

If one of them is not very helpful then loose the connection and try again. Some of them are really helpful and some of them are paint by the numbers and don't know anything. Or google the steps and see if you follow.
Tell apple care you use professional apps and need your system working fast. That you want a clean system.

If you don't want to go it alone make an appoint at an apple store bring your computer and the 1TB self contained portable drive and give them both and tell them you want a clean install cause the Mac was sluggish and you use pro apps.

PS I am NOT an advanced editor or computer expert. Backing up your media and knowing how to organize media in FCP X is basic stuff you need to know. Check out ripple training and get their tutorial on media management.
and core training and you'll know more than I do until I have time to go through those as well. You are making a living doing this you might as well learn the basics. If they cover stuff that means nothing to your needs then skip that part.
And when anyone here uses a vocab you don't know, it will sound like rocket science. It isn't. Google it and you'll now know what they are talking about.
Not trying to be condescending. I've been there, post is not my favorite part of production, but you can't produce without it. And knowing it you can enhance your videos in many ways.

Imacs (i7); Canon 5D Mark III/70D, Panasonic HPX250P, FCP X 10.3, teach video production in L.A.


Return to posts index

Kasey Gay
Re: Copying Media and File Storage
on Sep 12, 2017 at 10:42:34 pm

Craig thanks so much for the breakdown man.
THat's a lot of info broken down into concise packages. I appreciate you taking the time to illustrate that for a novice.

Much appreciated and much learned.

-Kasey


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]