FORUMS: list search recent posts

Conforming offline to online version in FCP using XML Query

COW Forums : Apple Final Cut Pro Legacy

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
John Hopkins
Conforming offline to online version in FCP using XML Query
on Apr 19, 2014 at 11:45:24 pm

I am having real problems with 4 Glyph firewire drives daisy-chained and trying to work with the project. Crashing constantly. Just too many files and sequences to deal with. Works ok for view individual QT files but but not FCP with this size of project I have (100+ hours of footage). The only way I have been able to work with the project is to transfer all the proxy/xdcam offline files off the Glyphs to my RAIDS. Now I can work with them and not nearly the same problems with crashing, although there is still the odd crash but liveable if I back-up/save tons.

However, when I look at the proxy files on the four FW Glyph drives, there is need for re-organization. I'd like to organize it by the year things were shot and the month. Right now, I have lots of stuff on those Glyphy frustatingly organized by a since departed assistant editor, with folders mixed together with names like Sept._4_2011, 2012, 2013. The odd thought it seems here was to organized all being organized around the term "Sept." This just makes things hard to track down across proxy drives, which are now also duplicated folders of the same name on the RAIDS. I want to put all the 2012 stuff together, 2011 stuff together, all by month/year. That way its easy for me to find anything sequentially.

However, if I organize the drives of the RAIDS this way, is there going to be a problem re-link the files after I offline, with the high-res versions stored on other archived drives? If I export an XML file of the final cut (52 and 90 minute versions), will there be a problem re-conform my cut to from the xdcam, pro res proxy stuff, to the high res, ProRes 422 HQ material? Wouldn't the XML be just be searching and organizing based on the name of a particular clip and edit points within that clip? Or do my proxy version FW drive/duplicated on the RAID, have to be organized exactly as the archived masters with the same drive/folder namers fof the XML re-conform work from picture lock off the offline, to creating the online version with the high res files/footage?

Thanks for any help!!!


Return to posts index

Shane Ross
Re: Conforming offline to online version in FCP using XML Query
on Apr 21, 2014 at 5:52:25 pm

Do not reorganize a darn thing while you are in the middle of an edit. If you didn't organize it properly to begin with...then you'll need to leave it completely alone until you finish. Relinking in FCP is hit and miss. Don't do anything that will jeopardize the links now.

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index

John Hopkins
Re: Conforming offline to online version in FCP using XML Query
on Apr 21, 2014 at 7:56:14 pm
Last Edited By John Hopkins on Apr 21, 2014 at 8:10:36 pm

I have to get the footage off the Glyph drives. The project is crashing too often, no choice. No problem linking the footage, it's just time consuming, done. That works. It works better than the Glyph daisy chained 800 drives (which was a bad idea not of my design). However it is still crashing FCP with a project this big though not near as often. Still its unworkable. It takes 15 minutes to open the project and this zaps the creative workflow.

I've set up a new drive in my box which has a fresh copy of FCP Studio 3 and factory install of Snow Leopard, which is what is running the FCP now on my main system HD. I have tons of other programs and files on there which might conflict or suck up memory. I need to just edit without any variables from this to keep things clean and less crashable.

And yes, I need to keep the original drives (proxy versions) exactly like they were delivered to me on the Glyphs (though some shoots are spread across three drives!) Or I am not going to be able to conform the offline edit picture lock through XML to the online masters. Just have to organize that through FCP itself which is harmless. Anyhow, things have re-linked as per the project FCP file exactly, a few glitches, but fixable. The key is to be sure that the files on the RAIDS are named the same as each of the Glyphs. The project should conform to the masters with XML since the masters with the Pro Res 422 (HQ) perfectly match the badly organized Glyphs which were delivered with 5.7 terabytes of proxy footage in XDCAM-e and ProRes Light Format.

Any additional tips that might help would be welcomed!


Return to posts index


Shane Ross
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 7:24:59 am

[John Hopkins] "However it is still crashing FCP with a project this big though not near as often. Still its unworkable. It takes 15 minutes to open the project and this zaps the creative workflow."

Make a new project. Bring over only the latest cut, and your organized bins of footage. Save that. See if you can get it under 50MB.

[John Hopkins] " Or I am not going to be able to conform the offline edit picture lock through XML to the online masters. "

Conform how? In FCP or another edit system/online system? What footage are you working with now? Proxy...what kind of proxy? ProRes Proxy or the proxies XDCAM makes? How did you make those proxies? You mention relinking to the masters...but mention both XDCAM and ProRes HQ...which are they? How exactly did you bring in the footage, and start with the proxies when editing?

If you are conforming in FCP, there's no need for XML. You might simply be able to relink, but this all depends on how you brought in the footage and made the files are you working with now.

If you have 5.7TB of proxy footage...how much storage are the masters taking? where are those stored?

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index

John Hopkins
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 5:43:37 pm

Thanks Shane for your questions and help,

Conform how? In FCP or another edit system/online system?

Documentary production. Will be conforming to FCP.


What footage are you working with now? Proxy...what kind of proxy?

Footage is XDCAM-EX which is being edited with ProRes Light footage. The ProRes Lt footage was rendered from RED, Sony F3 ProRes (HQ), and Canon 5D footage camera originals.

ProRes Proxy or the proxies XDCAM makes?
We are editing with both these compressed formats together with Final Cut 7 (Studio 3).

How did you make those proxies?
They were rendered from the camera originals.

You mention relinking to the masters...but mention both XDCAM and ProRes HQ...which are they?
Both, and Canon 5D footage as well.

How exactly did you bring in the footage, and start with the proxies when editing?
The footage was organized and logged on a RAID in folders. Once the log was completed the footage was then rendered as proxies for everything but the XDCAM-EX footage. XDCAM-EX footage is already compressed and about the same bandwidth as ProRes Lt. Perhaps it would have made sense to render the XDCAM-EX footage as ProRes Lt also. The Post Coordinator deemed it unnecessary and was looking to save money on the assistant editor's time.

If you are conforming in FCP, there's no need for XML. You might simply be able to relink, but this all depends on how you brought in the footage and made the files are you working with now.

Good thought, I assumed it would be an XML if the online was going to migrate to AVID. But that's likely not going to happen at this point. Re-linking is likely the plan now. It depends on who does the online and what they are working with. It should re-link because the four external Firewire Glyph drives (which crashed the project constantly with bandwidth issues) I have here mirror the footage and file folders and exact logging structure on the camera original RAID (about 9 Tetrabytes of footage and it is located 250 miles from my editing suite). The proxy clones Glyphs I have here have everything down to 5.5 Terabytes composed of Pro Res Lt compressed versions of the same footage, along with XDCAM-EX which is the camera original footage for some of the shoots. We started shooting with XDCAM-EX and then later started shooting with ProRes 422 HQ and recording to a Mini KiPro and PIX240 on location. So re-linking to the masters should work if the file structures, folder names, and drive names stay the same, as the footage for each shot themselves are named identically as the camera logged them while shooting.

If you have 5.7TB of proxy footage...how much storage are the masters taking? where are those stored?

The Masters are stored on a RAID at another studio several hundred miles away. I should be able to send them the FCP Project file to conform later as you suggest. Masters are taking about 9+ Terabytes as per above.

What do you think?

The issue I am having now is that footage is crashing on the RAID I have here now, after I relinked the footage to a copy of the proxy footage on the RAIDS. It re-linked without too many issues.

To reduce conflicts and memory being sucked up by other applications or the Graphics card having to deal with tons of other files, I am installing a fresh copy of Snow Leopard 10.6.8 on a new formatted drive partition with a fresh factory copy of Final Cut Pro Studio 3 and will re-link all the files again. It worked fine once, should work again re-linking. Then I will see how that does as far as the frequency of crashes. I will also install 8 GB more RAM. I have 10 GB at the moment. The Project file is 82mb which I understand is problematic in FCP for projects this size. So my thought is to break it all down as 4 project files at about 20mb each and edit sequences on each with just the bins needed for those sequences. And then I was going to construct the final 90 minute assembly on a 5th master project file, using edited sequences from the other project files supporting the construction of this final assembly.

At least that is how I see doing this. But this is all somewhat unchartered territory for me. I know I have to stop the crashes to work and then conform to the masters from the offline. Any thoughts would be gratefully appreciated. Thanks for trying to assist.

My other thought is for the head production organization to send me the RAID over with all the masters on it and just transfer those to my RAIDS here and forget linking later. Does that make more sense at this point given all? I will have to boost the XDCAM-EX/Canon footage to Pro Res HQ (I believe that is already done)


Return to posts index

Shane Ross
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 6:31:06 pm

[John Hopkins] "How did you make those proxies?
They were rendered from the camera originals."


Rendered how? Using what method?

[John Hopkins] "I assumed it would be an XML if the online was going to migrate to AVID."

Avid doesn't do XML. EDL and AAF.

One way to make smaller project sizes is to have separate projects for your assets. One project for the sequences, one for the footage, another for the audio assets (music, SFX). The issue is that you cannot match back to the bins the footage comes from when you have separate projects.

I suggest more RAM, and breaking up the assets into different projects. See if that helps.

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index


John Hopkins
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 7:38:34 pm

Ok, thanks for the clarification on AVID, I have not edited with it yet, although I bought Media Composer 6 for $900 after scanning the FCP barcode on the box and sending that into a dealer to get this discount. This is the last project on FCP for me. I will have to learn AVID. Was thinking Premier Pro but AVID seems to be the way to go although the price for upgrades/hardware additions is worrisome.

"One way to make smaller project sizes is to have separate projects for your assets. One project for the sequences, one for the footage, another for the audio assets (music, SFX). The issue is that you cannot match back to the bins the footage comes from when you have separate projects."

So what you are saying is to create three new projects from the one I am working on now. First I will need to duplicate three times and then rename them as:

1) Sequences.

2) Footage.

3) Audio.

and then a (4) Master Sequence Project where all the sequences are joined together to create a ninety minute or three hours assembly to cut down?

Or do I cut the timeline of the Sequences Project with all the parts of the Sequences within that project?

I assume I would open all there projects at the same time, and copy and paste in building the main sequence?



"The issue is that you cannot match back to the bins the footage comes from when you have separate projects."

I am confused by this a bit. Are you saying if I split the project into three: sequences, footage, audio then I can match back to the bins of the footage in the proxy or master project.

I will get more RAM.

I assume that the Assistant Editor used Media Manager to make the proxy copies. There are Pro Res Lt versions of everything we shot except the XDCAM-EX footage we shot. This is compressed footage already as it came from the SXS slots which is supposed to be for offline, but our DP at the time wanted to lighten the rig to shoot from boats at sea and took the KiPro off and shot everything to the SXS cards. So that's why it's like this.

I really do appreciate the help. I've have not run into these issues with FCP with project file size and want to iron out the kinks so it does not crash as often and I can actually work on the thing in a creative fashion. I was warned by another editor FCP is famous for this issue.


Return to posts index

Shane Ross
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 7:48:44 pm

[John Hopkins] "and then a (4) Master Sequence Project where all the sequences are joined together to create a ninety minute or three hours assembly to cut down?

Or do I cut the timeline of the Sequences Project with all the parts of the Sequences within that project?"


One project for all sequences. Or if you want, a master project for master sequence. That works too. Same warning applies...no match back.

[John Hopkins] ""The issue is that you cannot match back to the bins the footage comes from when you have separate projects."

I am confused by this a bit. Are you saying if I split the project into three: sequences, footage, audio then I can match back to the bins of the footage in the proxy or master project."


If you cut footage into a sequence in the same project as your footage bins...if you MATCH BACK, you can match to the bin the footage came from. If you put the footage into a sequence in a different project, you cannot match back to the bin the footage came from. It'll just beep at you. Once you put the footage into a different project's sequence...you break the file path. It can get annoying.

[John Hopkins] "I assume that the Assistant Editor used Media Manager to make the proxy copies."

Don't assume...find out. Never assume. Knowing how these low res files were made means EVERYTHING when it comes time to online.

Shane
Little Frog Post
Read my blog, Little Frog in High Def


Return to posts index

John Hopkins
Re: Conforming offline to online version in FCP using XML Query
on Apr 22, 2014 at 7:58:58 pm

Ok, I've asked for specifics on how the proxies were made. What I understood was that they made a master footage RAID and logged it and locked down that structure. The Assistant then went to every folder and made proxies of the camera originals and offload those to four Glyph drives with the same names as the four main folder in the master footage on their RAID drive.

Is there any other additional info which might be required?

If I can't match back (link back is what I assume you mean) then I am going to have export and XML if I edit a sequences project, instead of re-linking?

Sounds like a logistical mess... guess there is no choice if I can't edit and the thing is crashing all the time.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]