Can I get FCPX to link to Proxies created by Redcine-X?
Hello, this is my first post.
I got a project consisting of Red footage. I imported the raw files into FCPX, made selects, and then created proxy files just for the clips I intended to use in the edit. That alone took 24 hours to convert the footage, and it would have taken over 48 hours had I converted the entire project.
While working on the rough cut, I decided I wanted to incorporate some more content that was on footage not yet proxied. However, I didn't want to have to stop working in FCPX and I wanted the files converted asap. Therefore, I used Redcine-X to convert about ten more clips at the same time as I continued working in FCPX.
The problem is, now I cannot get these Redcine-created proxy clips to be read by FCPX in proxy mode along w/ all the other proxy media. My guess is I have to go back to square one and have FCPX do the conversions, but just in case, is anyone aware of a workaround or some way to "trick" FCPX into accepting the Redcine-created Proxies in proxy mode?
** I have already tried moving the Red-created proxies into the Proxy Media folder, and it didn't work.
Why use Proxy mode?
Just use regular mode. Do the edit, then relink to R3D in the end using RCX.
You can then send that relinked FCPXML back to Final Cut Pro.
My computer can't handle the raw Red files ... it kept crashing.
I assume you have the latest Red plugin?
Even so, just use the ProRes files in FCPX, and don't use Proxy mode. Just use the regular full res mode.
I didn't create Regular ProRes files. I went straight from the native to proxy. Does it take less time to create regular ProRes?
It doesn't matter what codec files you made. If you made ProRes Proxy files, that's cool.
Just don't use the Proxy mode in FCPX. Use "Original/Optimized" mode and use the files you created from RCX to edit with.
You can reconnect to R3D later in the process.
I'm having similar issues. I want to use the proxy files that were created by the RED camera. They are .mov files and the codec is Linear PCM, Apple ProRes 422 LT, cn32. I COULD import these directly into FCPX, but I want FCPX to recognize these as the Proxy media. For some reason, FCPX only recognizes Proxy files that it makes by itself. So if I use the RED-camera-created Proxy files, then how do I link back to the original full RED media at export?
You could try simply relinking to the R3D files after you import the Red created Proxy files using FCPX.
Or you could try this
Or you could send an FCPXML to Resolve or RedCineX and relink there, but those will be more difficult to do.
Thank you Jeremy for these suggestions. The first one is the one I want to do. You wrote "simply relinking to the R3D files" but sadly I am not finding this so simple. The RED-created proxy file, a .mov file that is ProRes 422 LT, is viewed by FCPX on import as Original media. If I right click on the clip in either the browser or the timeline, there is no option to Relink. Under File menu it says Relink, but this OVERWRITES the proxy file directly in the browser and timeline which cannot be undone and then I no longer have RED-created proxy files to work with in the timeline. Plus, FCPX will batch reconnect all .mov files to .mov files but relinking .mov to R3D file would have to be manually attempted clip by clip, and on a timeline with 1000 clips that would be insane. So I'm stumped. I'll go and read this long article next.
I usually make Proxies though RCX (making sure to keep the same audio setting as R3D), and then I import that footage in to FCPX and just use the Original media method.
Then I select the clips I want to relink on the timeline, and relink to R3D, then export.
I can then relink to the ProRes movies for editing. It's pretty seamless.
I do not use the Proxy switch in FCPX.
[Jeremy Garchow] "You could try simply relinking to the R3D files after you import the Red created Proxy files using FCPX. Or you could try this" [proxy cheat using Finder aliases" method]
FCPX is not currently designed to use new proxies created outside the product. It's not a matter of those being a certain format. The SQL tables inside CurrentVersion.fcpevent have pointers to the symlinks inside the library, which themselves point to the proxy files. Besides those internal pointers, those tables also contain data about the file to prevent a mis-match. This includes pixel aspect ratio, audio config and timecode. The SQL tables also seem to contain Primary/Foreign keys which tie together the rows corresponding to a given proxy and media file.
However even if your constructed proxies match the config data exactly, they will not match the internal pointers or the inter-table relational links, so those will never work. In fact the SQL rows corresponding to proxies will not exist, since FCPX never created any proxies. The proxy and media pointers apparently each consist of two independent locators, the file inode to the proxy file plus some method of locating the symlink -- either another inode or a path pointer.
If the FCPX-created proxies (or media files) are relocated within the disk volume, renamed, or even if the symlinks inside the library are totally deleted, FCPX will successfully relink those and rebuild new symlinks based on an inode lookup. However that doesn't help for new proxies created by external methods.
The reason the above "proxy cheat" works is there is evidently an undocumented backup method of relinking *existing* proxy references inside the library to FCPX-created external proxies. That method is triggered by copying Finder aliases (which are different from symlinks) inside the library. That in turn causes FCPX to scan those, read the path information from the alias, and build new symlinks. The library proxy folder will be left with two sets of pointers to each proxy file, the aliases you dragged in plus the FCPX-created symlinks. The new symlinks are disambiguated with an (fcp1) suffix. The "proxy cheat" method also can be unreliable -- even for existing FCPX-created proxies, if those are relocated to a different-named hard drive.
Note: Finder calls both aliases and symlinks "aliases", so this is confusing. In Finder easiest way to differentiate an alias from a symlink is do "Get Info". If there is a "select new original" button it's an alias. If not, it's a symlink. In terminal ls-l will show the symlink path whereas an alias appears as a regular file.
There are ways to create proxies that satisfy the FCPX check for audio channels, aspect ratio, and timecode -- articles have been written about that. However this requires that FCPX-created proxies once existed -- either inside or outside the library. If those FCPX-created proxies never existed, there are no references inside SQL tables inside CurrentVersion.fcpevent, hence there is nothing to relink those proxies to.
Since the data store is based on SQLite and that on-disk file format is documented, you may wonder why can't a SQL guru write an app to do this. It's because the application-layer schema and data dictionary are totally undocumented. IOW what the column names and values mean. Inside FCPX, the programmatic interface to the underlying database is apparently an object-oriented "graph database" using Core Data, which in turn uses SQLite as the persistent data store. The SQL column names and values are generally not human-readable CHAR fields, but either complex descriptors or Binary Large Objects.
The bottom line is I don't know of any way to use externally-created proxies when FCPX-created proxies never existed.
Wow, Joe, that was very in-depth and super fascinating! Ok, so now it all makes sense...and means I'm stuck, LOL! Because I'm editing a feature film and I can't be stuck on final export to Colorist depending on a "proxy cheat" method that may or may NOT work. Alright, well, thank you all for your help.
[Paul Munger] " Ok, so now it all makes sense...and means I'm stuck, LOL! "
Jeremy's method might work but I haven't thoroughly tested it. In simple testing it seems to work. If you create external proxies with FCPX or any utility like RCX that can produce files with the the right characteristics (matching file suffix, pixel aspect ratio, audio config and timecode), you can then import and edit proxies as regular media, then later relink to the original high res media. The project must also be adjusted from 1080p to 4k.
If your camera-generated proxies won't relink to the original R3D files, I don't know why. Maybe Jeremy could advise.
Outside of RED, this could be a useful technique -- assuming it proves robust at high stress and diverse conditions. That's always a possible issue, e.g, the above "proxy cheat" technique *seems* to work, but I've tested that extensively and it is not reliable when dealing with several thousand files if the drive volume name has changed.
Outside of the camera-generated proxy case, why would anyone do this? Because normal FCPX proxies cannot be relinked and are unreliable if the drive name has changed. The library can easily get messed up so changing the drive name back doesn't fix it. The proxies then won't work even if the symlinks are rebuilt by hand as either aliases or actual symbolic links.
In a large project using "lean" libraries and proxy-only workflow for downstream editors, it could take a week to generate proxies. Those are sent to editors with a note the drive must retain the given name. What if they have two or three of those drives? Then what if they reboot their computer and the drives mount in a different sequence, giving them a different internal ID? The proxies turn red (maybe permanently). They can be sent new proxies but the fault isn't there -- it's inside the library. Proxy symlinks can be rebuilt manually using various methods but the upstream pointers inside the library SQL tables cannot be rebuilt. Regenerating new FCPX proxies just to refresh the internal pointers could take a week which isn't practical.
Using FCPX-generated proxies as orig. media would mean you can't rapidly switch between regular and proxy mode, but at least the "proxies" could be relinked since FCPX thinks it's regular media. Then when editing is finished that library can be relinked to the orig. 4k media (presumably).
I'll test this more tomorrow. Thanks to Jeremy for mentioning this.
Even if this method proves reliable, it still might not work for my doc group. When FCPX generates proxies (whether internal or external) those are rearranged in date-named folders, with the original folder names lost. Yet before ingest our data wrangler puts lots of work into the folder hierarchy, which in turn produces keyword collections upon import. Those keyword collections are vital since they have info on shooting location, camera name, etc. Without camera name info the clips cannot be prepped for multicam sync.
We could change our ingest procedures to not store any info in folder names, only finder tags. If that survives the round trip to proxies and eventual relink to 4k media, that might work.
FindrCat can tag the media files with keyword names; not sure that works with proxies. Without that, proxies distributed as original media would not retain metadata from the original media folders : http://intelligentassistance.com/findrcat.html
[Joe Marler] "(matching file suffix, pixel aspect ratio, audio config and timecode"
Suffix is less important. I can link between MXF and mov or r3d and mov easily as long as the audio is the same. Even frame size doesn’t matter. If the audio channels don’t match, then fcpx simply won’t relink.
I certainly do not know the ins and outs of sql or fcpx’s internal structure. I just do what works. Fcpx’s proxy relink is way more strict than even the original media link as you mention. It’s location dependent, original relink is not. It’s definitely odd.
[Joe Marler] "If your camera-generated proxies won't relink to the original R3D files, I don't know why. Maybe Jeremy could advise.
Depending on the drive format chosen on the red (UDF or FAT32) the QuickTime movies don’t follow the same spanned format as the r3ds (or MXF) FAT32 4GB chunks of media). The camera generates proxies don’t seem to have the same metadata as the RCX generated proxies. Somehow, fcpx knows how to relink to the rcx generated proxies and the spanned r3ds but can’t do this with camera generated proxies. Again, I don’t know why this is, or how it works in fcpx, but it seems to work.
This is why, with Ref especially, I use RCX to generate ‘proxies’ before editing and are usually high quality 444 redlogfilm proxies. On quicker jobs, I’ll even send those to grade rather than go through an r3d relink and file transfer.
[Jeremy Garchow] "Suffix is less important. I can link between MXF and mov or r3d and mov easily as long as the audio is the same. Even frame size doesn’t matter. If the audio channels don’t match, then fcpx simply won’t relink."
That's odd, in my tests importing 1080p .mov proxies (as regular media) generated by FCPX from 4k .mp4 files from Sony Alpha cameras, it will not relink to the original 4k files unless I rename the suffix from .mp4 to .mov.
[Jeremy Garchow] "...Fcpx’s proxy relink is way more strict than even the original media link as you mention. It’s location dependent, original relink is not. "
The basic problem is there IS no proxy relink. For original media there is the "magic" relink courtesy of FCPX storing inodes (provided by the HFS+ file system) for each orig. media file. If those are relocated or renamed within the same volume, relink automatically happens and the symlinks are rebuilt with the new location and file name when FCPX starts. However there is no procedural step or menu -- it just happens.
I said previously this works for proxies, but I re-tested it and it does not. However the "proxy cheat" trick will usually work -- if you delete the proxy symlinks inside the library and copy aliases from the new proxy location via click/drag while holding OPT+CMD to the proxy folder inside the library, the red proxy clips will start working again. Internally FCPX scans those and rebuilds true symlinks to the revised proxy location. However this doesn't always work for a renamed disk volume.
Path resolution by inode only works within a volume. If the drive name changes, a manual relink is required -- even for orig. media files. FCPX only provides that for media files not proxies. Why they don't provide proxy relink is unknown -- especially since the above "proxy cheat" method is essentially doing that. The fact this doesn't work reliably on a changed drive name could indicate there's some deficiency in the SQL schema or inode resolution for this case, so maybe they decided not to formally document and support that.
Using aliases instead of true symlinks is also not the problem cause. I also built new symlinks using the Symbolic Linker utility and those behaved just like aliases: https://github.com/nickzman/symboliclinker/releases