APPLE FINAL CUT PRO: Apple Final Cut Pro X Creative Community Conversations FCP Legacy FCP Tutorials

Slower 1080p FCPX Exports compared to Old System

COW Forums : Apple Final Cut Pro X

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Paul Gracia
Slower 1080p FCPX Exports compared to Old System
on Aug 23, 2018 at 8:58:08 am

Hi all!
I've been a COW reader for a long time, and today is the first time I'm posting ☺

Introduction
I have an issue that I haven't been able to solve, and I create this post in the hope of being saved!
I am a video editor, and due to what I'm asked for, my main workflow is editing and exporting 1080p and 4K in h264, no ProRes.

I usually get the originals sent to me through the cloud, already in h264 (cameras, capture cards, etc) and I then do my thing.

Background
Up until a couple of months ago, I had been working since I started with a 2014 5K Retina iMac, full-spec: i7 4GHz, 32GB 1600MHz DDR3, 500GB SSD, AMD Radeon R9 M295X

I started to work with 4K around 2 years ago, and this last year has been a pain in the ass working with it, due to overheating mainly: My GPU would reach 105ºC pretty fast (I'm talking after 30s of exporting), and I needed to turn off the screen in order to keep it there. If I tried just to write an email while exporting, temp would go all the way up to 110 - 111ºC and then shut down. It was way worse with 4K, of course, but it also happened with 1080p.

I had been holdin off of buying a new mac because I was waiting for a new Mac Pro, and the iMac Pro doesn't cut it for me because it makes me buy a screen after all, and I already got two. So since there doesn't seem to be any signs of a new Mac Pro at least until... what, 2020? Well, I decided to go build a hackintosh.

Gotta say the building was easy, the configuration not so much, but now it's flawless and works amazing. Specs:
i7-8700K, 32GB DDR4-3200, 2TB Samsung Evo 860 (M.2 NVME), Radeon Vega 64.

Problem
System works great, don't get me wrong, but there is one issue that I can't get my head around: 1080p exports when there are little or no effects are slower with the Hack than with the old iMac. 4K is faster always.

I did some tests with my iMac and then ran them in the hack. Here are the results:


1080p (5 min)

TASK iMAC HACK NOTES
Extract .acc 04:03 00:39 *Mp4Tools, HD dependant
Split 5min 00:24 00:10 *Mp4Tools
Import 5min 00:12 00:12 *FCPX
Proxy 5min 00:52 00:56 *FCPX
Optimz 5min 02:18 01:31 *FCPX
Export 5min 01:58 03:23 *FCPX


4K (5 min)

TASK iMAC HACK HACK OC 4.3GHz
Import 5min 00:21 00:20 -
Proxy 5min 01:44 02:03 01:55
Optimz 5min 05:11 03:20
Export 5min 11:47 08:45 08:36


Conclusion
As you can see, leaving aside imports and all, there are some things I don't quite understand:
1. Transcoding to Proxy is a little faster on the old iMac, (7% on 1080p and 15% on 4K). (no OC)
2. Transcoding to Optimized is faster on the Hack (34% on 1080p and 35% on 4K). (no OC)
3. Export times on 1080p (no effects) are a 41%!!! faster on the old iMac.
4. Export times on 4K (color correction and some chroma keying) are a 25% faster on the Hack.

If it helps, BruceX test is around 35 seconds on the iMac and 12 seconds on the Hack.

I first thought it was due to the higher CPU clock of the iMac, 4GHz vs 3.7GHz, but after the OC times haven't changed much (you can see it at the right part of the code), so... I'm clueless.

I mostly do 4K now, so this is not a huge deal, but still... does anyone have any idea of what is going on?
Gotta say that I had some issues activating hardware h264 acceleration, but I finally did. It now fully works, tested and verified. (Prior to activating, export times were... well, you can imagine: crazy long).

Thank you!


Return to posts index

Noah Kadner
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 28, 2018 at 4:43:04 am

This kinda illustrates why a Hackintosh is a dicey prospect in terms of getting performance gains vs. OS compatibility issues- especially when you don’t get the former. In this case a lot likely has to do with the built-in hardware H.264 encoder/decoder the iMac has which most likely there’s no equivalent on the Hackintosh. That would explain why tasks that would leverage it are slower.

Noah

FCPWORKS - FCPX Workflow
FCP Exchange - FCPX Workshops
XinTwo - FCPX Training


Return to posts index

Joe Marler
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 28, 2018 at 11:48:14 am

[Paul Gracia] "I mostly do 4K now, so this is not a huge deal, but still... does anyone have any idea of what is going on?
Gotta say that I had some issues activating hardware h264 acceleration, but I finally did. It now fully works, tested and verified. "


Getting Quick Sync to work reliably on a Hack is usually difficult. There is code in FCPX which can use the similar AMD logic called UVD/VCE, which is used on the iMac Pro. They have different performance characteristics. Maybe on your Hack it's using Quick Sync on 4k and VCE on 1080p.

FCPX on the 2017 iMac is about 2x faster than the 2015 at H264 encoding. This is possibly due to the improved Kaby Lake version of Quick Sync, or some refinement in how FCPX uses it. You would probably have been better off getting a used or refurbished 2017 iMac. It exports to 4k H264 faster than my 10-core Vega 64 iMac Pro. It's also faster on the decode side, and it can edit single-stream 4k H264 without proxies. It's almost fast enough to edit multicam 4k H264 without proxies.

However the iMP can export proof copies to 1080p H264 faster than the iMac, is faster on most other tasks and is very quiet and stable. The slower export to 4k H264 is probably an anomaly or limitation in AMD's VCE; hopefully future versions of FCPX will improve this.

Here are some quick tests I ran using 5 min. of UHD 4k H264 XAVC-S 100 mbps 8-bit 4:2:0 on my 2017 top-spec iMac 27, using FCPX 10.4.3 and macOS 10.13.6. This is just a straight clip with no effects.

Create proxy 5 min: 1:40
Create opt 5 min: 3:40
Export 5 min 4k to 1080p H264 faster encode: 2:08
Export 5 min 4k to 4k H264 faster encode: 2:34

Same test on 2017 iMac Pro, 10-core Vega 64 version:

Create proxy 5 min: 1:25
Create opt 5 min: 1:53
Export 5 min 4k to 1080p H264 faster encode: 1:44
Export 5 min 4k to 4k H264 faster encode: 3:35


Return to posts index


Paul Gracia
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 29, 2018 at 9:14:46 am

Hi! Thanks for the responses.

Noah, quicksync depends on the processor and all Intel chips have it, so it should be the same. On the other hand, as Joe says, there might be some issues with FCPX or even the OS when engaging these hardware accelerations. This is because when "hackintoshing", you have to manually enable some memory addresses and pipelines, and some might be workarounds that work at a 80% performance.

I guess that there is no way of checking whether or not FCPX is using quicksync instead of vce, right?

Maybe some future EFI updates that are more appropriate for the current intel chips may solve this, in the meantime as I said, I'm not particularly worried because the one thing that was taking forever (4K transcode, edit and export) is much more fast now. 1080p... well, a 41% isn't much when we're talking an average of 10 mins exports, it now takes maybe 14 mins...

I also wanted to say that I wanted a hack because I needed a modular system, a computer that I could open and clean, upgrade one single part, install a new fan if needed, add another hard drive, etc. Also there is no iMac that can beat this CPU power (besides the pro that is), even though it's not its main task, I also use Handbrake quite often and oh boy... that's fast!!

--------------------

On another topic and since you mentioned proxy editing on multicam 4K... I have an "issue" I've chatted about with other editors and they all suffer it to some degree, (with actual macs as well).

Even when editing with proxy files, if my timeline has multiple 4K video tracks (for instance a base + 2 chroma keys), every time I do a cut the timeline freezes for like... half a second. In my old iMac this was like 3-4 seconds.

Since everyone I talk to suffers from this, I'm guessing this is FCPX related, but maybe you guys know what causes it?

Thank again,
Paul


Return to posts index

Joe Marler
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 29, 2018 at 12:26:45 pm

[Paul Gracia] "...quicksync depends on the processor and all Intel chips have it, so it should be the same. On the other hand...when "hackintoshing", you have to manually enable some memory addresses and pipelines, and some might be workarounds that work at a 80% performance..."

I thought most Hacks must disable the on-board GPU which in turn disables Quick Sync. Max Yuryev has built lots of Hacks and he mentioned that in one of his videos, I can't remember which one. See his Youtube page for details.

[Paul Gracia] "....I guess that there is no way of checking whether or not FCPX is using quicksync instead of vce, right?..."

No way I know of. iStat Menus has a GPU activity monitor but it doesn't correlate with Quick Sync activity. But the performance difference with and without Quick Sync or VCE is so great, you can usually infer this. The problem is how to force that change. On a Hack you can usually turn off the integrated GPU in the BIOS. On a Mac you can't do that. I'd suggest turning it off on your Hack, running a timed encode test, then turning it back on and comparing the times. If there's a big difference, Quick Sync is probably being used. If little difference it's not being used.

VCE use by FCPX is more complex, and much newer. It was only with the iMac Pro that Apple started using this. I don't know how the macOS and FCPX layers determine which to use in a machine with both Quick Sync and VCE hardware. However maybe you could temporarily remove your discrete GPU, disable the integrated GPU, run timed encoding tests and see if there's any performance difference vs using the discrete GPU with integrated GPU disabled. For all those tests use only ProRes material, no effects or edits and export to H264 "fast encode". This reduces the possibility of regular GPU graphical operations distorting the results.

You formerly could infer if FCPX was using Quick Sync by using "Fast" vs "Better Quality" encoding -- the multi-pass encoding was software only. However I believe the latest versions of FCPX use hardware acceleration for multi-pass, so it's slower but not the huge difference in previous versions.

Premiere Pro before the 2018 version did not use hardware acceleration for H264 encoding on Mac. It was about 4x or 5x slower than FCPX on the same iMac hardware. Starting with 2018 it uses hardware acceleration on both iMac and iMac Pro, so it's roughly as fast (encoding) as FCPX.

However Premiere does not use hardware acceleration for *decoding*. It is still plodding and sluggish on a 4k H264 timeline. By contrast FCPX is much faster and more responsive.

I think in former macOS versions, access to hardware acceleration required using Apple's Video Toolbox framework, a low-level interface below AV Foundation. The complexity of this might explain why apps like Premiere did not use hardware accelerated encoding on Mac until recently. Now it can be accessed via the higher-level AV Foundation framework.

If anyone is more interested in the low-level details of macOS and encode/decode acceleration, that is here:

Video Toolbox and Hardware Acceleration: https://www.objc.io/issues/23-video/videotoolbox/

WWDC 2014 "Direct Access to Video Encoding and Decoding": https://developer.apple.com/videos/play/wwdc2014/513/

[Paul Gracia] "... I'm not particularly worried because the one thing that was taking forever (4K transcode, edit and export) is much more fast now..."

If you inspect your encode times vs mine on the 2017 iMac, mine is 3.4x faster than your Hack (8:45 vs 2:34), and 4.6x faster than your 2014 iMac (11:47 vs 2:34). This indicates neither your iMac nor your Hack are using Quick Sync or something in the test is distorting the results, such as effects, non-rendered timeline, etc.

For doing encode tests it's best to use only a ProRes timeline or at a minimum optimized media, have no other edits or effects in the timeline. Those perturb the results. If possible it's best to use the exact same H264 codec between platforms, because some H264 codecs differ in the computational demands and compatibility with hardware acceleration.

[Paul Gracia] "....I wanted a hack... there is no iMac that can beat this CPU power (besides the pro that is), even though it's not its main task, I also use Handbrake quite often and oh boy... that's fast!!"

There are available Mac Handbrake builds which use Quick Sync but they aren't normal production releases. Those might be considerably faster, but I haven't tested them for stability or image quality.

[Paul Gracia] "....proxy editing on multicam 4K... I have an "issue" I've chatted about with other editors and they all suffer it to some degree, (with actual macs as well)....Even when editing with proxy files, if my timeline has multiple 4K video tracks (for instance a base + 2 chroma keys), every time I do a cut the timeline freezes for like... half a second. In my old iMac this was like 3-4 seconds...Since everyone I talk to suffers from this, I'm guessing this is FCPX related, but maybe you guys know what causes it?"

I don't recollect ever seeing this, and I edit 4k H264 multicam every day, but I usually don't do chroma keying. I just finished a documentary that was full of multicam, probably had 2,000 color correction effects, sometimes 10 keyframed corrections on a single clip, and it ran fine using proxies. The media was on a 4-drive Thunderbolt 2 SSD RAID-0 array. This was on both 2017 i7 iMac and 10-core iMac Pro.

You could possibly greatly improve your FCPX H264 encode performance by just getting a used or refurbished 2017 iMac.


Return to posts index

Paul Gracia
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 30, 2018 at 9:40:28 am

[Joe Marler] I thought most Hacks must disable the on-board GPU which in turn disables Quick Sync. Max Yuryev has built lots of Hacks and he mentioned that in one of his videos, I can't remember which one. See his Youtube page for details.

Most do, yes. But I chose a golden build per say. Everything works, including iGPU, or in other words, the integrated Intel graphics and therefore Quicksync. There are several video converters and terminal tests to check if hardware acceleration is enabled, and mine is. Plus, my h264 export times would be impossible without quicksync.

[Joe Marler] You formerly could infer if FCPX was using Quick Sync by using "Fast" vs "Better Quality" encoding -- the multi-pass encoding was software only. However I believe the latest versions of FCPX use hardware acceleration for multi-pass, so it's slower but not the huge difference in previous versions.

Ohhh my friend! This might be the thing here. In my old iMac the last time I updated anything was... well, a lot of time ago. I'm a CCC user and around 1,5 - 2 years ago my system "broke" after an update, so I went back to my previos backup and didn't update since then. This was my main and only work machine and didn't wanna risk it again, so FCPX stayed in 10.1? Or 10.2 for a looooong time. I only updated to 10.4 now with this new hack, 10.4.3 actually.

I use my own compressor settings which had Multipass disabled, but if FCPX now forces it... it makes sense that times are slower... In a same computer it might be 2x, but since this computer is faster it's just a 1.4x times slower?
Makes sense in my head. Also my 4K tests were including chroma key, color correction, etc; so the much higher performance of the Vega 64 vs the M295X is doing the trick. With the same graphics it'd probably be slower, but since the Vega outperforms the old one it's actually a 35% faster.

Joe Marler Premiere Pro before the 2018 version did not use hardware acceleration for H264 encoding on Mac. It was about 4x or 5x slower than FCPX on the same iMac hardware. Starting with 2018 it uses hardware acceleration on both iMac and iMac Pro, so it's roughly as fast (encoding) as FCPX.

Interesting. Didn't know that. Only time I tried Premier I went back at lightspeed due to the low performance. I was shocked to see so many people could use Premier on the Mac. Exports were insane, as well as transcoding etc.

Joe Marler If you inspect your encode times vs mine on the 2017 iMac, mine is 3.4x faster than your Hack (8:45 vs 2:34), and 4.6x faster than your 2014 iMac (11:47 vs 2:34). This indicates neither your iMac nor your Hack are using Quick Sync or something in the test is distorting the results, such as effects, non-rendered timeline, etc.

Well, that comparison isn't fair, is it? I used a 120Mbps 1080p file with a custom 1080p compressor export setting. I don't know your source file, but you probably used FCPX's h264 default export option, right? We could upload a test file and try both using the same export method, would be interesting.

And both my old iMac and this Hack are using quicksync, of that I'm sure. If you manually get rid of quicksync, export times of good quality 1080p is incredible slow, it's almost hard to believe.

There is something that could be interfering though: My chip is a Coffee Lake and hackintosh patches include adjustments for Skylake architecture. Maybe, just maybe -and I'm fully speculating here-, Skylake used pipelines or memory addresses or something from 1 to 100 (just to put an example) and Coffeelake uses from 3 to 102, which means I'm using 2 less than the actual needed. I don't know if I explained myself enough, hehe.

Joe Marler I don't recollect ever seeing this, and I edit 4k H264 multicam every day, but I usually don't do chroma keying. I just finished a documentary that was full of multicam, probably had 2,000 color correction effects, sometimes 10 keyframed corrections on a single clip, and it ran fine using proxies. The media was on a 4-drive Thunderbolt 2 SSD RAID-0 array. This was on both 2017 i7 iMac and 10-core iMac Pro.

My library is in my M2 drive, 2500/3000 MB/s write/read speeds, it can't be that. I don't know what it is really. My workflow is a little weird though, I got used to working in a strange way but it saves me several time and hardware:
* I capture 2K + 1080p in one same file in a 3640*1440 resolution canvas.
* I also capture a native 4K file in a regular 2160p canvas.
* 2-3 audio tracks, sometimes more.
In FCPX I create a 4K project and put my base 4K video track and over it the 2K+1080p video track. In some projects it stays disabled the whole time, in others I crop it so it uses the 2K one, in others the 1080p, and in some I duplicate it to use both. These are the tracks I put more effects (and maybe chroma key) on.

I know FCPX sometimes struggles with timelines which have tracks not its actual size, so this might be it? It's very suble as I say, and usually only when "cutting" a track in the timeline, a half second timeline freeze. It might happen from time to time when adding the chroma key effect as well.

Joe Marler You could possibly greatly improve your FCPX H264 encode performance by just getting a used or refurbished 2017 iMac.

I know that is not the case. The graphics power is very low, my Vega 64 isn't easy beatable. I tried to export 5 mins of the 4K projects in the app store in their computers and times were much slower. They even had top tier iMacs and MBPs but none got close to my Hack.
In 1080p maybe, but it only represents like 10% of my exports, and the extra time is so little... 4K projects on the other hand, with all the effects... the Vega 64 help is amazing, at this point I think only the iMac Pro can beat it because of its extra 8GB of GPU RAM (mine is the 8GB version), but the iMac Pro doesn't have quicksync so... it might even be slower in my projects.

Thanks!


Return to posts index


Joe Marler
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 30, 2018 at 12:25:19 pm

[Paul Gracia] "I use my own compressor settings which had Multipass disabled, but if FCPX now forces it... it makes sense that times are slower... In a same computer it might be 2x, but since this computer is faster it's just a 1.4x times slower?..".

FCPX doesn't force multi-pass H264 encoding, it defaults to single-pass. What changed was multi-pass now uses Quick Sync or VCE, whereas previously (I don't know how far back) only single-pass used Quick Sync.

[Paul Gracia] "... I used a 120Mbps 1080p file with a custom 1080p compressor export setting. I don't know your source file, but you probably used FCPX's h264 default export option, right? We could upload a test file and try both using the same export method, would be interesting..."

Yes it's not a good comparison. If you're interested in evaluating *encoding* performance, that should be done with all parties using the same ProRes clip, no effects or edits, and encoding to the same H264 parameters. That prevents any decode-side distortions of the results and prevents intermixing render performance with encode performance.

Ultimately you want the whole thing to go fast -- render *and* export. But without breaking down the problem and testing individual pieces it's impossible to tell the relative contribution of each. E.g, if I used Digital Anarchy's Flicker Free, Neat Video, and Imagenomic Portraiture on some clips, then tried to measure encode time, this wouldn't tell me anything about Quick Sync or encoding performance. Those are extremely compute-intensive plugins.

[Paul Gracia] ".....And both my old iMac and this Hack are using quicksync, of that I'm sure. If you manually get rid of quicksync, export times of good quality 1080p is incredible slow, it's almost hard to believe...

OK that's good but your encoding results can't really be compared to anything else because your timeline has features which make that impossible. If you want to compare we should both use the same clip with no effects or edits. Then we should carefully add specific effects or edits we both have. This could be interchanged with XML. Unfortunately I'm leaving on a field assignment until mid-September so can't do any work on this until then.


[Paul Gracia] "....My workflow is a little weird though...
* I capture 2K + 1080p in one same file in a 3640*1440 resolution canvas.
* I also capture a native 4K file in a regular 2160p canvas.
* 2-3 audio tracks, sometimes more.
In FCPX I create a 4K project and put my base 4K video track and over it the 2K+1080p video track.
In some projects it stays disabled the whole time, in others I crop it so it uses the 2K one, in others the 1080p, and in some I duplicate it to use both. These are the tracks I put more effects (and maybe chroma key) on.
I know FCPX sometimes struggles with timelines which have tracks not its actual size, so this might be it?..


It might somehow be related to this. Your workflow is so different from mine, we can't draw any conclusions about platform performance. You would need to take the time to try alternate workflows, starting with a more simple approach and see at what point the performance problems happen.


[Paul Gracia] "...The graphics power is very low, my Vega 64 isn't easy beatable. I tried to export 5 mins of the 4K projects in the app store in their computers and times were much slower. They even had top tier iMacs and MBPs but none got close to my Hack..."

OK that was a good idea, and more people should test like that. This is difficult because most stores only have base-configured machines, not CTO (Configure To Order). You could probably try this on a base iMac Pro at the store, that might be interesting.

[Paul Gracia] "...4K projects on the other hand, with all the effects... the Vega 64 help is amazing, at this point I think only the iMac Pro can beat it because of its extra 8GB of GPU RAM (mine is the 8GB version), but the iMac Pro doesn't have quicksync so... it might even be slower in my projects..."

I agree despite the high performance of my 2017 iMac handling H264, my 10-core Vega 64 iMac Pro is preferable. It's just more robust and stable when under extreme loads, quieter, and the more powerful GPU and additional cores help with compute-intensive plugins. I only suggested the 2017 iMac because it's super-fast on H264 in FCPX but when you add lots of effects it slows down.


Return to posts index

Paul Gracia
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 31, 2018 at 8:04:37 am

[Joe Marler] FCPX doesn't force multi-pass H264 encoding, it defaults to single-pass. What changed was multi-pass now uses Quick Sync or VCE, whereas previously (I don't know how far back) only single-pass used Quick Sync.

Ahhh, ok ok, I didn't understand you properly then!

[Joe Marler] Unfortunately I'm leaving on a field assignment until mid-September so can't do any work on this until then.

I don't know how to prepare a XML file with effects, but I could upload some prores file. You can do the tests whenever you can, no rush here. As I said, in the end I'm still very happy, because my workflow has speed up (in general) like x5. These small problems represent like a 1%, but I'm a perfectionist so...

[Joe Marler] OK that was a good idea, and more people should test like that. This is difficult because most stores only have base-configured machines, not CTO (Configure To Order). You could probably try this on a base iMac Pro at the store, that might be interesting.

They did have one there, but after almost 2h in the store, it didn't free up at any time, there was people in it constantly, including kids looking at facebook. (Won't ever stop amusing myself with these people that log into personal accounts in apple stores or the likes).

I live in Barcelona, we have two Apple Stores, and they always have the top tier computer of each category. A friend of mine that worked in one of them told me that they do this because a lot of people want a better computer than the base one and don't wanna do it through the internet, due to fear or impatience, and they end up buying the top tier one even though they probably didn't want to spend that much. Free money I guess.

[Joe Marler] I agree despite the high performance of my 2017 iMac handling H264, my 10-core Vega 64 iMac Pro is preferable. It's just more robust and stable when under extreme loads, quieter, and the more powerful GPU and additional cores help with compute-intensive plugins. I only suggested the 2017 iMac because it's super-fast on H264 in FCPX but when you add lots of effects it slows down.

Yeah. Color correction isn't that bad, but chroma keying a 150Mbps 2K file which is a cropped down 4K file is pretty challenging. Gotta say though, my Vega 64 has only gotten to a 55% of GPU use, and that was a special occasion which involved several effects I don't regularly use. In normal operation (regular workflow) it sits between 35-45%.

Maybe with one of your projects with 2000+ effects... haha!

-------------------

Since we have time here, I'll try to prepare a ProRes simple 4K file, nothing fancy, just to test export times.

Thanks!


Return to posts index

Paul Gracia
Re: Slower 1080p FCPX Exports compared to Old System
on Aug 31, 2018 at 8:32:44 am

Oh my god... After writing that last post, I realized the canvas size might have everything to do with that issue, so creating compound clips came to mind... voilá!!

Once the 1080p or 2K video are cropped down, I just need to make them a compound clip, which makes it 4K like the timeline, and editing is flawless. No more "half a second freezes" upon splitting, enabling disabling... anything :)


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2020 CreativeCOW.net All Rights Reserved
[TOP]