"Best" way for motion-interpolation?
I'm currently looking into various motion-interpolation methods to achieve the smoothest possible viewing experience of my web video content, which is being hosted on YouTube. The video game gameplay footage is recorded in 60 FPS with NVEC (Shadowplay), Mirillis Action! or MagicYUV (Dxtory) codec. So far I've come across:
- SmoothVideo Project (SVP)
- ReelSmart Plugins (Twixtor & RSMB)
- Adobe After Effects Pixel Motion / Frame Blending
- Sony Vegas Resampling
My idea is to combine multiple methods, as the video should not only look smooth at 100% speed, but also on slow motion parts. But since I don't know how exactly these algorithms work, I wanted to know if my idea is flawed in some way.
1. Use Twixtor at 10% or 6% slow-motion speed on the 60FPS clip to interpolate frames.
2. Adjust lenght and export the clip which is now 10/16.66 times longer.
3. Use avifrate or vdub to virtually fake a 600 or 1000 FPS recording which puts the clip back in its original length.
4. Let Sony Vegas Resample or After Effects Optical Frame Blending do the work when exporting as final 60FPS product.
Now the question: Does this make sense?
I feel like RSMB should be able to achieve the same, yet I don't even come close to the visual smoothness of the method above. Obviously the Sony Vegas Resample is missing with RSMB, as the video would be imported and exported with 60 FPS at all times. My described method also makes it a lot easier to edit fast velocity changes, as I can just use the velocity curve editor with lagfree preview, instead of keyframing the twixtor effect, which can be heavy on performance. The only downside right now is that rendering the clips with the Twixtor effect takes very long. I know Twixtor is not made for this purpose, but I have yet to test if I get the same result with Adobes built in optical frame blending.
Here's a random example of video footage (apologies for Fortnite) that is actually recorded at ~200FPS (~200FPS ingame, 240FPS variable in OBS) @1080p and then exported at 60 FPS with Sony Vegas smart resampling. The new Nvidia graphics cards in conjunction with NVEC allow for such recordings, if there's no HDD/SDD writing speed bottleneck. By the way, the video is just upscaled to 2160p so YouTube gives it more bitrate on the various quality settings. Because of the high recording framerate, the whole procedure of creating interpolated fake frames is not required here.
But obviously you cannot achieve such high frame rates in every game, and that's where my method comes into play. My examples:
Raw 60FPS footage:
Fake 600FPS footage:
The Twixtor is on default settings and no tweaking has been made, simply ignore the few extreme blur moments. This is simply a proof of concept.
I can clearly see the difference on my 165hz monitor (if that matters). I don't have an example with RSBM right now, but I know for a fact that my fake 600 FPS example looks smoother than any attempt with RSMB.
Thanks in advance for any input.