Perfect Vimeo Encode EXCEPT FIRST SHOT!?
So I have achieved an x264 encode (settings below) that looks better on Vimeo than any other high-grain 16mm footage I've ever put online before. There are literally zero artifacts to my eye except for on the very first shot of clouds. At 0:03 you can see how the otherwise fairly charitable rendering just explodes into huge blocks of recycled pixels for about a second. This does not happen whatsoever for the remainder of the video. Is this the fault of Vimeo's encoder? NOTE: The exact same file does not have this issue when being played back YouTube. Is there a Squeeze setting that could help this without sacrificing what has been achieved thus far?
My question is, how on earth could this be happening?
I am 99% close to relative perfection for this medium > web video. I am so close I can almost taste it. But why, why, why this on the first shot!!!?!?!
PIXELS EXPLODING OUT OF THE BLUE (no pun intended) [0:03]:
Squeeze 10 Standard Settings (*settings not shown are automated):
Any help or ideas that can be offered will be greatly appreciated!!
I see your keyframe rate is one every 300 frames.
I believe Vimeo recommends one every second so you might try changing that.
Yeah, I was planning on experimenting with that here. With high-grain material though, I've always had better results with a larger keyframe distance (at least with recent Blu-Ray encoding) with the theory being that grain info typically needs to be "new" in almost every frame for grain to continue to appear "running" as it does in the source footage. Often I find that thick film grain confuses most compression algorithms which are typically meant for material shot natively as pixels (digital) or perfectly exposed, fine-grain film material, and as a result the "running" grain comes and goes being replaced on and off by recycled artifact "blocks", sometimes even within the same shot, just as it does here.
But it is definitely worth a try.
Just double-checked and yes, Vimeo recommends one every second.
I will run some tests here...
Appreciate the response.
Anybody else have any ideas?
Just ran a couple tests encodes of the first 10 seconds here.
Keyframing every 24 frames...did exactly as I expected, recycled too much info and rendered lots more "blockiness" even in the other frames where the footage was perfect in the original.
I added about .5 seconds of black space in FCP to the start, re-encoded with otherwise all original settings (inlcuding 300 on keyframing), and it appears that the momentary "block explosion" that was once at 0:03 is now at 0:01 and less disgusting. Still unacceptable but this seems to have again isolated the problem only to the first shot as before and then shift it back a couple seconds--which leads me think this all has something to do with how Vimeo is interpreting the first keyframe. Sound plausible to anybody?
Third Test (Underway):
I have now added a whole second of black space to the start of the original .mov and I will re-encode and upload that here to see if that will further offset the misinterpretation of the first keyframe into the darkness :) Will update in a few minutes...
So I have conducted a number of tests only to conclude that this is just a crap shoot with Vimeo's encoding engines.
I have run multiple test uploads of the first 10 seconds with different amounts of black space added before the start of the footage. That seemed to correlate to shifting the problem around a bit, but whenever I upload a full version (with the exact same settings) the problem resurfaces.
THE MOST RECENT TEST:
I even took a couple frames out to Photoshop to get rid of the the big white hair on the film at 0:03 to see if that was messing things up for Vimeo's encoders and could help...and IT DID HELP. My most recent test clip (with 1 second of black space and the removed hair) looked absolutely perfect and then when I uploaded the full version with all the same settings, the problem resurfaced again.
This obviously confirms that there is something going on with Vimeo's encoder relative to the length the video and how it thereby globally is reading something like the keyframe distance. Which in turn, seems to render these 10-second test clips I've been trying to be meaningless, and furthermore, it means that I will have to do all future tests with alternate full versions which will each take 5 hours to render :/
Since I know, based on that one successful test clip (mentioned above) that it IS possible for this opening shot to make it through Vimeo's encoders without getting jacked up, the plan is to basically just keep uploading full versions with small additive changes.
Currently I am taking the the version I just uploaded and removing the black space at the beginning to see if a full version with all original settings and just the hair removed will give Vimeo the info it needs to not screw up the opening shot.
If that doesn't work, I plan on trying such things as extending the keyframe distance to 400 or 500.
If anyone has any other experimental ideas at this point that are "additive" in theory (in the sense that they do not have potential to hurt the other 99% of what is the most perfect Squeeze encode I've ever uploaded), then please, please offer away.
ONE BIG QUESTION I HAVE (as I am having to use multiple vimeo accounts to upload all the tests) IS: "Is it possible/plausible for one to upload the same video file to Vimeo twice and come up with different results? Or in other words, would it be safe to assume that if you put the same exact thing in, you will always get the same exact thing out? Do we have any reason to believe that encoder algorithms on YT and Vimeo are not constant?
Did you ever figure out a solve for this? I am having the exact same problem. I tried both H264 and prores... The original exports are clean on my computer, but once uploaded to vimeo, the video starts to break right around 4 seconds, with crazy pixelation artifacts (looks like frame blending) for a few seconds, then is clean the rest of the video. It's driving me insane, as its the same spot no matter what compression.
No I have not. I've now spent close to 20 days over the last 6 weeks working all day (and most of the night) trying to figure this one out. Over that time, I've rendered and uploaded probably 50+ tests (for the original music video and other projects), become a Vimeo Pro member to facilitate more test uploads per week, test rendered with every imaginable effectual change in Sorenson Squeeze 10 and Adobe Media Encoder CC, gone back into FCP and experimented with adding varying amounts of black space to the beginning to buffer/offset what I may be the first to dub, "THE 2015 VIMEO 4-Second MACROBLOCKING PHENOMENON", as well as brainstormed numerous other render workarounds or possible offsets (enough to fill 15+ legal pad pages)...all to absolutely no avail.
We finally settled on a roughly 2 second buffer of black space for the original "Make The Races" music video: -- As you can see however, there is still a pathetic explosion of pixels (macroblocking) at the 5-6 second mark in the first shot. This was the most minimal degradation that we could achieve.
I have found something rather telling, reassuring, and disturbing. I've been clicking around Vimeo's staff picks...and every single one of them that has substantial video footage at the 3-4 second mark (not just text), seems to suffer from this problem.
Exhibit A: https://vimeo.com/channels/staffpicks/124523399
Exhibit B: https://vimeo.com/channels/staffpicks/124191188
I am left to conclude that something in Vimeo's render engine has changed drastically or been cheapened in some way in the last couple years. I mean the very nature of the so called "Compression Guidelines" is obviously stated as it is, just as much, if not more so, to curb the workload of Vimeo's servers than it is to help professionals achieve the best results.
In terms of just general quality, it doesn't take much searching around Vimeo's forums to see that they will subtly encourage people who are stuck trying to achieve good results (especially with film footage) to keep all the guideline settings the same but "raise" the data rate higher than their documented suggestion. I found a phenomenal example of this where the Vimeo moderator kept telling the guy to "just raise the bitrate", though he continually insisted that he could explain why. Unfortunately I did not bookmark that particular example. Here is another great example though that I just found: https://vimeo.com/forums/help/topic:42395 (see Daniel Hayek's fifth comment: "the 10Mbps recommendation is not something I throw around a lot b/c 5Mbps works for most people just fine").
However, I've been aware of this for years through continually having the deck stacked against me trying to encode high grain film footage for web, which frankly, just doesn't really work. What remains very interesting though, is the fact that, though I had issues in the past with the global fidelity of my uploads, prior to really the last 1.5 years, I NEVER EVER had these problems with macroblocking, particularly the "THE 2015 VIMEO 4-Second MACROBLOCKING PHENOMENON". It just didn't exist before, and now it does.
AS A SIDENOTE:
Many other professionals have chimed in on how to "not follow Vimeo's guidelines" for best Vimeo results:
http://philipbloom.net/forum/threads/keyframe-settings-for-encoding-h-264.1... (Matt Davis' response here is a good point regarding keyframe distance, which I always set at 300, and also regarding raising the data rate...however, and I believe he says this but I may have read him incorrectly, he is simply wrong when he says that you should upload the highest possible quality video to Vimeo. If you upload an uncompressed .mov file self-contained straight from FCP, and I have done this numerous times (twice in the last month), Vimeo will crush it way more than if you encode certain things yourself beforehand. Having said that, set the data rate of your x264/h264 encode to peak at the original data rate of the footage and KF interval at 300. Trust me on that one. I can't imagine that many people on the face of the earth have spent as much time as I have going after this stuff. And like I said, the reason I have had to do more than most people is because of the nature of the material I've encoded---example: .
Anyway, this is my Vimeo Manifesto. I can't wait until we have internet as fast as they do in S. Korea and when Vimeo thereafter is forced to employ superior encoding methods; that or, (after all the hair pulling I've done in the last 5 years), the day the internet explodes and the robots take over. At this point, I will welcome either scenario :)
Hope that helps.
Glad to know I am not alone.
Thanks for the detailed description of your process. Your manifesto will be of great service to a lot of people that are banging their heads on the wall. I myself have come to the conclusion that Vimeo's encoders just cannot process fine film grain. I still don't know why that makes it crap out just at the top of the video, but I took out the grain overlay on my video, and no more problems. I noticed your music video was shot on 16mm, and has fine grain as well...