h.264 lip-sync pb problems Premiere
I'm working with 720p h.264 web cam interviews shot in a mixed bag of non-standard frame rates. They look fine when I play them in Qt Player.
I started to work in Premiere CC (2014 v8.2) and was surprised to encounter lip-sync playback issues with them (picture feels late, sync seems soft, intermittent & drift-y). It's worth noting that I tried Avid MC 8.2 and it handled playback (and output) reliably. But since an Avid project wants to be at a single frame rate (and only standard ones at that) I felt it would be best to edit in software that can mirror the source file, so as to avoid any frame rate interpolation and artifacts.
I *think* the sequence exports played in Qt Player are ok but it's disconcerting. If there a gotcha or Magic Bullet that I'm missing?
I just verified that of two source clips with similar frames - one will play in Premiere just a little rubbery, the other about 15 frames (or more) out of sync - yet play perfectly well in the Qt player (OS 10.8.5) what gives? Then when it's time to export the problem one's seq from the TL - the sync problem is very much there.
Premiere says the two clips have this frame rate 24.02, however the Qt player says the problem one is 24.33 while the decent one is 23.99. (I've yet to get an answer why these were shot at different rates - I'm not holding my breath)
I just double checked that Avid MC 8.2 will play the problem clip in sync, however is freezes up constantly and then plays ok for a few seconds
You have fallen into the OIS trap.
This type of image stabilizer is on by default on some cameras and will either speed up the frame rate or slow it down to stabilize an image. If you are on a rig or tripod, turn this off and make damned sure you check that before shooting. After shooting, if you are experiencing frame rate problems, the missing frames will be marked and the frame rate change will be as well, but only in the bitstream, not in the header. SO.... ...You'll have to transcode to a useable format. Find the base frame rate for the cameras, looks like a 25p on the 24.03, and a 24p setting on the 23.xx (25 frames and 23.9x in media encoder). You should transcode them with "USE FRAME BLENDING" so that the frame rates are filled in. This will fix any problems with the videos themselves.
Next, you'll have to import those into premiere. I also advise you to right click the clip in project panel and Interpret the Footage to your sequence frame rate. This will flag it and play it as if at that frame rate, but conforming it to seconds instead of frames. WARNING: you may drop some frames (though not many, and usually not enough to degrade the quality unless in areas of high motion).
Wow, sounds like you've nailed it. I'll try it and get back to you. Would you mind if I posted your reply to the Avid-L where I asked a variation on this question? I will credit you as "Ht Davis" or any way you'd like.
Not at all on the posting thing... ...HT Davis. That's me.
I've run into this problem a few times lately with a cheap CANON HFr50. It keeps defaulting to putting that setting on, even after I turn it off. I'll be ref lashing the firmware when I can get another external to run parallels on. Burned up my last one running a decrypt test for a virus repair on a friend's machine... ...didn't work to well...
It's strange to me that Adobe Media Encoder (in the Broadcast options) doesn't over Quicktime. The next best alternative seemed to be mxf. I ran a test to covert the funny file to 24fpf (w frame blending) and the result was out of sync. I turned to Apple Compressor 3.5 and converted to 24fps ProRes (don't think frame blending is an option) and it looked ok. (This is a talking head). I've lost so much time troubleshooting today, I think I'll just go with Compressor.
Not shown in broadcast options, but you can create your own preset, and choose quicktime. Compressor will do a blend if you force the frame rate setting in video options (no auto setting). If you want pro-res, you may have to download it from the adobe site for media encoder. If you are using older compressor, use the cs6 presets, newer compressor, use the CC presets from the site. I've done both. It will show up under broadcast afterward.
It's probably asking for a lot but I wish I understood why I get such different sync results with different frame rates (sources) in Premiere (just playing back and in seq with matching frame rates.. no conversion)
Again in Avid playback of the same worst case h.264 Qt file it will keep sync, though it hangs-up on a freeze a lot, releases & plays, hangs-up on a freeze a lot, repeat.
Is irregular performance something about h.264? I'm just not used to it
It occurred to me that (maybe I'm slow on the uptake) that the "peed up the frame rate or slow it down" is happening on the fly, yes? In other words do you think the frame rate is constantly changing throughout the mov? Is there a way to analyze that (monitor with metrics)? This is not a academic question as it would help explain to the client why this is taking so long to sort out.
Avid doesn't care about the sequence frame rate. It plays back at the frame rate of the video you place into it. IF that video varies, it doesn't show.
Premiere sets all to the Sequence rate. If the video has VFR problems, you can turn on the option in the program monitor to show dropped frames and play it. Areas where there are dropped frames will be marked in the sequence on the render indicator, and the time indicator before and after in the program monitor will be colored different to the others. Premiere will allow it to play back at the rate of the sequence, and won't freeze.
Avid freezes if too many frames are dropped at once or if too many frames in total are dropped, as it tries to fix it by guessing with a single pass blur frame when more than one clip is on the timeline. When it can't keep up with the drop, it freezes and even fails.
Premiere doesn't fail, it just shows there's a problem, and let's you decide how to fix it. Use AME with frame blending on, and it will guess a large amount of frames (which, if extreme, will look like a still frame) in order to make your video useable.
"Sync works if the frame rates are the same."
Yes. True. But is a Variable Frame Rate at 29.97fps the same as a Constant Frame Rate of 29.97fps? (answer: Variable Frame Rate has an Average rate of 29.97fps and will vary in some areas due to natural motion algorithms and stabilization).
"Avid and other editors play it fine."
They do play, but try to sync and line it up with another clip that has a CFR, it will freeze. Why? It can't understand what's happening with the two frame rates. You've got one demanding variable, one constant. It gets caught asking which to stick to: <"play, play, play, pl... ...um... ...this one says 27.6fps now... ...this one 29.97fps... ...27... ...29... ...humena, humena, humena, CRASH!">
These are not designed for post production nesting workflows. They are more broadcast oriented, so you rarely sync two videos for multicaming by nesting. You're encouraged instead to use clip edits and in\out marks to create sub clips that fit into place. If a camera angle starts to wane off sync, you just put in a different clip. Playing clips at the same rates is easy. Playing back clips at different, but known ratio frame rates is also not difficult, as the algorithm is built in for different combinations. However, if you play several tracks that start at the same frame rate and they drop frames at different times, you're going to have problems playing back unless you drop from both tracks. If you play according to video frame rates, you're going to have some problems playing these back if even one of them varies it's frame rate.
Premiere is Post\EDIT oriented. It can fit multiple workflow styles. Premiere lets you see a problem without crashing as it plays the Sequence frame rate, not the video frame rate. It will play the video frames according to the rate of the selected sequence. Where frames drop in a video, it won't try to guess the frame, it will simply play the next available frame, keeping the same rate, which will throw off the synchronization for that video (showing you the problem). Other tracks are unaffected. This is how a professional tool should behave. There are also built in ways of fixing this.
There are two schools of thought here:
1. Mark the first visible sync error, and slowly go back until it's gone. Then blade the audio there and retime the audio to sync for the rest using audition. Works, but the output is then VFR, and may even be faulty when played back in some players.
2. Fix missing frames if possible by transcoding or creating your own blend. In AE you can do this pretty easily if you know which frames are dropped, and the direction of motion of the frames or the objects in them. You can then adjust the objects or frames as necessary, and create a blend here and there to blur them (start by clipping that section out, then adding it to a comp where you split the frames into images, first by rendering out to images; add the missing frames by estimating the movements and using either the frame before or at the end of the drop, one that isn't dropped, and then add frames between those that are blends in between. The wider the gap where frames drop, the more start points you should fill first, then blend between those all the way through; You should end up with 2x your missing frames +2 for the ends; now export that to a finished product at the proper rate, and turn on frame blending, which will blend them all further; this will give you the proper clip to replace your broken one with in premiere).
OR In AME you can fix short drops with "Use Frame Blending" and a transcode. It really does an excellent job.
Compressor will retime footage if you use the retiming box, stay away. Instead, set your output frame rate exact, don't use auto. Handbrake does well, also, but I haven't used it; only seen results from others.
For all NLE's:
With multi cam footage, you can simply clip to another angle, but your other camera will go out of sync. If you mark the area where the frame is dropped with edits, and delete that whole second, you will get a broken up clip. You can sync the 2nd section of this to the rest of the audio in one operation, by making sure that you clip at the 0 frame of the second, and where the last frame should be (if one frame is dropped, you clip at last frame #-1, if more than one is dropped you just subtract the ones dropped and clip there). If you have to cut 2 seconds this way, do it, but line up the now separated clip with where it should begin (at the 0 frame of a second), and you'll be back in sync with the audio. Note where this occurs in the sequence timecode, and make sure you work around it in the nested sequence.