I've compressed an mp4 H.264/AVC file in AME. The video looks great especially compared to the same bitrate and size of the Squeeze Main Concept equivalent (and I do mean WAAAAAY better).
However, when played in QT Player 7.6.4 the first frame shows as black instead of the first frame of the video. When you hit play it does start at first frame, but appears as a sort of jump cut from the frame it had displayed to the first frame of the video. When you've played it for a few seconds and stop and try and take it back to the start of the movie, it will never get to the first frame even though the counter says "00:00:00". Hence, another jumpy looking start when trying to play it using this method.
This behavior is not seen in QT Player X.
What is wrong with the movie file created by AME and how can it be fixed?
I've tried this test on two different Mac computers noted below.
MacBookPro 17" (2008)
Mt Lion 10.8.5
QT Player 7.6.4
QT Version 7.7.1
QT Player 10.2
Mac Pro (Late 2013)
QT Player 7.6.6
QT Version 7.7.3
QT Player 10.3
AME CC 18.104.22.168
Sounds more like a problem with QT. Have you played the file in any other media players - MPEG Streamclip, VLC, Flip player, Cineplay (an iOS media player), etc.
I don't trust QT7 for too many things. It has quite a few bugs and sometimes displays the beginnings of my videos incorrectly too. Confirm other media players produce the same result before blaming AME.