Can you send us a screenshot (or send me a clip of your video)?
Note that when you export a Quicktime movie with the Animation (Millions+ colors) codec,
it will look very aliased in the Quicktime Player application. This is because the Quicktime player does not properly show the alpha channel. You may find that the video looks fine once it's imported into Avid Media Composer.
If this isn't the case, feel free to send me your movie (or a portion of it).