Hi Paul,
I'm not entirely sure my advice will prepare video for both RTMP and HTML5, but at least test it out. At the very least you will no doubt get some improvements in picture quality.
I can highly recommend using the x264 codec for H264/mp4 format video rather than Apple's default H264 codec found in QT and Compressor. I've recently switched and seen far superior results.
http://www003.upp.so-net.ne.jp/mycometg3/
While it can be plugged into QT and Compressor, I've seen much better results when using x264 in third part software such as Handbrake and MPEG Streamclip. Mind you they seem to be a heck of a lot faster than Compressor too. So faster and better = good. :D
Handbrake (which uses x264 by default) has the option "Web Optimized". When this is ticked
the header of the MP4 file is rearranged to optimize it for streaming across the web.
I'm not sure of the same option in MPEG Streamclip, but I'm sure it would have one.
I'd suggest running a small test file through handbrake and see how it goes.
