after 6 months experimenting with my new SONY Handycam which delievers AVCHD videos, I finally cut the project with Final Cut (including 5.1. surround) and worked myself through the Encore Handbook (because DVD Studio Pro can't handle Blu-Ray authoring). Finally I bought myself a Blu-Ray Burner and did some more or less sucessful tests.
Damn, it's hard to be a early mover and NOT doing authoring as a profession :-)
Long talk, short sense:
I send my 85 min. cut from Final Cut (imported as Apple ProRes444 interlaced (upper halfframe) to Compressor using the following adjustments:
1920 x 1080i (upper halfframe)
I thought Encore will not transcode that file (as it doesn't do with MPEG2s or Ac3s), but it does.
Is there something wrong with my presets? What should I adjust to
unless you give Encore h.264 or m2v it will re-transcode. You said you made Quicktime h.264 - this is not the same as mp4 if your file is an MOV (I speak PC by the way where we use file extensions). So maybe Encore thinks the MP4 you made is an MOV (which can be a container for MP4 but Encore may not know that).
Try changing the file extension..? Not sure this is a trick you can use on a Mac. On a PC you can trick the computer into thinking one file format is another by changing the extension, which can be enough to test if this is your problem or not.