Why does this not work like it does it DVD studio pro and almost every other dvd authoring program where you can just have a flat background image as your menu, then use individual flattened full frame black and white overlays with the same alignment as your background to use to designate selection highlights for your buttons? IS there a workaround?
Encore's menu editing system is based on Photoshop files. Everything is based on layers in a PSD files. Your button elements need to be turned into layers.
You don't necessarily need to cut out the button areas from your background image, but you at least need to create a layer set for each of your buttons that contains the button region.
If I'm understanding what you have already (and I've never really used DVDSP), here's what you want to do:
1. Open your background image in Photoshop.
2. Add each of your button overlays as layers.
3. Get rid of the empty space in the overlay layers.
4. Put each of the overlay layers in it's own layer set. Make sure the layer set name starts with "(+)", which is what tells Encore that the layer set defines a button region.
5. Inside the layer set, turn off the visibility of your overlay layer.
That should do it (again, if I understand what you're starting with). Note that this won't create highlights for the button, but you can do that in Encore. Select the button, then choose Object > Create Subpicture. That'll give you a basic highlight, you can change the subpicture color sets and/or you can tweak it further in Photoshop.