How to handle green surpression through glass and fire?

I've got a semi-complicated key to pull, with the following bits...

• Glass
• Fire
• Fast motion

So...that's exciting. :)

Here's a photo of where I'm at:



First is the original footage. As you can see from the track marks, the 100 frames or so of this particular shot is all a fairly quick motion from the camera.

The second shot is the footage I'm putting on the green square (I haven't added any motion blur to it yet), it also shows my garbage matte around the candle.

The third is as far as I've gotten with Keylight and gets at my ultimate question here.

How can I surpress the green in that last shot? All my attempts at cleaning up my key just result in complete transparency, which doesn't look right because the glass isn't actually completely transparent.