Did you precompose the layer with its masks? Often, when an effect seems to be ignoring the masks on an input layer, this is the solution. Precompose, and use the precomposition layer the matte layer.
I'm not in front of After Effects right now. If this doesn't work, I'll see if I can help out when I can play along in After Effects.
BTW, the After Effects documentation is deliberately sparse on a lot of the details in Timewarp, since the folks at the Foundry have already documented it (as Kronos). That's why one of the first paragraphs of the
"Timewarp effect" section points to the Kronos documentation. Of course, if this isn't adequate, I'd like to know so that I can fix any deficiencies or defects.
---------------------------------------------------------------------------------------------------------
Todd Kopriva, Adobe Systems Incorporated
putting the 'T' back in 'RTFM' :
After Effects Help on the Web
---------------------------------------------------------------------------------------------------------