Michael,
I have now seen this playback issue while visiting a client using Mac and it has been reported to the Dev Team. We believe we have found where the problem is thanks to some other Mac users. It has been reported that is you disable the Hyper-threading the playback issue is solved. I know this is not a solution but if it can get you using Maya Composite it is a start. Try it and let us know what results you see.
You can find information about turning off the Hyper-threading at the following link.
http://developer.apple.com/mac/library/qa/qa2001/qa1141.html
Ken LaRue
Autodesk Application Engineer
Maya Composite / Toxik / Flame / Combustion
Training DVDs:
http://www.thestreetproductions.com