I've noticed this problem over the years but don't know why it happens.
Sometimes when I play a timeline at double speed (hitting 'L' twice) I can distinguish the audio without a problem. Other times the audio is hard to understand...choppy and stuttery. It seems very arbitrary.
In my projects I have up to 150 timelines of podium speakers, each giving a talk around 8 minutes long. I have to check each of these for content, so double speed is essential!
I've tried quitting all extraneous programs to free up the processor, among other things (making sure waveforms and keyframe overlays are turned off...). Can't seem to find a "that explains it" solution. MacBook Pro 2.2 GHz i7, 8 GB RAM running OS 10.6.6 and FCP 7.