audio problems with multi-cam

I am creating a multi cam clip from two cameras (HDLR's) and audio recorded on a separate Tascam DR-40 recorder. Sync everything up and edit with selecting the audio from the recorder and just cutting video back and forth from the two cameras.

All works except that when I switch from camera to camera there is a slight click or even an audio quality change. The Tascam audio is selected throughout the cut and doesn't change (the only other audio is from the HDSR's internal mics and would be obvious if that was accidentally selected).

When I play just the audio file from the event browser, all is seamless and perfect. But on the timeline it sometimes changes quality or has soft 'clicks' audible, even in the rendered out file.

Anybody else have this problem or a workaround? Why are the video edits 'bleeding' onto the audio track?

Thanks!

Both audio file and project are set to 48kHz. It is an intermittent problem though - some projects don't have it while others (multiple EDL's) have it.