Just in case it helps anyone else looking for this, here's the solution that I found:
1 - Drag your footage into a new comp
2 - Create an adjustment layer above your footage
3 - Apply TimeWarp to the adjustment layer
4 - Add a slider control to the adjustment layer
5 - Set the TimeWarp Method to 'Whole Frames' and set Adjust Time By to Source Frame
6 - Apply this expression to Source Frame: time*29.97+effect("Slider Control")("Slider")
7 - In the above example, 29.97 was my framerate, but use whatever fps you have
You can now simply keyframe the slider control to create your jumps in time. Personally, I set two keyframes about four frames apart. The first was linear, and the second had an ease-in applied. I simply timed the keyframes to the beat of the song, and it looks great.
Hopefully this helps someone else!
http://vimeo.com/explosivegraffix