FLV stutter problem (even locally)

I'm trying to create an flv for my site, but I can't get the video to play smoothly once it's in an html page. I've compressed using Flash CS3, On2 VP6 at 700kbps, 30fps, MP3 audio at 128kbps. The video is only 45 seconds and file size is about 5mb. I'm on a Mac dual 2.5Ghz G5, so I don't think it's a CPU issue. The swf plays perfectly smooth locally. But if I play it in the html page, it stutters like the frame rate got messed up, even after it fully downloads and even locally, so it doesn't seem like a buffering problem either.

In Flash I tried just using the FLVplayback component with and without controls. I also tried Sorenson Spark with the same problem. I've tried to view the video in Safari and Firefox on a Mac and IE on a PC. It did seem to play more smoothly on the PC, but still not as it should be.

I'm not very experienced with Flash, so hopefully it's just something simple I don't know about. Anybody know what the issue is? Any advice would be really appreciated. Thanks a lot!