What is the final destination for your animation? the web? Maybe you need to try to export (render is better) and flv, if flash is your final destination).
For swf to be small they need to be from Vector layers and raster effects like stroke are killers for file size, as are the jpegs.
Audio Waveform is a nice vector substitute for Stroke (just don't reference any audio in the plugin and it looks like Stroke). But the jpeg layers will always be raster and therefore big. The only options are AE solid layers or Illustrator (vector) layers in AE.
Here is the place to see which layer will keep a swf small:
http://livedocs.adobe.com/en_US/AfterEffects/8.0/help.html?content=WS387852...
See "Supported Features for SWF Export" at bottom.