The raw footage is 540 * 1080? That's an unusual size for a camera to output. Am I understanding correctly that the image is 540 wide by 1080 high? HDTV has a height of 1080, but the width would be 1440, 1280 or 1920. NTSC D1 square pixel would have a height of 540...
I'm wondering which it is because it may be that you are using non-square pixels in your raw footage. In your project panel, select the raw footage and take a look at the info displayed at the top of the panel. It will give the file's name, then below information on dimensions and pixel aspect ratio, e.g. 1280 x 720 (1.0). Check that AE is interpreting your footage correctly. That is, it's recognised the footage dimensions and aspect ratio and provided the correct figures. If it hasn't, select the asset and hit cntl+f to open the interpret footage dialogue box and change the necessary settings.
In your comp, did you rescale or modify your raw footage in any way? A lot of people, when they start to use AE, make a comp from a non-square pixel asset and don't understand why it looks horizontally squashed, so they stretch it until it looks right. Then when they render it, it looks bad! The reason the display looks squashed is because AE displays non-square pixels as though they were square by default. You can check what the output will look like by toggling the "toggle pixel aspect ratio correction" switch, which you can find on the bottom of the composition window, towards the right (the box with the arrow above it). I don't know if this is your problem as I don't know if / how you've modified your footage.
Anyway, as I said earlier, I imagine FLV will work best with square pixels. Even if your raw footage is non-square, you can still add it to a square pixel comp and it will work. AE will just crop the edges of the footage.
Other than that, I'm stuck. If you post a link to the vid itself, assuming it's live, that may help more.
Simon Bonner
youtube.com/simonsaysFX