Your source is HDV which is quite noisy (even if you don't see the noise).
Recompression (H264) will add noise.
Try some Color Correction, crunching a bit the blacks and if you have some decent de-noise filter, apply it.
rafael
http://www.nagavideo.com