If it is very quick, you suspect the downconversion to be bad?
I'm not sure what part of the entire chain does the downconvert, but you could try putting your HD movie in an IMX sd project, and render to a simple codec, see if you get the same speed.
Otherwise, pick any app. you like to do a good downconversion, put that in an SD project and export...
But, did you look at the outputfile on a CRT? Does that look right?
(I hate the look of SD on my gorgeous expensive HD monitor and switch back to a good old CRT when i can...)