I installed a new font called veneer, and it took me a couple of hours of wondering why AE was running so slowly (particularly previewing and making even the smallest of changes) to think that this could be the problem. As soon as i uninstalled it, AE returned to normal.
In this overview of the font, (Attached at the link below) it specifies that it could make some applications run slowly. what I would do is install veneer back on and do all your text work in Photoshop at your video resolution and then export it out as a PNG to then work with in After Effects. once it is down to a manageable size and set to a fixed alpha for the distress then I think Ae will no problem working with it.
From what it looks like this font is very hi-res so probably not vector. It also has a large number of custom features added into it, like multiple grunge type alpha channels, several anti-aliasing options and so forth, so that is a lot of stuff for a font. So I say flatten it down with its alpha and then bring it in.