I got a video layer with a Lens Blur effect. I only wanted the effect to affect only a few seconds of it, so I keyframed its Iris to 0 when its not needed. Despite that, the rendering of the non affected frames are also as slow as hell.
Alternatively, you could split the layer so the blurred bit is on its own layer. This is sometimes an easy way to do it and you can find the part easily when looking at the timeline. But it can complicate things if there is a lot of stuff happening.
One of the great things about AE is that there's a million ways to do everything ... you could also put the blur effect on a short Adjustment Layer above the portion of the footage you want blurred instead of putting the effect directly on the footage layer. Keep in mind that Adjustment Layers affect all layers below them so, if you have multiple layers at the same point in time the blur needs to occur,you should probably use one of the other methods.