How are your monitors calibrated. Setting levels for surround is vERY tricky. You need to have a fiull surround monitoring setup that is all calibrated with a spl meter. Only then can you know who much gain to give any channel.
But remember that LFE stands for low frequency effects. the only element that should ever be routed to that channel are sfx that have a lf content. Never voice or music etc.