Can you please tell us exactly what manufacturer & model of USB preamp you have?
I've never heard of phantom power damaging a condenser microphone - a ribbon mic, yes, and maybe some cheaper/ill-designed wireless mic receiver outputs, but never a condenser mic.
The biggest issue I normally face is making sure the channel is muted so that the loud "pop" doesn't blow a HF driver (though I've yet to have that happen, either). I guess it's more of a precaution. Better safe than sorry.