Switches work fine with Macs and PCs as long as you have an external monitor. So no, the iMac won't work out.
But with Mac Minis or Powermacs you'd be fine. My mom has a Dell and a Powermac. There is a logitech mouse plugged into the USB port on an Apple keyboard. The keyboard and the monitor goes to a USB/VGA switch that runs to the PC and the Powermac. Everything works just fine on both machines.
Using a USB keyboard is easiest since it works with both systems, but you can use a PC-style one instead of a Mac keyboard if you like. Pick the one you use more often. She went with the Mac-style because of the built-in USB hub.