Sometimes, the bounce physics do strange things. I see you have the layer set to infinite plane, which is correct. Try this, change the floor layer to none and change the wall layer to your solid layer and see if that makes a difference. Also, sometimes if you are using a layer, box or grid emitter, you set the z depth to greater than 0, you can accidently have particles being emitted below the bounce plane, which causes errors. Check those things and let me know if you are still having issues.