The referee is talking nonsense. As Phil says, you're being courteous enough already.
For me, it depends on where the person following is in their routine. If there enough time before they have mounted the gun, I chuck em in then. If not, I wait until they have shot. But never when mounted and...