There are advantages and disadvantages of both. The "Slip Hub" in a stainless steel prop is supposed to reduce the risk of damage to your lower unit if your prop strikes a solid object. If you hit something at high speed you do have slightly less risk of damage to the lower unit if you're using a alloy prop, which will absorb a little bit more of the impact AND be damaged a lot more than a stainless prop under the same circumstances. If your prop strikes a stump at low RPMS there isn't much risk of damage to your lower unit with either type.
You definitely don't want to run your motor at high speeds very long with a damaged prop but idling back to the boat ramp isn't going to cause any damage.
A stainless prop is more efficient than an alloy prop and if you're careful and go slow around stumps you shouldn't hit anything hard enough to hurt your lower unit. But it doesn't take much of a bump to damage an alloy prop to the point you'll have to get it repaired. Stainless props are generally more expensive to have repaired but much less likely to need repair.
I've had both and I'm using a SS prop now.