I have thrown dozen's back over brush piles I fish. I have never had the problem you described. Maybe because the brush I'm fishing is in 15' to 25' of water ? There are a lot of other things that will spook fish besides throwing one back on the spot you are fishing. I say whatever works for you anyway.