I have it happen to me too but as a longliner I just troll around 'em and keep going. It can upset you if you let it but then that ruins a fishing trip. There's nothing wrong with saying something to them but I wouldnt say anything in a nature to start even more trouble. Now after that being said...I've mellowed ALOT in my years. I used to get ugly with some of 'em.