Let him fish. No need to be ugly to him, although I have the same primary response you are thinking. This happened to mme once, I struck up a conversation and he became a fishing buddy and taught me a lot on how to fish where I was fishing. You never know, it may work out to your advantage.