Were you spider rigging when he ran at you or were you under power? If you were spider rigging he would have to pass by on the available side since he is more maneuverable. I doubt the "port to port" rule could apply when slow trolling. When traveling under power, expectation is port to port passing with oncoming boats, unless obvious they are on different parallel paths.