Early in my fishing life I had a similar experience with a buddy who was using 4# and I had 8# test, both mono. I changed out to 4# and it didn't help. Why? 1. He was reeling about 1/3 as fast as I was so his jig was falling into the fish.
2. When I forced myself to slow down I missed 4 out of 5 fish because I was waiting to feel a thump, when he was watching his line go slightly slack and then he set the hook.
I had to learn to reel slowwww and watch the line like a hawk. Felt like an idiot setting the hook when I felt absolutely nothing, until I started catching fish as regularly as he did.