I'll bet that your rig has a sinker above the hook. When this is falling you're in contact with the sinker, not the bait. Try changing your rig to a weighted jighead tipped with a minnow and I think your catch rate will improve. At least you might feel the thieves in action.