You are using baits that channel cats eat. If you want to catch blues or flatheads, use natural baitfish such as shad or skipjack or bluegill.
If you want bigger fish, use bigger baits. Big fish do eat small baits, but small fish can't get big baits in their mouths.