Bigger fish hang out in deeper water. My guess is you're not fishing deep enough.
Read some of Alphahawk's and LedHed's reports. They fish deep water. I don't report much but I do catch a lot of gills suspended over deep water.
If I could offer one tip to bluegill fishermen, it would be to "get off the banks".