ANY body of water, (that is healthy) has 4 good years and 3 bad years in a run of 7 total years. Might be 2 good, 3 bad and 2 good, might be 3 good, 1 bad, 1 good, 2 bad, But on average, you will see 4 total good years and 3 bad years for crappie in a 7 year period on any healthy lake. Crappie tend to regulate themselves sometimes during spawns, water levels, weather, whatever, you aint gonna have 7 banner years in a row. You may be experiencing that to some degree. For instance, when crappie have that "bad spawn" it really eliminates an entire age class of crappie, almost skips them. Each body of water will have a limit to how big those fish can get too. (outside of the occassional freak). But lets just say for arguement sake that at peak growth, that fish will get to 2.8lbs. That takes (depending on the body of water) 6 to 7 years. Now, they can live to be 10 or 11, but wont get much bigger depending on genetics. Food is almost irrelevant, they either have enough to eat or they dont, its genetics that determines ultimate size. Most guys dont realize how LONG it takes a crappie to reach that max size limit, most guys think they dont live that long. Most guys are wrong.
So if you have a small pond and you have 2 good years of HAMMERING the larger fish...AND you had a couple bad spawns a few years prior, well, those age class of fish that SHOULD be replacing thos big ones you been catchin...aint there. Not so noticeable in say...Grenada Lake, REAL noticeable in a small barrow pit.
Most of this I got from a guy that developes ponds for a living and wrote his thesis at the University of Auburn on crappie. Hes a great guy to talk to about this stuff. He runs Southern Pond Management.
Of course I dont know for SURE this is what is happening to you, just thought it might be a possiblity. Lots of factors witih crappie and ponds. Good luck.