Depends I guess.
Years ago I put in several trees and stake pallets near our family dock. The first year they were pretty sparse. After about a year I started to catch a lot of shorts 4-5 inchers. By the 2nd year they were mostly 5-6 inches. Every year the majority of fish that claimed my brush/area as their territory, as it were, would grow and get bigger. About year 4 they were mostly keeper sized fish. It was really cool to catch them through the years and watch them grow. I should have kept planting more cover to replenish the decaying trees, but never got around to it - now we don't have access to that property, so no big deal.
Last July, at our new property, I put out several trees and they are just now starting to produce good size fish, but this area is in deeper water and closer to the mouth of a cove vice the back of a shallow cove at the last place.
So, yeah, it depends.
Swan-Diving off the tongues of crippled giants