It depends on the day. I've seen days where the fish wont touch it unless you let it still for quite a while. Then again I've caught many fish moving around like you describe. Sometime the movement of the bait triggers something in the fish to strike. Often times crappie will strike a bait either on the way up or down. How many times have you grabbed poll that has been sitting idle for a long time only to have something try to take it out of your hands? Other times they need to see a much more natural presentation before they are persuaded. The best thing to do is try both whenever you fish. Whichever proves more productive is probably the best bet...till tomorrow:D