Very true, but I can usually get loose pretty easy so just don't loose many. I keep a sinker with a bit paper clip and a string attached near and I can slip that on and get most back with out much disturbance to the structure.
First though I just try to wiggle it loose and that works way more than you would think unless you have berried the barb in the wood. Then my Power Pro will get it back a lot of the time.
Skip
Hand tied jigs are always on #4 sickles in my boat. I use #2's and even some #1's with plastics to ensure enough hook gap for consistent hook-ups.