My 2006 Limited also developed similar issues as you described. The mileage was around 160,000 at the time. On a hunch, I had a great local mechanic replace the front half-shafts (axles). Thes have two Constant-Velocity joints per axle, as I understand, and the noises and vibrations reminded me of U-joint problems in old lifted Jeeps I abused.
The after-market axles, as I understand, are a great solution to fix torn boots over those joints. I've heard that a torn boot, from Toyota, requires the removal of the half-shaft, then the boot goes back on the old axle, and you still have high-mileage joints in that axle, only with a new dust boot over the old joint. Toyota, so I heard, doesn't sell a complete half-shaft like the aftermarket vendors offer.
I chose to do both axles at once, because both sides have the same wear, and the labor saved was worth it. IIRC, the half-shafts were around $180 each, and the labor was about $300, and problem solved.
Good luck!
-Warren