Minor alignment problems with the axle, or the bearing being loose around the bearing seat and thus shifting in position when screwed together, have magnified effects on a yo-yo. The center of the yo-yo is basically where all meaningful “vibe” comes from.
The exception is bimetals, where the rings can be misaligned, etc… but on a monometal, the center is the whole game.
I used to think that too, until I ran these tests. Science. It frickin’ works.