If you’re talking to the man on the street, “thrower” could have many meanings, probably none of which have anything to do with yoyos. “Yoyoer” is also somewhat silly.
As @eternalmetal says, “I like to yoyo” is very descriptive, and leaves little to the imagination.
QED