The only physical requirement for a “sport” is that the human body is integral to the process. A sport doesn’t need to be really physically demanding, it just needs to be a test of physical prowess. This excludes things like chess because, while it is a competition, the competition would not be affected if the human hand weren’t required to move the pieces. It could be done on a computer and still be the same game. Not true for archery, so it is definitely a sport even if a child could hold the bow back.
Yoyo is definitely physical enough. Your physical skill is everything. It generally doesn’t meet the requirement for a sport in the competition aspect.