Its been like 2 weeks since I have started practicing “fulltime”. That means I am practicing a LOT. I had the same concern in my mind when I thought of telling this to my girlfriend or my parents. I am an IT analyst at a biga$$ auditing firm. But now I dont care. Most of the time YoYo is like a stress buster for me.
And yeah a few people did comment about it in my office (I don’t like those people) when I was playing my arctic 2 in the elevator. My answer was “Dude, my yoyo is costlier than both of your ties.”.
But then I was like: “why would I care? I like doing this and it is as simple as that”.
I try to add multiple dimensions to my personality. Yoyoing is one of them. Feel proud about it. Feel good about it. We are in the world’s most advanced country. Not judging someone on how are they looking or what their interests are, is not a culture here. When people can color their hair with bright pink color and can go to walmart and feel good about it, heck yeah I can play yoyo in public 
Just do it. Who cares what people think. I throw everywhere school, work, waiting for food, it doesn’t really matter who watches. The average joe can’t do basic yoyo tricks. If people hate on you so what. I live in San Francisco, filled with yuppies, hipsters, etc. if anything throwing in public just shows people that there is more than walking the dog.