I’ve only played 6, 7, 8, and 9.5 (plus Scarlet Weather Rhapsody, but that’s a fighting game and not danmaku), so this is based partially on what I’ve seen and read of the other games.
I’d probably start on either Perfect Cherry Blossom or Imperishable Night. They both have unique gameplay mechanics that help the player out (borders in PCB, which can effectively give you a shield for a short time; Last Spells in IN, which make it much easier to death-bomb, but costs 2 bombs instead of 1). I think they are generally considered the easier tier of Touhou games along with Ten Desires. Ten Desires also has a unique mechanic that helps the player (trance mode, which gives you invincibility for a short while), but it is harder/more complicated to collect resources (extra lives/bombs) in that game.
One downside to starting with IN is that it can give you bad instincts for bombing in the other games, because the last spell system means you can wait longer to decide to bomb in IN. Learning on another game first might also help you avoid using the last spells in IN, which is good because last spells use up 2 bombs. So maybe I’d start with PCB, but IN would also be a good choice. PCB also gives you the option to start with 4 bombs if you play Sakuya, as opposed to the standard 3 (SakuyaA also has a wide unfocused shot range and homing focused shots, and has decent power for having range and homing, so she’s a great beginner shot type). IN gives you the option to start with 7 lives instead of just 5, and it has a spell practice mode that means you can practice any spell card you’ve seen by itself without having to play through the whole stage to try to learn it. Spell practice is extremely useful and makes learning tough spells less of a grind. I think the only games that use it are IN and Ten Desires.
If you want to start with the beginning, Embodiment of Scarlet Devil is the first Windows game (the first five were all for an older Japanese OS and have a more 8-bit feel). It is harder than PCB or IN, and it doesn’t give you some of the advantages those games do. It is more or less the basis for the later games, though. The later games all have some kind of unique gameplay gimmick (like cherry points in PCB or time points in IN), which sometimes means you have to learn the system to get good at the game. EoSD is pretty much just about dodging stuff and surviving through the game. So if you care about starting at the beginning or seeing the basis the other things built around first, you might want to start with EoSD. Otherwise I think PCB or IN are better starting points.
The games are sold through Japanese retailers. Some of them have English websites and ship overseas: http://touhou.wikia.com/wiki/Buying_Stuff
Fan-made English patches are also available: http://touhou.wikia.com/wiki/English_patches
