OK, I'll start small and get bigger.
In my college town, New Paltz, NY there's a couple places:
Jack's Rhythm's - 54 Main St, New Paltz, NY
Good but small selection of vinyl and CDs. I bought my first Teengenerate LP there and some of the 45s I distro (The Playmates, Radio Shanghai, Trio) are on shelves there and some CDs like The Fadeaways, Teengenerate live and The Dogs Tribute.
Rhino Records - 3 Church St., New Paltz, NY
Great selection of all types and lots of garage, surf, exotica, punk and indie on vinyl. Waaay back in the day it used to be affiliated with the Rhino label.
In NY city, I can recommend two places although there are plenty others, like Bleecker Street Records, Academy etc.:
Other Music (Manhattan) - www.othermusic.com
Mostly out music, avant garde, experimental and indie rock, but there's also some punk, garage, electronica and indie on vinyl. I put some experimental CDs there (Flight of Idea, Six O'Minus, Varo).
Passout Recordshop (Brooklyn) - http://www.myspace.com/passoutrecordshop
Mostly vinyl punk, hardcore, garage, exotica, rockabilly, psychobilly, etc. I put some 45s (Playmates, Radio Shanghai, Trio) here and some CDs (Fadeways, Teengenerate live, The Dogs tribute). The owner Pee Wee is also in the punk band I.C.U.
In Japan there are tons and tons, but I'll give you a short list of the more garage punk ones:
Disk Union is a chain that has everything including ample vinyl selection depending on the location. Shimokitazawa is the largest I've seen, but Shibuya has a good punk shop.
Japan Vinyl in Shinjuku, two shops, one dealing in more classic rock and mainstream and the other selling more udnerground punk, indie stuff. They bring bands like the Rezillos, The Boys, The Only Ones, Pretty Things, The Undertones to Japan.
Record Shop Base, another chain with punk, hardcore, garage stuff. The Tokyo shop is in Koenji. They reissued and released Nikki Corvette stuff in Japan and more recently the Baby Shakes and Test Patterns.
Senseless Records in Hachioji carries more punk (pop punk, hardcore, post hardcore, oi!, ska) but you can find the odd ball country, garage, neo-rockabilly LP. Run by Tsuchiya from Peace of Bread and it's connected to a studio which has regular events on the weekend. I distro Armedalite Rifles LP and other upstate NY punk here.
Micky Room in Yokohama, very tiny place in a seedy apartment near Yokohama station. Lot's of underground hardcore, classic and rare punk to find punk from all over the world. I distro Armedalite Rifles LP here as well and other NY punk stuff.
Last might be the best for this crowd. TIME BOMB RECORDS. I haven't been to Osaka yet, but I'm dying to check this shop out. Simply the best place to find garage punk, Japanese or otherwise. The Time Bomb Label has released stuff by bands like 5678's, Acid Eater, Mad3, Supersnazz, First Alert, etc.
There's plenty of other places I haven't check out or I can't remember the names, but these are the ones that stick out.