I lived in Tokyo for a year. In general, you will pay a premium for recently released games, new or used. But if you're looking for stuff that's a generation or more old, you are in luck, my friend. Even crazy awesome retro stuff like Chrono Trigger or Final Fantasy 6 are only 300-500 yen (3-5 dollars). There are a bunch of great stores out there selling used stuff (and used is generally pristine shape, so don't worry about that) quite cheaply, but my favourite was Book Off. Sells tons of great used games, and there are stores in almost every neighbourhood (Akihabara included). Also sells CDs and books and such, but c'mon, it's all about the games.
Oh yeah, it will help if you can read a bit of Japanese, mainly because used games are almost always accompanied by either a sticker that completely explains what is included and what shape it's in, or just an explanation of what is not included for some stores. If you can't read any, and don't know anyone that can, then if you see a bunch of games that just have price tags, and some with price tags (maybe lower than normal) but also a sticker with writing on it, the writing probably describes something that is not included (manual, box, etc). That's just a general guideline though, it's not always the case.
