I have a GP2X and I really am quite happy with it, I got a 2gb SD card for $40 from Newegg and it's really a great combination. The video is pretty nice, not having to convert anything is great, just pop it on the SD card and play it.
The emulation is sort of where its lacking a bit, I haven't tried all the emulators yet, but the ones I have tried so far are still in the early stages with the excepion of a master system emulator and a GBC emulator. They both work like you'd want them to, 100% playable. I was hoping for a playable NES emulator, but the ones that are out right now, either have messed up sound, play too fast, or too slow. Theres a SNES emulator that shows some promise, i think with some further development that it will be 100% playable. There are tons of other emulators for it, I'm just talking about the ones I've tried.
There are a few bad things about the GP2X. For one the batterly life aint so hot. Watching video really makes the CPU work, so the battery only lasts about 2.5 hours. Just playing some emulators it lasts a bit closer to 5, but it's still a good idea to have a ton of spare recharable AA's. The other bad thing is the control stick, after doing some forum searching I found out that the stick has larger contacts for the diagonal inputs than the straight inputs. So when you are playing a game, you almost always are hitting a diagonal. And the stick nub is just hard plastic, and not very textured, so it's easy to lose your grip, luckily though you can use those Naki PSP replacement nubs and pop on, a rubber grip is quite nice.
Another problem is with the older firmware windows has a hard time recognizing the GP2X when you connect it with a USB cable, I had to use a special program I found on their site to be able to put files on the SD card through the GP2X, I got sick of that so I used a SD to USB adaptor, but luckily with the recently release 2.0 firmware that is no longer a problem, the system boot times used to be fairly long, but they also shortened that with the new firmware.
Even though the GP2X has it's problems, I would still recommend that you buy one, especially now that Lik-Sang has them for $150, get that and a 2gb SD card from Newegg and I think you'll be quite happy.
Edit
I took a crappy pic for you, comparing the GP2X screen to the PSP.
