The PSP has a couple of advantages as a mp3 player, one of which is the RSS feed. I download podcasts straight to my PSP and the files never touch my desktop. I would say that it depends on what you're using the music player for.
If you're using it to work out then it might be a little big to lift weights with, unless you have long enough headphones and set it on the floor. But if you're on the treadmill then just set the psp on the console and start running. I've even watched movies on the treadmill with my psp.
If you're riding on the bus and just need to kill time, I usually just slip the psp into my coat pocket and it works just fine. The only problem with the size of the psp is if you need to be actively mobile while you listen, like when you're going jogging through the park, I'd probably go with a shuffle or something lighter. But if you're stationary then the psp is great!