Custom firmware also lets you run digital backups of UMDs you already own, assuming you have a way to make backups (and if you already own UMDs, chances are you already own an older PSP, so there you go). Again, notpiracyhooray!
installing custom firmware on it can bring emulators and homebrew.
Note that I am not advocating piracy.
So, not only is the Go perfectly capable of playing nearly the entire PSP library, it's also smaller (which, personally, I prefer, despite the slightly smaller screen), has 16 gigs of memory built-in, and you can enjoy video-out from the comfort of your couch with a Dual Shock 3. Go is the only model of PSP that can be paired with a DS3 (EDIT: can the Vita do that? I honestly don't know). In fact, if you're playing a PSOne game on a PSPGo with a DS3, you can set it to take advantage of every button on the controller, dual analog sticks and four shoulder buttons and all.
I've beat this horse to death in other threads, but I'll say it again: the PSPGo was a stupid value proposition at $250, but that doesn't mean it's not an awesome system, and a no-brainer at $50.