Sadly, you'll be missing out on some of the PSP's best games as they only exist on UMD or never had their digital release ported. It also seems they've pulled a bunch of digital releases over the years as I can't find them anywhere now.
But anyway, here's what I recommend after having a quick scroll through the list of Vita-compatible PSP games:
Final Fantasy III
The 3rd Birthday
Dissidia 012 (I've only played the first game, but I hear this makes it obsolete)
Riviera: The Promised Land
Jeanne d'Arc
Castlevania: The Dracula X Chronicles
WTF: Work Time Fun
Half-Minute Hero
DJ Max Portable 3 (haven't actually played this installment, but the older ones were great -- of course, DJ Max Technika Tune for Vita releases tomorrow, so if you'd rather have a touch-driven rhythm game, get that instead)