Hmmm, I'll keep it to exclusives in the first list (these I've played):
Gravity Rush
Uncharted
WipEout 2048 (which if you have the PS3 WipEout HD+Fury gets you an insane amount of DLC)
LittleBigPlanet Vita
Dokuro (PSN)
Unit 13
Super Stardust Delta (PSN)
Lumines
escapeVektor (PSN)
Touch My Katamari (nothing ground breaking, but it's fun and it's Katamari on the go... neat tricks incorporated via trouch screen for squishing the ball in various ways)
Treasures of Montezuma Blitz (free! Plays like Bejewled, sorta.)
Part of why I like the system though is because it offers quality ports of PS3 titles I like and I can play them without needing a TV. Here's a list of other games with notes:
Mutant Blobs Attack (on Steam, but intended for Vita originally. Phenomenal.)
Rayman: Origins (console perfect, pretty much)
Sound Shapes (cross-buy on PS3, but developed for Vita originally; interesting music driven platformer... not a must buy for everyone, but it's worth if it you find it on sale)
Blazblue:CS2Ex (pretty much console perfect)
UMvC3 (again, 60fps, pretty much console perfect)
MK9 (gameplay is console perfect, all kontent, extra challenge tower with Vita gimmicks, character models and textures took a hit)
Sine Mora (pretty much the silky 60fps of the consoles/Steam; awesome game, great to have it portable)
Ninja Gaiden Sigma+ (it's okay if you're the kind of person who can deal with 30fps action games... some off-putting design choices regarding bow aiming, but a solid fix for action games on the go if you can handle the lower fps)
Zen Pinball 2 (cross-buy, solid arcade-y pinball title with lots of tables to choose from)
Urban Trial Freestyle (pretty competent Trials game on the go)
Oddworld: Stranger's Wrath HD (pretty much same as the PSN/Steam/XBLA but portable)
Sony Smash (if you own the game, it's pretty much the same thing and you already own it via cross-buy)
Motorstorm RC (if you liked the old Off-Road games, this is great, cross-buy with PS3)
Guacamelee! (phenomenal Metroidvania brawler -- I've put like 13 hours into it over the past two days and can't put it down. Cross buy with PS3)
Metal Gear Solid HD (transfarring if you own them on PS3 they play and look amazing)
Honorable mentions that aren't exactly "my" recommendations:
Hot Shots Golf (if you like the game, it's supposed to be wonderful)
P4G (the definitive version of Persona 4)
DoA5+ (if you like DoA5, it's supposed to be good)
Add in a bunch of PSP titles you can buy off PSN and PSOne games on the go.
Also, apparently Sony is making it an Indie machine, so it's getting a

ton of the indie games you'd see on Steam or something like that. There's a bunch on it with a bunch more coming. Nun Attack, Jetpack Joyride, Plants vs. Zombies, etc. Coming soon: Terraria, Frozen Synapse, Binding of Isaac remake, and a bunch of other shit I'm forgetting.
Another thing worth noting is that if you ever plan to pick up a PS4, all native games to PS4 are supposed to have remote play on this thing, making it like a WiiU gamepad for the PS4. Anything that's doesn't require Move or whatever, at least.
Also, if you have a PS+ sub, you get another 1gb of cloud storage for your Vita, auto-uploads, and of course, the IGC for Vita in addition to what you're already getting for PS3. Someone else already mentioned it, I think, but it's currently Gravity Rush, Uncharted, Tekken 6 PSP, Disgaea 3 Vita, and something else (or two) that I'm forgetting.