I don't dislike the PSP, but I don't own one either, and I could easily afford it.
I LOVE my DS.
I guess the thing with PSP for me is that those types of games I like better on the console. I'm not looking for open world 3D gaming on a portable. I'm not looking for God of War, or Gran Turismo on my work commute.
I love the DS style of game, 2D, slower paced gaming. I love DragonQuest, and Pokemon, Castlevania, Puzzlequest, etc.