The GBA is one of my favorite systems, great pickup!
If you really want to experience the handheld to it's maximum potential, I'd pickup a pearl blue/pink or graphite gameboy sp. These have the brighter backlit screen, which in my opinion, made the GBA a lot more enjoyable for me. As a kid, I always had to play directly under a lamp or something because the screen is simply too dark. Play a game like Castlevania: Circle of the Moon or Metroid Fusion with the original GBA vs an sp, and you'll never want to pick up the GBA again. That said, I did miss how comfortable the GBA was in my hands.
Anyways, here's the games I'd recommend:
Advance Wars 1&2
Fire Emblem, both of them
Golden Sun, both of them
Metroid Fusion/Zero Mission
Pokemon Emerald
Pokemon Leaf Green/Fire Red
Astro Boy, this game surprised me the most in how much fun I had
Ninja Five-0, old school gameplay reminded me of genesis games
Final Fantasy Tactics Advance, easy game but lots of fun building your team
Tactics Ogre
Super Mario World
Yoshi's Island
Castlevania, if I had to pick one I'd go with Aria of Sorrow or get the double pack
Boktai, great time to pick this game up
Warioware
Bomberman Tournament, multiplayer is lots of fun and the campaign was enjoyable
Mario & Luigi
Riviera
If you enjoyed Sonic as a kid, Sonic 3 was the best one in my opinion.
If you enjoyed Mega Man, the Battle Network games are get repetitive but I'd recommend playing one of them. I played through 2, 3 and 6; I'd get any of them but 6 had the best ending. I didn't really enjoy the Zero games though.
Some people enjoyed the Breath of Fire games, but I never picked them up.
The DBZ games are fun if you're liked watching it.
The Mario Kart game was pretty lame, despite good reviews; the worst one I've played for sure.
Tony Hawk was fun, I remember completing all the gaps in THPS3. I'd get one of them.
The GBA is backwards compatible with GB games of course, so feel free to pickup anything you may have missed then. Enjoy!