I found 4-1 relatively easy--if you can get to the boss, then just go for it since he's probably the easiest in the game. I think I was about SL40 too when I did it, too. For me, the Golden Skels were more difficult since you have to fight them on the cliff and there's really no room to roll/dodge. I ended up using magic to get through that part since my attack power wasn't too good, I didn't have a mace/blunt weapon, and I just didn't want to spend a lot of time fighting them. I keep thief's ring on pretty much all the time, so for the black skels, the first one I can kill either with arrows from below the ledge he's on, or just bait him into the long corridor and backstab him when he gives up and walks back. The second one, I just lure him into the courtyard so I have more room to dodge/roll. The third I can kill w/flame toss, but there's enough room to melee if you want to just turpentine your weapon and have at it.

Right now, I'm just farming the easy reaper in 4-2. Rescued Urbain, but I don't feel like seeing the entire level through just yet as I'd like to have a bit more attack power/vitality to get through it. I might try finishing 2-2 first as I'd like to get better weapon upgrades, then maybe try 3-1 to at least rescue Freke.