Although if a battery is 4 years old I am likely to replace it anyway. Many fail with little to no notice after 3 or 4 years anyway.Why are you sure it's the battery? If you can get a jump I'd drive it to Autozone and have them run diagnostics on it to be sure. Sucks to drop ~$100 and find out that wasn't the problem.
I think it has been a couple of decades since I have an alternator fail. More often then not it is the battery now a days it seems.
I wouldn't think twice about replacing a 4 year old battery in a dead car. Unless maybe I left the lights on or something that drained it.