People throw around assault and battery interchangeably. Think of it like this: An assault is often called an "attempted battery," while battery is often called a "completed assault."
Throwing the brick at a car is goes beyond assault both criminally and tortuously to battery because the car is considered closely connected to the people inside. (However all of the elements of the crime or tort - depending on who is doing the suing here - still need to be fulfilled.)
Criminally, assault is necessary to have a battery, and you can be convicted of both but only sentenced for one. And OP has no defense of self-defense because throwing a brick isn't going to be seen as necessary and proportionate to the verbal harassment.
In tort law, assault and battery are separate things and because he threw the brick at the vehicle closely associated to the persons inside, its battery, not assault.
OP should have called 911 or simply diverted his walking path. If the people in the car stopped and got out and came after him, only then would throwing the brick possibly be legally justified.
Realistically, OP needs to find a new walking path because they'll likely cruise by again and you don't know if they'll do something stupid like bring a gun this time (and OP is lucky they didn't do that last time).
And of course, what OP did from a vigilante justice standpoint sounds cool but there's a reason vigilante justice usually only plays out successfully in fiction.
Finally, anyone with the means to leave Bakersfield, CA should leave Bakersfield.