The main defense against it is that it would be a waste of time/money, as the Fed isn't doing anything wrong ever. Or something.
Aside from that, it's a crazy random happenstance you'd ask - because I was just reading an article published by Eichengreen yesterday on why we shouldn't audit the Fed.