Yes, obviously, you do need to keep all of your records and such but if you do it regularly it's not THAT hard/ I've been doing it and properly paying my taxes for so long that it's all second nature now.
Don't do it, and you run the risk of getting audited, which is slim, but if you do...