Income tax deductions

Quite often I'm asked what happens if I don't file. Isn't it against the law that we have to file taxes. It's always a good argument but the congress does have the power to levy and collect taxes so...