It isn't unfair. However, if we are to have an income tax, we must make a moral judgment as to whether we should tax someone when it could cost them "essentials" such as food, clothing, and shelter.
I think we should have a single standard deduction for individuals then a lesser one for dependent children or the elderly. Perhaps $30K for each non-dependent adult then an additional $10K for each dependent. Two people living together with 3 kids would then have a $90K deduction and pay 20% of all marginal income above that level. You make $100K combined... you pay $2k in taxes. You make $1.1 million... you pay $202K.
You could have a similar system for corporate taxes that took .5% of gross revenue with no deductions. Companies could gain ALOT of efficiencies if every financial decision weren't shaded by tax implications.