We're just spending it in the wrong places.
I think debt does matter.
However, spending a trillion dollars a year on the military and military/security related programs is way too much money down a rathole.
Because of overpopulation and resource depletion, we are not ever going to have the kind of 'growth' economy that the era of cheap fossil fuels provided for us over the last one hundred years ... so debt is going to matter even more in the future since we cannot 'grow' our way out of even the debt we have now.
And the working and middle class certainly is not in a position to pay anymore fees and taxes; inflation is a real concern for the 80 percent-plus of Americans who aren't seeing any raises in pay. Debt induced inflation would be catastrophic for us.
So, we need to rearrange our federal government spending priorities -- a woman working to build a highway in Ohio will give us a greater multiplier than a twenty year old getting paid to stand guard at some military base in Bulgaria.
Sadly, both major political parties in this country are pretty much wedded to the old fashion notions of growth. If we don't see the sequester, we will undoubtedly see only a slowing of the usual upward appropriations for the military and probably a slow but certain decay in the safety-net programs.