In fact, CBO has hardly been way off "in every aspect" of its predictions, as Spicer said.

A study by the liberal-leaning Commonwealth Fund foundation found that CBO's predictions were more accurate than other forecasters. "CBO did much better than other estimators — that has also been true in the past," said New York University professor Shelly Glied. "Nobody can actually foresee the future, but CBO seems to do a better job than anyone else."

It's also worth noting the widespread support CBO enjoys both from both Democrats and Republicans. Its current director, Keith Hall, was chosen by Republicans and served on the Council of Economic Advisers in the most recent Bush administration. Speaker Paul Ryan, R-Wis., a former chairman of the House Budget Committee, was a vocal supporter of Hall's predecessor, Doug Elmendorf, who was named by Democrats.

The CBO analysis matters because it'll probably predict how many people would lose health coverage under the measure, which could be politically damaging. And if the plan doesn't meet the goal of at least saving the government a modest $2 billion, the legislation as written could be subject to a filibuster from Senate Democrats.