The influential consulting firm McKinsey & Company caused quite a stir when it published an article last week predicting that nearly a third of employers "will definitely or probably stop offering" health insurance to their workers after 2014.
Republicans, including some presidential candidates, seized upon the piece as evidence that President Obama was wrong when he vowed that people who liked their health coverage could keep it after a federal health overhaul took effect.
But just how accurate is the McKinsey study?
While some news organizations (not including this one) accepted the study at face value, others, including Time, Talking Points Memo, and the Washington Post's Plum Line blog, started raising questions fairly early on. The biggest one: Why did McKinsey refuse to release the methodology by which it surveyed the 1,300 employers the article said it queried?
We asked McKinsey for details. The firm declined to comment.
So we turned to the American Association for Public Opinion Research for help figuring it out. They referred us to Scott Keeter, director of survey research for the Pew Research Center. "We're not the polling police," he said. "But I'm suspicious of research if people won't tell me how they did it."
Another thing that raised eyebrows: The McKinsey findings were so out of line with studies done by other outfits, the Mercer consulting group, the Urban Institute and the Rand Corporation. All those studies found the impact on employer-provided insurance likely to be much smaller.
Speculation aside, there's the real-life example of Massachusetts, where the requirement for most individuals to have health insurance has been in reality since 2007.
And there, access to employer coverage has increased, rather than gone down. According to a study released in April by the Blue Cross Blue Shield of Massachusetts Foundation, "There has been no evidence of subsidized coverage 'crowding out' employer-sponsored insurance, and employer offer rates have grown from 70 percent to 76 percent since implementation of reform."
All of which prompted irritated Democrats to lash out at McKinsey. "Honest public discourse requires a standard level of transparency — one McKinsey simply has not met," said Senate Finance Committee Chairman Max Baucus in a letter Thursday to McKinsey's Managing Director Dominic Barton asked the firm release the study's methodology.
Nine leading House Democrats, led by Energy and Commerce ranking member Henry Waxman, sent Barton a similar letter late Thursday.
"We are concerned that, if the survey based its conclusions on a questionable instrument and potentially biased methodology, McKinsey may have provided the American public with invalid information about the impact of the Affordable Care Act," the House members wrote.
Backed into a corner, McKinsey may be ready to comply. Unofficial word is the methodology will soon be forthcoming.
ROBERT SIEGEL, host:
The latest round in the battle over the Obama health care plan has to do with some pretty startling survey results. The report comes from consulting firm McKinsey and Company. It says nearly a third of businesses are likely to stop offering health insurance to their employees when the new law takes full effect in the year 2014. And that was quickly seized upon by the president's critics.
But some serious questions have been raised about this study. And now, Democrats are demanding more information about how it was conducted.
Joining me in the studio to sort out some of this is NPR health policy correspondent Julie Rovner. Hi, Julie.
JULIE ROVNER: Hey, Robert.
SIEGEL: And to begin, let's be blunt. There have been enough surveys and studies on what last year's health care law might or might not do to fill library shelves, if not entire libraries. What so different about this one to make it so important?
ROVNER: Well, there's two things. First, McKinsey is a very well-respected firm in the health policy community. And second, the finding that almost a third of employers said they would, quote, "definitely or probably stop offering health insurance in the years after 2014," was really quite eye-popping. And it was one that obviously was going to be repeated a lot by opponents of the law.
SIEGEL: And this finding is actually quite different from that of lots of other studies that have posed the same question?
ROVNER: Absolutely and by a lot. The Congressional Budget Office, the Rand Corporation, the Urban Institute, several other groups have all looked at the question of how many employers are likely to drop coverage under the law. And remember, most employers will have to pay a penalty if they do that. And all of those studies have found a likely change will be much smaller.
But you don't have to just look at studies to wonder about this. There's at least one real world example and that's up in Massachusetts, where they've required individuals to have health insurance for four years now. And in Massachusetts, access to employer health insurance has actually increased. It's gotten easier.
SIEGEL: On the other hand, one could say the economy is a moving target; this is a study that's come out with a different finding but not necessarily an incorrect finding. What is it about this study that raises eyebrows?
ROVNER: Well, what really got people suspicious was that McKinsey has been refusing to release its methodology - how it did the study. They said they interviewed 1,300 employers for this study. But no one knew whether they were large employers or small employers, which makes a very large difference. We didn't know if they talked to human resources directors or to owners of companies. So they might not have even surveyed the people who can make the decisions about whether companies will continue to offer health insurance.
And I spoke to several independent pollsters, including people at the American Association for Public Opinion Research. They said while they're not the polling police, if someone does a survey and refuses to release its methodology - that raises a big red flag.
SIEGEL: And then the politicians got into it.
ROVNER: That's right. Democrats, of course, are worried that this number will take hold. We're already seeing Republican presidential candidates using it out on the campaign trail. So yesterday, Senate Finance Committee Chairman Max Baucus and nine top House Democrats basically demanded that McKinsey release the methodology it used to do the survey. Now, McKinsey says it will, although we haven't seen it just yet.
SIEGEL: Julie, is there a chance that this study is accurate and that all the other studies are actually underestimating how many employers think they'll probably drop coverage after 2014?
ROVNER: Well, there is. And I've talked to at least one health policy consultant today, who I do trust a lot, who does believe that.
But, you know, there's another important point about all of this. One pollster I spoke to pointed out that even if the survey is valid, there's so many things that are still unknown about how the law will actually roll out between now and 2014. There's regulations still to come, it's still unknown what these health insurance exchanges will look like. There's other things that are still yet to be decided.
It's likely that no employer really knows what they will or won't do in 2014 or slightly after that. So you really need to take all of these surveys with a really big grain of salt.
SIEGEL: Okay, that's NPR health policy correspondent Julie Rovner. Thank you.
ROVNER: You're welcome. Transcript provided by NPR, Copyright NPR.