PHILADELPHIA -- At the height of 2009 Capitol Hill debate over health care reform legislation, a struggle punctuated by a protracted blitz of opinion polls, David W. Moore, a Senior Fellow at the University of New Hampshire's Carsey Institute,
wrote that the "recent series of polls brings to mind a bobblehead doll, whose head wags from side to side and from front to back in a random fashion. That disconnected movement seems to be a visual representation of what the polls have been saying about the general public and its views on efforts to reform the health care system."
Moore, author of books including "The Superpollsters: How They Measure and Manipulate Public Opinion in America," was expressing a frustration felt by many struggling to make sense of poll reports that seemed so contradictory and confusing; many who were keenly interested in knowing what the public's opinion of health care reform actually was.
A new study A year and a half later, a research team led by David Grande, MD, of the University of Pennsylvania's Leonard Davis Institute of Health Economics (LDI), has completed a comprehensive analysis of that exact question and published the results in Health Affairs magazine.
The bottom line: Not much of what you heard about health care reform polls on TV or read about in the nation's newspapers or blogs was very accurate. In fact, the most important data to come out of the period's frantic polling appear to demonstrate how answers to seemingly similar questions can be very different depending upon how that question is asked.
The scientific analysis of more than 80 poll results found that rather than constantly shifting its position on health care reform issues like the individual mandate or the public option, the majority of Americans supported both policies throughout the entire debate. For instance researchers found that public support for the public option -- which was ultimately dropped from the legislation -- averaged 57.6 percent throughout the entire legislative debate period.
'It became a circus' "It really became a circus as each week a different poll would show a wildly different result from a poll just a week prior," said David Grande, MD, lead author of the study entitled "Public Support For Health Reform Was Broader Than Reported and Depended On How Proposals Were Framed." An assistant professor of medicine at Penn's Perelman School of Medicine, and LDI Senior Fellow, Grande noted that the media's excessive focus on the Capitol Hill "horse race" resulted in inaccurate reporting and public confusion.
Grande and co-authors LDI executive director, David Asch, MD, and Sara Gollust, PhD, an assistant professor at the University of Minnesota's School of Public Health, analyzed more than 80 different poll results to determine what variables could have caused each of them to report such different results for questions about policies such as the individual mandate and the public option.
'Pretty surprising' "When we looked at the public option polls over the course of the entire debate, we found wide variations of as much as 30 percentage points," said Grande. "The fact that the differences were so large was pretty surprising."
"But when we looked at how the questions were asked by each polling organization, there were differences in the way the policy was described," he said. "We were able to explain more than half of the variations in these results based on the inclusion or exclusion of a few key words and phrases used to describe the policy. For example, describing the public option as 'similar to Medicare' was associated with about a 10 percentage point increase in support."
"The problem with the press," said Grande, "is that they were reporting on each and every poll without thinking at all about how that poll differed from the dozens of polls before it. They were reacting in isolation to each and extrapolating from that, that the public's views were rapidly changing. What they should have been doing was looking at the polls in their entirety and asking why there were such large differences -- but that never happened."
"At the same time," said Grande, "media coverage strongly influences members of Congress who take cues from news reports about what policies they think may or may not be palatable to the American public. So, we could well have had some different legislation. I think the public option was dispensed with because members of Congress were worried the public wasn't in support of it but a close look at the polling data doesn't really show that that concern was true."
appear to demonstrate how answers to seemingly similar questions can be very different depending upon how that question is asked.