Lying with Statistics

The headline at RealClearPolitics: “65% of doctors think new law will worsen care.”
The headline at CNBC: “Survey: U.S. doctors fear healthcare reform.”
The headline at the Wall Street Journal: “Survey of U.S. physicians finds pessimism on future of health care.”
The reality: This poll was conducted through a “fax-response methodology,” which means it didn’t survey a random sample of American doctors. It took a self-selected sample of the most passionate doctors, the ones who felt inspired to return a fax.
But you’d never know that from the media reports. CNBC and the WSJ only tell readers that the poll was conducted by fax; the fact that this tarnishes the poll’s credibility is not noted. You can only access the full polling report by going to the HCPlexus website (a company that “makes management products for physicians” and which sponsored this poll) and filling out a personal-information form. But once you get the report, and you scroll to page 33 of 36, you’ll learn that “the cohort of respondents could represent some bias.”
Look, I’m not the biggest fan of polls and their role in the media even when they are statistically valid. But can we at least set statistical validity as the bare minimum required to get a poll any sort of media attention?
Note, I’m not saying that American doctors aren’t skeptical of health care reform. I’m just saying that this poll can’t tell us that.