On the basis of an interview by Katherine Barrett of Governing Magazine about my new book, The Problem with Survey Research – http://www.governing.com/columns/bg-report/col-residency-requirements-connecting-citizens-performance-reporting-new-uses-data.html – she writes:
We’re regularly surprised by the sheer quantity of surveys being published these days, ranging from those that ask about “faith in government,” to those that look into questions of unfaithful spouses. A great many of these results don’t pass our own smell test, and yet many are presented as though they represent absolute truth.
We recently talked with George Beam, an associate professor in the department of public administration at the University of Illinois in Chicago. Beam explained why he worries about over-reliance on surveys. For starters, there are the many external factors that affect how someone answers a question — the gender of the person who is asking, the way the question is worded and the format of the survey itself. “No matter what sorts of questions we ask, or how carefully we try to word them,” he says, “we’re still always getting responses that may or may not be correct.”
Beam, who has written a new book on this topic, The Problem with Survey Research, says the drive to get people to give their opinions and feelings on issues pervades all aspects of current culture, from academia to consulting to newspaper reporting and TV interviews. “Everyone is asking everyone about everything everywhere,” he says, adding that he’d like to see much more reliance on observation, experimentation and document and content analysis.