Survey Research Generates Conflicting Answers

Many published studies in academic journals, newspapers, and online indicate that survey research (any instrument or procedure that asks questions of respondents) generates conflicting answers. This often occurs when an asking instrument contains numerous questions on a particular topic. Answers to some questions affirm the phenomenon (behavior, opinion, etc.) investigated, whereas other responses imply, or explicitly state, the opposite.

Also, conflicting answers are common when different instruments pose questions about the same phenomenon. For example, a number of survey researchers using different surveys asked about the similarities and differences between public and private organizations. “[T]his research . . . refute[s] widely held . . . assumptions about similarities and differences between public and private organizations but . . . in some ways support[s] such assumptions” [Rainey and Bozeman, “Comparing Public and Private Organizations”, J-Part, 2000, p. 447].  Paraphrasing The Rolling Stones, you  can–in survey research, if nowhere else–have it both ways round: sun and moon, both A and ~A.

There are many reasons why survey research generates conflicting answers: different samples, respondents’ biases, question wording, etc.  See the above-mentioned Rainey and Bozeman article and an online article about surveys of U.S pet ownership for discussions of reasons for this particular limitation/weakness of survey research.

Regardless why survey research produces conflicting answers, the fact that it does means that you shouldn’t ask to find out what’s really going on.  See my book, The Problem with Survey Research, for a complete statement of the limitations/weaknesses of survey research and, in Part Six, brief descriptions of observation and other procedures that produce reliable information.   On the latter point, see also my course syllabus, PA 544, Qualitative Methods: How to Find Out What’s Really Going On.

 

About georgebeam

George Beam is an educator and author. The perspectives that inform his interpretations of the topics of this blog–-as well as his other writings and university courses -–are system analysis, behaviorism, and Internet effects. Specific interests include quality management, methodology, and politics. He is Associate Professor Emeritus, Department of Public Administration; Affiliated Faculty, Department of Political Science; and, previously, Head, Department of Public Administration, University of Illinois at Chicago
This entry was posted in Survey Research and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s