Asking Settings Make Answers Unreliable

Settings in which questions are asked and answers given make answers unreliable; that is, answers that may, or may not, be accurate. This happens because components in the two types of settings (societal and immediate, discussed below) skew answers, and since askers/survey researchers only have answers to their questions, it’s impossible for them (or anyone else) to determine which, if any, are either accurate or inaccurate. The only way to know is to check or verify answers with data/information from non-asking sources; for example, from observations or experiments. Askers do not have this type of data; all they have is unreliable information.

There are two types of asking settings: societal and immediate.
• Societal settings are cultures experienced by respondents, and that includes social values, illusions/myths/religions, and descriptions and interpretations of political and economic systems. Also included in respondents’ societal settings are their positions in society—e.g., socioeconomic, organizational, familial, and so on—and the norms and beliefs associated with these positions. Via socialization by parents, schools, colleagues, governments, and advertising, respondents take as their own the societal preferences, priorities, and outlooks they experience and answer accordingly.
• Immediate settings are specific places where asking and answering occur: for example, respondents’ homes and workplaces (via phone and Internet), on streets outside polling places (entrance and exit polls), schools, doctors’ offices, and so on.
Both types of settings (societal and immediate) have powerful attributes or components that affect answers. Characteristics of, and forces within, societal settings—such as cultural values, laws, and media—bias answers, as do aspects of immediate settings, including third parties, gender of interviewer, and so on. Essentially every component of every asking setting “contaminates;” forcing respondents to say what’s compatible with each setting.

This post is from my book, The Problem with Survey Research, p. 165, where sources are cited and where, on pp. 165-95, you can find an expanded discussion of how asking settings make answers unreliable.

About George Beam

I'm an educator and author. The perspectives that inform my interpretations of the topics of this blog are behaviorism and system analysis. Specific interests include American politics, socioeconomic issues, survey research, and effects of the Internet and attendant hard- and software. I'm Associate Professor Emeritus, Department of Public Administration, Affiliated Faculty, Department of Political Science, University of Illinois at Chicago.
This entry was posted in Survey Research and tagged , , , , , . Bookmark the permalink.

1 Response to Asking Settings Make Answers Unreliable

  1. Pingback: Askers Make Answers Unreliable | George Beam's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.