Non-random Samples and Bribes in Survey Research

Only research based on random samples provides the best possibility for obtaining reliable results.  Survey researchers/askers often can’t obtain random samples.  Rather than using a non-asking procedure–such as observation, experiments, or predictive models–to study the population they are investigating, they employ multi-step, convoluted, procedures–e.g.  Respondent-Driven Sampling (RDS), described below) that often include bribes or, as survey researchers nicer-name them, “incentives”.  Bribes succeed in getting askers more answers but, as much research indicates, bribes affect answers and, thereby, make them unreliable.   Moreover, RDS, with or without bribes does not produce random samples; thus, answers produced by RDS with bribes are, to coin a concept, “doubly unreliable”.

Here is a description of RDS that was sent to me by a university survey research center.  I’ve put my comments in the text of this description within brackets and in bold:

“Respondent-Driven Sampling

Respondent-driven sampling (RDS) can be used to recruit individuals who belong to hidden, hard-to-reach, stigmatized populations, where the members of the population are known to each other—for example, illegal drug users.

RDS includes several steps. The recruitment of the initial respondents [recruited respondents are NOT random respondents]—called seeds—is done by the researcher; however, all subsequent recruitment is done by the selected respondents.  [respondents recruited by respondents are NOT random respondents] Following initial interviews, the seeds are given coupons that they are asked to give to other eligible members of the network. If the second-stage recruits wish to reveal their identity, they can contact the researcher to be interviewed. Researchers also give incentives [bribes skew answers and make them unreliable] to respondents when they participate and tell them that they will get an additional incentive [more bribes!! and, in this instance, bribing the bribed] if their recruits also participate. If the second-stage respondents complete an interview, they are in turn given coupons and incentives for the recruitment of third-stage respondents. [who are triply bribed and, consequently, to coin another concept, “triply nonrandom”]  The process  continues [survey researchers have no shame!] until the needed number of completed interviews has been attained”.

Concerning triple stacks of weaknesses in survey research, see Diane Johnson‘s comment that interviews are triply subjective.

For a complete assessment of interviews and other modes of survey research, see my book, The Problem with Survey Research.


About georgebeam

George Beam is an educator and author. The perspectives that inform his interpretations of the topics of this blog–-as well as his other writings and university courses -–are system analysis, behaviorism, and Internet effects. Specific interests include quality management, methodology, and politics. He is Associate Professor Emeritus, Department of Public Administration; Affiliated Faculty, Department of Political Science; and, previously, Head, Department of Public Administration, University of Illinois at Chicago
This entry was posted in Survey Research and tagged , , , , . Bookmark the permalink.

One Response to Non-random Samples and Bribes in Survey Research

  1. Pingback: Increasing Acknowledgement of Survey Flaws | George Beam's Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s