Sexual assault surveys are unreliable for many reasons. First, these surveys (like all surveys) only produce what respondents say and, as we all know, what people say often does not correspond to what they do and, in the case of sexual assault surveys of females, what the women who responded to the survey have actually experienced.
Second, sexual assault surveys (once again, like all surveys) rely on respondents’ memory–the ability or capacity to retain and recall past experiences–and memory is innately elusive and often flawed; thereby contributing to the unreliability of answers. For example, one survey asked about sexual assault “in the past year” (Chicago Tribune, 9/3/15, sec. 1. 14). Most people can’t remember with accuracy what they experienced last week, let alone in the last twelve months and, moreover, how many respondents counted their experiences in the last thirteen, eleven, or even fourteen months? There’s no way to know.
Another sexual assault survey of “female undergraduates at an array of prominent universities” (described in a Chicago Tribune article, 9/22/15, sec. 1, p. 9) was an open Internet questionnaire and produced, according to the article’s authors, “overstated victimization rate[s]” because, “there was evidence that hundreds of thousands of students who ignored the electronic questionnaire were less likely to have suffered an assault”. Not so fast! Given the hundreds of thousands who did not respond, it’s possible that this particular sexual assault survey understated victimization rates, or maybe the survey result—namely that more than 20 percent of female undergrads were victims of sexual assault—is right on target! There’s no way to know. Open Internet surveys are self reports by self selected respondents and always produce, as the Stones tuned, useless information.
Sexual assault surveys also are made unreliable—and this is true for all surveys—because words in questions do not necessarily have the same meaning for all respondents. For some, “assault” means physical pain; others say they are assaulted by a light touch on what they consider the wrong place (which may not be the same place for others); still others consider they are assaulted by a leering glance. And the meanings respondents give to “sexual” likewise varies, including, for example, kissing, penetration, but for some (Bill Clinton comes to mind) does not include oral sex.
If you want to find out about sexual assault (or, for that matter, anything else), don’t ask. Instead, use what I call “proper methods” and/or help foster what’s referred to as the “ingenuity of the shared enterprise” to develop non-asking procedures appropriate for your specific object of investigation.
I discuss proper methods and research ingenuity in Part 6 of my book, The Problem with Survey Research.