Answers to Questions Are Not Reliable

I just posted my review of Jennifer Senior’s, All Joy and No Fun, on Amazon.  I titled it: Answers to Questions Are Not Reliable.  Here’s the review:

I’m not recommending Jennifer Senior’s All Joy and No Fun because it’s based on answers to questions which, according to extensive research , are not reliable. Even Senior acknowledges (unintentionally, I presume) the unreliability of answers when she writes that some surveys generate contradictory results or, as she puts it with nicer words, “disparate trends” (p. 151), and that respondents are sometimes dishonest or, again, nicer, “more ambivalent . . . than we care to admit” (ibid). Similarly, Zoe Heller, in an overall favorable review of Senior’s amalgam of responses, writes: “of course” some respondents could be “lying or at least sentimentalizing the truth” . Respondents’ memory also can skew answers. Moreover, answers to questions are made unreliable, not only by respondents, but also by the other three components of the asking method: asking instruments, situations/environments in which questions are asked and answers given, and by askers themselves.

To be sure, not every answer to every question is incorrect; thus, some of the information in this book about the lives of parents could be accurate; but how can you distinguish between what’s accurate and what’s contradictory; between correct answers and the incorrect responses given by dishonest respondents? Well, you can’t! When all you have are answers—and that’s essentially all that Senior has—it’s impossible to know which, if any, are correct or incorrect. The only way to know is to check or verify answers with observation, documents, or from other non-asking sources of data. Senior does not check her answers with data from these other sources; all she has are answers, all she has is unreliable information. If you want to find out what’s really going on (concerning parents, or anything else) don’t ask!  That’s the theme of my book, The Problem with Survey Research: Don’t Ask!

About georgebeam

George Beam is an educator and author. The perspectives that inform his interpretations of the topics of this blog–-as well as his other writings and university courses -–are system analysis, behaviorism, and Internet effects. Specific interests include quality management, methodology, and politics. He is Associate Professor Emeritus, Department of Public Administration; Affiliated Faculty, Department of Political Science; and, previously, Head, Department of Public Administration, University of Illinois at Chicago
This entry was posted in Survey Research and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.