Survey Research Norm: Acknowledge and Use Flawed Results

The norm in survey research is that any results are acceptable for academic and/or popular publication and/or broadcast as long as the survey researcher/asker or the user of survey research results describes the research design, method(s) of collecting answers, and flaws in results. It doesn’t matter how poor the methodology or how poor the results. Paraphrasing Bachman-Turner Overdrive, any results are good results, so survey researchers and users of survey research take whatever results they can get.

This norm, the acknowledgement and use of flawed results, is evident in D. Paul Sullins’s survey research based book, Keeping the Vow: The Untold Story of Married Catholic Priests.  He acknowledges a few times—but only in the Appendix—that the results he relies to tell his now-no-longer-untold story are flawed. In one instance, he confesses that some of the surveys have low response rates and, thereby, might cause “over [or under] statement”. Also, he admits there’s a wide range of “complications” in the surveys used to compute and compare compensation of married and supposed-to-be-celibate priests that produces “uncertainty” in results. A few pages later, he admits there’s “a good deal of uncertainty and wide variation in [the survey-based] estimate. . . . [of] the . . . average difference . . . . in the expenses to a parish to maintain a married verses a [supposed-to-be] celibate priest.” Because any results are good results, Sullins takes flawed results and uses them to tell what can only be a flawed story of married priests.

For a further discussion and more examples of acknowledgements and uses of flawed results in survey research, see my book, The Problem with Survey Research.

About georgebeam

George Beam is an educator and author. The perspectives that inform his interpretations of the topics of this blog–-as well as his other writings and university courses -–are system analysis, behaviorism, and Internet effects. Specific interests include quality management, methodology, and politics. He is Associate Professor Emeritus, Department of Public Administration; Affiliated Faculty, Department of Political Science; and, previously, Head, Department of Public Administration, University of Illinois at Chicago
This entry was posted in Survey Research and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.