Diane Rado (Chicago Tribune, 8/9/13) says that Illinois’ first statewide education survey “provide[s] details on how teachers teach, children learn and principles lead”. No way!! The survey only provides statements about what respondents (K-12 teachers and sixth-through 12th-graders) say about these matters. And, as we all know, what people say often does not correspond to what they, or others, do or have done.
The stated goal of this survey, Rado reports, “is to drive improvements in the state’s public schools”. Well, when you want to find out how to improve the schools (as well as how teachers teach, children learn, principles lead) DON’T ASK. Instead, observe teachers, students, principals, and other relevant people (such as school staff, parents, etc.) and analyze documents (such as budgets, non-asking based reports of socioeconomic conditions, etc.) that provide information about circumstances experienced by students, teachers, etc., as well as information about results.
Optimizing understanding and betterment requires abandoning survey research as a method for finding out what’s really going on and using other procedures; not only observation and document analysis, but also experimentation and other, as I call them, “proper” methods of data collection and proper research designs. I discuss the limits of survey research and proper procedures for finding out what’s really going on in my book, The Problem with Survey Research. Check it out and let me know what you think!