I’ve been listening to Steven Levitt’s Freakonomics the past week while commuting. While I find the perspective and insight fascinating, I truly detested Levitt’s handling of the “cheating teachers” studies in Chicago.
His perspective on education is obvious when he talks about “bad teachers” and “good teachers” (as defined by standardized test scores). The disdain with which he talks about the teachers supposedly proven to be cheating the tests is also disappointing. The real problem, though, is that he asked the wrong question. The question shouldn’t have been, “do teachers cheat?” Sure, some teachers cheat – so do some people in every profession in the history of the world. Teachers are human too.
The real question should have been, “why are these teachers cheating and what does it tell us about the educational system?” Or, better still, “are standardized test valid measures of teaching and learning?” Instead, huge assumptions were made:
- Teachers are cheating to make themselves look good and to get pay raises (or avoid being fired) – while this may be entirely true, individual teachers’ motivations may have been drastically different;
- Standardized tests are valid measures of teaching and learning – obviously I don’t believe this at all;
- Test scores should be relatively stable from year to year – anyone who has pored over data from frequently administered tests knows this to be untrue. I’ve seen kids’ scores on the MAP test go up and down dramatically within a year
No debate as to the validity, reliability or value of standardized tests is mentioned. No concern about cohort effects is addressed.
Why should we expect an economist to understand the full scope of the issue?
And yet, considering Levitt’s premise of using data and statistics to tell the story behind the story (I found his analysis of cheating in sumo wresting fascinating) I think he missed the true opportunity here.
The story Steven Levitt should be telling is the correlation between family income and test scores and what this indicates about our educational system and the future of our nation.