I do not deny that some teachers are more talented than others.  My first year I was a very ineffective teacher who spent half of each class trying to maintain order.  My second through fifteenth years, I was much better.  But even within the very small sample size of myself, there is variation.  Some days I am better than other days, some weeks I am better than other weeks.

When I have a ‘bad’ week, my students don’t do as well on my test as they would have had I had a ‘good’ week.  The biggest variable, I think, for me, is that my ‘bad’ week tires me out and then I’m not able to give as much individualized attention for students who are struggling.

But aside from my first year where I really only ‘taught’ for half the time, there is not a giant difference, in terms of final test scores, between a ‘good’ year and a ‘bad’ year.  That doesn’t mean that I don’t inspire kids in different, unmeasurable ways, when I’m really on, but in this current ‘data driven’ climate, that doesn’t really matter anyway.

An often quoted ‘statistic’ by various ‘reformers’ is that an effective teacher is three times as good as an ineffective one.  Sometimes it is said that the ineffective teacher gets a half year of progress while the effective teacher gets one and a half years of progress.

I don’t doubt that there are a small percent of teachers who have little classroom control, mostly new teachers, who only manage to get a half a year of progress.  I also can imagine a rare ‘super-teacher’ who somehow gets one and a half years of progress.  (I think I’m a pretty good teacher, but I doubt I get a year and a half worth of progress.)  I don’t think there is a very accurate way to measure this nebulous ‘progress’ aside from test scores, but I could still imagine that there is a ‘true’ number, even if we will never be able to accurately calculate it.

As this statistic has been quoted by Melinda Gates recently on PBS and by Michelle Rhee in various places, including the StudentsFirst website I thought, in response to a recent post on Diane Ravitch’s blog I would investigate the source of this claim.

It seems they are quoting a 2010 paper by Stanford professor Eric Hanushek called ‘The Economic Value of Higher Teacher Quality.’  I had to waste $5 on this to find who he quotes when he writes “Some teachers year after year produce bigger gains in student learning than other teachers. The magnitude of the differences is truly large, with some teachers producing 1½ years of gain in achievement in an academic year while others with equivalent students produce only ½ year of gain”  So I look in the footnotes and he is quoting HIMSELF from 18 years earlier in a 1992 paper called ‘The Trade-off between Child Quantity and Quality’ which I downloaded from JSTOR.

The main point of this paper, it seems, is that when poor families have many children, it negatively affects the academic achievement of the kids.  So, ironically, this is a paper about how poverty ‘matters’ a lot.

He uses data from 1,920 student from Gary, Indiana (no relation to me), from reading comprehension and vocabulary test scores (no math!) from the years 1971 to 1974.  In his opening paragraph, he is clear about the limitations of his conclusions about variation in teacher quality.

An empirical investigation of trade-offs between number of children and their scholastic performance confirms that family size directly affects children’s achievement. Though parents show no favoritism to first-born children, being early in the birth order implies a distinct advantage, entirely because of the higher probability of being in a small family. Recent large changes in family size explain a portion of aggregate test score declines, but increased divorce rates and market work by mothers have no apparent impact. Finally, teachers are shown to differ enormously, even though performance differences are poorly captured by commonly measured teacher characteristics. (emphasis added) The evidence supports a teacher skill interpretation of differences in classroom achievement.

Other passages that indicate he admits his conclusions are not ‘strong’

(page 91) A subsequent section investigates the possibility that differences in classroom performance are not entirely attributable to differences in teacher skill but instead involve more complicated interactions of teachers and specific classes of students.

And again

(page 92) This finding of little bias from omitting school inputs, however, could well be an artifact of the specific sample. All children are drawn from the same school system, thus limiting the variation in school inputs and the range of correlation between family and school characteristics. The systematic effects of families are portrayed in the top portion.

On page 105, there he concludes something that is bad news for TFA

Within this sample, there is no indication of differences in performance for male and female teachers, but white teachers do significantly worse than black teachers. The sampled students are all black. Therefore, this result may reflect either that black students do better with teachers of their own race or that the white teachers that are attracted to this setting are otherwise poorer, given their measured characteristics.

For the very small sample size, and the admitted biases in the data, this paper has definitely not earned the right to be quoted 20 years later as ‘proof’ that firing ‘ineffective’ teachers is the answer to closing the achievement gap.

Even if there are a few teachers who get one and a half years of growth while other teachers get only one half a year of growth, he doesn’t indicate what percent of these superstars there are and what percent of incompetents there are.

Basically, this would be comical if this myth wasn’t being used to shut down schools and fire teachers.