Aug 20 2011

Class Warfare: Fact checking pages 1 through 100

Follow garyrubinstein on Twitter
pages 1 to 100
pages 101 to 200
pages 201 to 300
pages 301 to 350
pages 351 to 400
pages 401 to 437
When the new book ‘Class Warfare’ came out, the story of the current ed reform movement featuring TFA and KIPP, I refused to buy it, since I didn’t want anyone to profit from passing fiction and fantasy off as non-fiction.

But when I received a complimentary copy as a gift from my frequent debate opponent, Whitney Tilson, I promised that I would, at least, read it.

Analyzing a short research paper or an hour long debate is one thing, but a 500 page book — well, I was hoping that I wouldn’t find much to say about it since I really don’t have the time, yet I can’t resist.

Page 1 starts innocuously enough, but then one of the most misleading sentences in the entire first 100 pages occurs on page 2. Jon Schnur is one of the protagonists of the book, designing Race To The Top and advising Clinton and others on education. Schnur is a very sophisticated thinker, so it is tough to summarize his views in just one sentence. But for this introductory section, to set one of the most important recurring themes throughout the book, Brill attempts to do so with “Truly effective teaching, he came to believe could overcome student indifference, parental disengagement, and poverty …”

This is the battle cry of the corporate reformers — that teacher need to be motivated to work harder and not hide behind their job security provided by the selfish unions who always put the needs of the adults ahead of the needs of children.

Reading this summary of Schnur’s philosophy struck me as odd as I had recently corresponded with him on this very issue. What follows is my email to him and (with his permission to publish this) his much more nuanced response.

Dear Jon,
This is Gary Rubinstein. I’m a veteran math teacher at Stuyvesant High School, a TFA alumnus (Houston 1991), and I also participated in one of your ‘think tank’ sessions last summer. The reason I’m writing is that I’m concerned about how the current debate over school reform is playing out in the media and throughout the legislatures of different states.
I read your rebuttal to the Diane Ravitch Op-Ed, and I wanted to examine the ‘subtext.’ Anytime someone writes something, there’s the danger that the subtext is interpreted in different ways by different people with different interests.
As a teacher (and as someone mentioned in the Op-Ed), I was not at all insulted by Diane Ravitch’s comments.
I did not think that she was trying to put down the schools by highlighting their poor test scores.
A lot of the public listens to comments from Michelle Rhee which seem to indicate that the problem with education is that teachers are unqualified and that those who are qualified are lazy. A school that touts high test scores and claims that the only difference between it and its low-performing neighbor is that the teachers are working harder. This supports Rhee’s claims (or at least how the public perceives her claims) and leads to states closing down schools and firing teachers because they are not performing ‘miracles’ with their schools test scores.
By showing that most, if not all, of the proof that schools can be turned around by getting teachers to work harder should be dismissed because a school like Bruce Randolph isn’t even getting the results it claims.
In your Denver Post rebuttal, you mention two other schools that are getting even better results, and I don’t doubt that they are. But you know that these schools are doing a lot of things besides just getting motivated teachers. I know that New Leaders For New Schools trains leaders who learn a lot more than how to scare teachers into working hard. I’m sure that a lot of what you teach your New Leaders is strategies for overcomingthe symptoms of poverty.

This is where I think you and Ravitch agree. You both think it takes a lot more than just motivating qualified teachers and firing unqualified ones.

I wish you could make this more clear in your writing so it doesn’t get misused by people like Michelle Rhee who will use your credentials to support her heavy-handed approach to a very sophisticated issue.

It’s easy to characterize Ravitch as someone who is defending status quo and saying that teachers can’t do any better, but that’s not really what she’s saying. Teachers are being attacked because they need support to perform miracles. Even with such support, miracles are tough — as can be seen by the four schools that were labelled turnarounds though they haven’t gotten their scores up yet. And the ‘true’ miracles, like West Denver Prep, well I’m sure they are doing a lot more than just threatening teachers. (I also notice that some of their test scores are good, but others are still pretty bad)

This issue is so important that I think it’s important that people like you and Diane Ravitch figure out what you agree upon before the things that you don’t agree upon dominate the discussion and people like Rhee with her huge budget for PR get to make the decisions that will affect all kids.

Sincerely,
Gary Rubinstein

And his response:

Gary -

Hope you are doing well.

Colin Rogister shared your note with me – thanks so much for sharing your thoughts with us.  I really appreciate your taking the time to write.

First, I strongly agree with you about the need for dialogue. That’s why I reached out to Diane and suggested we tour some schools together and have a dialogue about what we’ve seen and their implications.  My sense is that she and I both have some important differences of perspective – and  common ground as well. Shared experience and dialogue can help crystallize what’s shared, what’s different, and can enable progress that can benefit kids.  Even if inadvertant (and I alluded to that possibility in my op-ed), I do believe that her New York Times op-ed communicated to many readers that low-income children and schools serving those children can’t make important academic and educational progress at least in today’s climate and context.  And while I very much look forward to a respectful dialogue (and I am sure some important common ground), I profoundly disagree with that (even if unintended) perspective. I think that risks paralyzing efforts to improve education and address poverty generally – just at the moment when we should be intensifying our progress.

Second, I agree with you emphatically that any strategy for improved education based simply on teachers working harder has no prospect for success.  There is no evidence to support the effectiveness of a one-dimensional strategy of getting teachers to work harder or be more motivated.   Indeed, our educational systems must drive dramatic improvements in teaching and school leadership excellence – with a blend of high standards, smart accountability, and very deep, meaningful support for the development of educators systemwide.  A teacher quality strategy that relies exclusively on better individual teacher performance won’t succeed – nor will a strategy that ignores the dramatic impact different types of teaching can make on student success and achievement.

Third, I am so compelled by efforts by educators that have led to dramatic progress for low-income students in a growing number of schools and classrooms across the country. This growing - but still too small, and too difficult – number of successes demonstrates the powerful evidence that all students regardless of background can achieve at high levels when we adults get it right – and get it right systemically.  I think the implications of this point are enormous -leading to the importance and urgency of making sure we adults at scale can help our kids pull off what they have demonstrated they are capable of.  The accelerated progress of these schools and classrooms as well as students needs to be a foundation on which we build even greater success.

Finally, I agree that we must make progress toward solutions on an issue that is more complicated than the one or two dimensional characterization often given of education today in the media. One way to do that is to recognize that there are many people working towards critical goals, and it will be most helpful to find a way to harness the energy, commitment and important ideas from a number of people with a range of perspectives including you and Diane, Michele Rhee and many others as well.

Again, thanks for sharing your thoughts – and thank you for all you do for kids.

All the best,

Jon

Throughout the first 100 pages, Brill repeats the idea that reformers want to link teacher evaluations to the growth their students get on standardized test.  On page 3 Obama tries to get legislation passed when he was senator with this in it, and on page 5 it finally becomes a part of Race To The Top “plans for using data and student-testing systems to evaluate teachers based on student improvement.”

To people who are not teachers, this sounds reasonable enough.  Shouldn’t teachers be evaluated on their performance like everyone else?  Well, the thing that Brill fails to mention is that teachers do not mind being evaluated on their performance.  What they do mind is being evaluated on their performance with an unreliable metric.  Standardized test scores whether the absolute ones or the growth models, otherwise known as value-added, have been shown to be completely unreliable in study after study.  This is why unions rightly object to them being used for teacher evaluations and also for school closings.

Now, this does not mean that ‘tenured’ teachers have a job for life.  There are plenty of other ways to measure teacher quality, which principals are supposed to do with classroom observations and writing up teachers who are not doing their jobs.  That system is already in place and though it is a lot of work for an administrator, that is part of their jobs to supervise and otherwise motivate their staffs.  To add in an unreliable metric to the mix is completely unfair.  Besides being unfair, using this type of metric in teacher evaluations has the unfortunate side effect of encouraging teachers to teach to the test, thus robbing students of true learning opportunities.  It is a lose-lose.

Page 7 “researchers at think tanks who were producing data about how teaching counted more than anything else”  Not true.  Even Michelle Rhee is careful to always qualify this common claim by saying teachers are the most important “in school factor.”  Much of this research is pretty suspect anyway as you can see in one of my recent posts.

Page 8 On charters “these are public schools with no admissions requirements or any other filters.”  What about the self-selecting nature of kids who have parents who have it together enough to find out about the lottery?

Page 9 On charters “ones that work not only demonstrate that children from the most challenged homes and communities can learn but also suggest how traditional public schools might be changed to make them operate effectively.”  Well, with attrition and also the self-selection component mentioned above, it is safe to say that they are not the children from “the most challenged homes,” but maybe, at best, “challenged homes.”  Most challenged don’t make it in, are counseled out before they start, or soon after they start, in many cases.  And traditional public schools can’t change to copy these tactics.  Where would the kids who get kicked out go?  Back to a charter school?

Page 11:  Reid had “forced out one of her new fifth-grade English teachers.” in mid November, but then later in the page described that Reid and the teacher “had agreed that she and Harlem Success were not a good fit.”  How can you be forced out and agree at the same time?

Page 12:  Why does her charter school class have only 22 kids, isn’t that a bit low?

Page 15:  About traditional school teachers that don’t use their sick days “or turn into cash when they retire.”  He makes it seem like a bad thing that there is an incentive to not missing school.  If you ‘bank’ your sick days, that’s a good thing and you should get some kind of benefit from it.

Page 16:  Mentions in footnote that some critics claim that charters “skim” the top students and writes “None of the actual data support this.”  This is not true.  Here is a recent one that says they do it, but that it is a good thing.

Page 19:  Mentions that TFA has student teaching with 5 to 10 kids in a class — important point glossed over.  Poor training is bad for the kids they teach.

Page 20:  “none of the tricks she’d been taught at the TFA academy for regaining the class’s attention worked.”  Again, the TFA training is poor and hasn’t actually improved much.  This story about Reid’s terrible first day followed by her making a change and having a great second day is pure fiction, though Brill wouldn’t know this.  In general, kids are pretty well-behaved on the first day.  It’s just part of the group dynamic.  If they are bad on the first day, it will be a really tough year for a new teacher.

Page 21:  Rhee is failing in her first year.  This is an important point since if even the great Rhee can struggle so much, what does that say for the mortal TFAers and how they fare?

Page 23:  I’m glad that Brill mentions that Levin and Feinberg were mentored by an older union-member teacher, but then adds that Harriet Ball and other senior teachers that were helpful “hadn’t gotten the memo telling them they were supposed to fail.”

Page 25:  Usdin also got help from a union teacher.  I was glad to see this mentioned.

Page 27:  American public schools are failing because they ranked 25th in math achievement internationally.  But if you compare our low-poverty schools to low-poverty countries, were beat them.  And if you compare our high-poverty schools to high-poverty countries, we beat them also.  This is something that few people know about.  Also, the U.S. always fared poorly on those international comparisons going back to 1964 when we were 12th out of 12.

Page 28:  On the NYC rubber room “The city’s contract with the UFT required that charges against them be heard by an arbitrator, and until the charges were resolved — in a process that typically took three to five eyars and rarely resulted in dismissals — they continued to draw their salaries and accrue pensions and other benefits.”  The slow arbitration process is not the fault of the unions and the teachers — and if they were not dismissed doesn’t that mean there wasn’t enough of a case against that teacher?  Why would it take so long, then.  We live in a country where we don’t get convicted just because were were accused.

Page 41 (there was an interesting history of the teacher’s unions for a while, which I liked) then “This created a dynamic in which the union leaders and the bureaucrats were naturally inclined to find ways to coexist by worrying more about the adults than the children.”  This thing about adults vs. children is such a cheap trick, so let me describe a typical way that the union helps worry about the children and some of the adults over the interests of other adults.  There’s a union rule that schools should balance teachers schedules so that each teacher has a minimum number of ‘preps’, meaning that if I’m teaching Algebra, Geometry, and Trig, I’ve got 3 preps, which is a lot more work than if someone has one prep, just teaching Algebra.  So the union says this is not fair and we should each have two preps.  This benefits the children of the teacher who now can concentrate on the two preps rather than being spread thin by three preps.  This is just a small example, but it is typical of the way that the union protects teachers from administrators who want to ‘play favorites’ with the teachers they like best.

Page 43:  On ‘A Nation At Risk’, “or how teachers’ union contracts blocked the performance-based evaluations,” with no mention yet of WHY they blocked these inaccurate evaluations.

Page 44: “There are still thousands of skilled, highly motivated teachers in America’s classrooms, most of whom, polls show, aren’t active in their union and don’t worry much about how the small print in their contracts protects them.”  What poll is this?  This is divide-and-conquer, and it is not true.  I consider myself to be a motivated teacher who actively supports my union.  Remember — if a teacher is really too lazy to do their job, they probably will be lazy in supporting the union too.

Page 45:  Poor teachers make up “a fourth or even a third of the teaching corps.”  That is a big percent.  My experience in four schools puts the number of hopeless causes at less than 1%.

Page 51:  Good reminder that original point of TFA was to “ease the shortage of teachers across the country.”

Page 53:  Interesting that original plan in Wendy’s thesis was for corps members to co-teach with a mentor.

Page 54:  “67 percent of all TFA’s 21,000 alumni would be working in public education.  That’s more than 14,000 of the “best and brightest””.  I’m so sick of hearing this stat.  I wrote a whole post on it once. In short, this is based on a self-selecting alumni survey in which less than 60% generally responded.  Also ‘in public education’ is very broadly defined including being an admissions officer at a college, or something like that.  This stat sounds nice, but it is not true.

Page 57:  Escalante was not the hero that he is always portrayed as.  In reality, he team taught with another teacher and built his calculus program up over a number of years with the top students in his school.

Page 59:  Clinton and Porter propose “an education system that develops first-rate teachers and creates a professional environment that provides rewards for successes with students, real consequences for failure, and the tools and flexibility required to get the job done.”  But ‘successes with students’ wasn’t defined as performing on a standardized test only.

Page 63:  “TFA training was not destined to get better for many years.”  Unfortunately, it is still completely inadequate.  As they’ve scaled up to 5,500 corps members, the training has gotten worse, I think.

Page 66:  Finally an admission “It was the beginning of using data — however inaccurate the data might be because the tests were imperfect — to drill down and link students to increasingly more targeted sources of their performance.”

Page 69:  Says that TFA’s retention rate in 2008 is 92 percent.  This only counts the number of TFAers that begin their first year (don’t drop out in institute, which is a significant number as they realize that their training is setting them up for failure) and make it through their first year.  Another 3% quit during second year, based on their numbers, making it 89%, or 1 out of 9 who make it though institute but not through their commitment.

Page 70:  most TFA recruits “had succeeded.”  Only by TFA’s self-reported metrics.

Page 73:  Describes KIPP contracts for parents and students.  This would scare away many families who feel they can’t uphold the contracts.  And when some kids do break the contract, Brill doesn’t mention what happens to them.

Page 75:  KIPP trains students with the SLANT model.  “Sit up straight.  Look and listen.  Ask questions.  Nod your head.  Track the speaker.”  Isn’t forcing a kid to nod his or her head kind of strange?

Page 76:  This is one of the strangest paragraphs I’ve seen.  It claims that ‘Special Circular #6′ “was sixty-seven pages devoted to giving teachers an extra period off during the day”  Now, I’m a teacher in New York City, and I can tell you for sure that I did not get an extra period off every day beginning in the 1997 school year.  I don’t know what he is talking about here, and this is one of those points that the casual reader will see and get very angry about the greedy teachers and their unions.

Page 78:  The famous 3 effective teachers in a row study from Dallas is finally invoked.  The idea is that one group of 30 fourth graders with a 60% starting score went up to 76% after having three highly effective teachers in a row while another group of 30 fourth graders with a 60% starting score went down to the 42% after having three ineffective teacher in a row.  I wrote a whole post about this Jordan-Mendro study recently. This study had a lot of flaws in it, as do most value-added studies.

Page 82:  Johnston is described as having been “brushed off b Greenville, Mississippi school officials when he suggested how they might restructure things to make the school work better.”  But he had just taught there for two years.  He has not yet earned the right as a new teacher to do that yet.  You’ve got to stick around.

Page 82:  “Schnur and Johnston quickly realized that they shared a belief that what really mattered in fixing schools was getting better teachers and better principals.”  This is the first mention of principals, I think, as people to blame other than teachers.  And based on my correspondence with Schnur from above, he believes it is more complicated than that.

Page 85:  “It is true, though, that states soon began to game the system by defining their standards of proficiency down.”  This is an important sentence that could be discussed more, but isn’t.

Page 87:  “Bloomberg had not talked about school reform in his 2001 mayoral campaign the way he and Klein later would discuss it — in terms of eliminating union protections.”  How is that ‘school reform’?

Page 90:  “recruits from Rhee’s NYC Teaching Fellows would have to join the union, but they were not likely to hide behind union contracts when it came to throwing themselves into their work.”  I trained the NYC Teaching Fellows for several years and they, just like regularly trained teachers, don’t ‘hide’ behind union contracts.  We do our jobs, which includes taking grading home and planning at home too since when the treadmill of the school day begins it is tough to get paperwork done at school.

Page 96:  Says “The teaching fellows turned out to have been slightly to significantly more effective.”  Another value-added study that is, I believe, flawed.  I trained the fellows and they, like most first year teachers, struggled so would not truly teach better than experienced teachers.

Page 98:  “granting paid sabbaticals” is mentioned with contempt.  So, as a New York City Teacher, we can, after 7 years, take a 1/2 year sabbatical where we have to take courses to further our professional development.  If you wait 14 years, you can take a year.  Is this that bad?

Page 99:  About principal Lombardi “had also fashioned an end run around the ost onerous union contract restriction, enabling him to get past the tenure and rating system he had complained to Klein about in order to stock his school with teachers committed to his mission and get rid of the rest.  Lombardi would target the teachers he though were laggards and make life miserable for them, constantly observing their classes — which was allowed — but the union would lodge a grievance for it, charging harassment.”  He makes it sound like Lombardi found a clever loophole, yet this is exactly what principals should, and do, do when they have teacher they want to fire.  This passage demonstrates that it is possible to make things uncomfortable for teachers.  Some of these teachers, they say, voluntarily transferred to another school — which doesn’t really fix the big issue, but surely there were some of these teachers who worked harder as a way of getting the principal off their backs.
Follow garyrubinstein on Twitter
pages 1 to 100
pages 101 to 200
pages 201 to 300
pages 301 to 350
pages 351 to 400
pages 401 to 437

33 Responses

  1. Very thorough post, Gary. Would you recommend reading this book?

    The primary question that I want answered by all in this debate from Duncan to Rhee to Ravitch is the following:

    “If a given public school were able to close the ‘achievement gap’ as measured by a standardized test, would the fact that poverty still exists remain a problem?” in other words, let’s say TFA fulfilled it’s mission; would policymakers then claim that poverty isn’t a systemic issue since it can be ostentatiously overcome? Is it a problem that the kids are born into poverty in the first place?

  2. Jack from MD

    Gary:

    Thanks for taking the time to summarize the first 100 pages of Brill’s book. I especially liked what you said about page 27 and the international PISA scores. Mel Riddile’s analysis of PISA does show that the US is first in the world if you consider only schools with less than 10% poverty. It is unfortunate that so few people know that, but not surprising since it doesn’t fit with the model that school reformers portray. I post that info a lot when I comment on blogs, but more people do need to know that. In fact, I believe it should be a source of pride for the US, that we are number one if… In other words, we US teachers are doing a great job, and maybe, just maybe, it’s the other countries that should be looking to us to see how we do that. Maybe they would want to copy some of our teaching methods then!

    Perhaps the area that we, the US, should be focusing on is how do we reduce our horrible rate of poverty? The data clearly shows that as the poverty rate goes up, our PISA scores go down.

    Jack, high school math and computer science teacher

  3. Stuart Buck

    If TFA training is as bad as you say, what explains the fact that study after study finds TFA teachers performing either as well or better than certified teachers, even those with more experience?

    • Gary Rubinstein

      Stuart, There are even more studies that say the opposite. I know someone at the DOE in NYC who said they have an internal report about value-added for first year TFAers and they are not doing very well. Maybe some of the other studies include second year TFAers who, on average, are very good. I’d let a second year TFAer teach my own children, but never a first year.

    • Demian

      The evidence TFA cites in its favor tend not to be peer-reviewed research. When a meta-analysis was done that looked at the results of peer-reviewed research, “Retention rates for Teach For America teachers are low; and corps members’student achievement results are, at best, mixed.” Refer to http://greatlakescenter.org/docs/Policy_Briefs/Heilig_TeachForAmerica.pdf

      And, these are using the rather crude metric of test scores – which, unlike a regular certification program, TFA puts particular emphasis on. So, we don’t know what other impacts (positive or negative) TFA made.

      • That’s not a meta-analysis in any way whatsoever — meta-analysis is a complicated statistical technique that those authors do not even claim to use.

        In addition, their representation of the research seems suspect. For example, they say of the Kane/Rockoff/Staiger study that “TFA teachers continued to have a negative effect on reading for two of three years,” whereas the actual study (which I link below) repeatedly emphasizes the superiority of TFA.

        • Demian

          You’re right, it’s a review of numerous research done on TFA. That’s my mistake, not the author’s.

          Of the study you refer (by Kane et al), the review notes some flaw in their methodology, “The comparison group was defined in a way that
          would minimize the effect of teacher preparation, because the authors included
          teachers licensed through “transcript review” and temporary permits in the same
          group as college-prepared teachers.”

          Still, despite the methodological flaw, what this study concludes is “…we find little or no differences in the average teacher effectiveness of certified, uncertified,and AC teachers…” This is hardly a glowing endorsement of TFA – especially when you factor in the significantly higher attrition. Moreover, if you look at the other peer-reviewed research cited in this the Heilig review, it paints a still poorer picture of TFA compared to certified teachers (even when accounting for years of experience).

          What is striking is that most teachers, including TFA, are not very effective in their first couple years. The few TFAers who do stay do pretty well (again, on a limited measure of test scores) – but most don’t stay. At best TFAers are stop-gap when more qualified teachers are not available. It is poor policy to think TFA is any kind of solution – at least how it currently works.

          What would be helpful is if TFA operated like AmeriCorp and _volunteered_ to help real teachers in high-needs schools, perhaps while they get their certification.

          • That study does ascribe higher effectiveness in math to TFA. But even if you ignore that finding and assume that TFA teachers are equal to other teachers, how did they get to be even that good if their training is so inferior?

        • Demian

          As I said, all first year teachers don’t tend to be very effective. So, even if we’re very generous in our reading of the research and say TFAers do as well, that’s not saying much. However, to the extent they can even do this, I would suggest it’s mostly due to how TFA training puts a narrower focus on raising test scores. Certified teachers are typically not trained to teach like this, but even so, they still tend to get scores up at least as much (actually more if we just look at peer reviewed research – which is what we should be doing)

          • Stuart Buck

            I cite three studies below that, by my reading, find TFA teachers to be better than experienced teachers, not just first-year novices.

            (Peer review isn’t relevant; lots of peer-reviewed studies aren’t very good, and lots of non-peer-reviewed studies are great.)

          • Stuart Buck

            To amplify, if you are interested in what a real meta-analysis would show, an elementary requirement of such a procedure is to seek out everything that has been written on a particular point — especially including stuff that isn’t peer-reviewed. That’s because peer-reviewed status is typically biased (look up the term “publication bias” or “file drawer effect”).

            Looking just at peer-reviewed publications is a bad idea if one wants to know the truth about anything.

    • This comment is so untrue – the studies on TFA, even the ones it likes to use, don’t show what you suggest in any significant way – that I have to wonder what your intention in posting it was.

      • Demian

        r u referring to the original comment or my reply?

      • Here are some of the best-done studies on TFA:

        http://www.dartmouth.edu/~dstaiger/Papers/nyc%20fellows%20march%202006.pdf

        Key finding: “classrooms of students assigned to Teach for America corps members scored .02 standard deviations higher relative to certified teachers. . . . The results of our analysis suggest that even the small positive difference in value-added for TFA corps members’ teaching math is enough to compensate for their higher turnover. ”

        http://www.urban.org/publications/411642.html
        Key finding: “The findings show that TFA teachers are more effective, as measured by student exam performance, than traditional teachers. Moreover, they suggest that the TFA effect, at least in the grades and subjects investigated, exceeds the impact of additional years of experience, implying that TFA teachers are more effective than experienced secondary school teachers. The positive TFA results are robust across subject areas, but are particularly strong for math and science classes.”

        http://www.mathematica-mpr.com/publications/pdfs/teach.pdf
        Key finding: “Looking at student outcomes, we found that TFA teachers had a positive impact on the math achievement of their students—average math scores were significantly higher among TFA students than among control students. . . . This impact is equivalent to an effect size of approximately 0.15 of a standard deviation and translates into roughly 10 percent of a grade equivalent. . . . When we restricted the analysis to novice teachers, the impacts of TFA were the same or larger than those reported for the comparison with all teachers. Compared with their novice
        counterparts, novice TFA teachers generated math test scores that were 0.26 standard deviations higher, on average. The impact on reading scores remained statistically insignificant based on the novice comparisons.

        • Demian

          What do you mean by “best” (beyond supporting the conclusion you like)?

          The first one suffers from the methodological problem I mentioned above (regarding how certified teachers were identified).
          The second was not peer-reviewed and was also flawed as the Heilig review notes, “the study was critiqued by the What Works Clearinghouse
          at the U.S. Department of Education’s Institute of Education Sciences (IES)
          for not linking students with the teacher who taught them; instead, students were
          matched to teachers based on a test proctor and classroom demographics. …IES identified this as an important limitation in the study’s data that could lead to
          “imprecise” and perhaps misleading estimates.”

          The third one is also not doing comparison to certified teachers. The study’s authors note: “Compared with a nationally
          representative sample of teachers, the control teachers in the schools in
          our study had substantially lower rates of certification and formal education training.” And yet, “Compared with this underprepared group, overall TFA teachers’ students
          showed gains similar to those of comparison teachers in reading and better in mathematics,
          though students’ scores remained low overall, hovering around the 15th
          percentile for both groups of teachers. However, the positive impact was found
          only for TFA teachers who had obtained training and certification in their second
          and later years in the classroom. First-year TFA teachers did not have a positive
          impact in either mathematics or reading; a negative coefficient in reading was not
          statistically significant.”

          • The third study: of course it’s not doing a comparison to 100% certified teachers, but it’s comparing TFA to the teachers who are otherwise in the same schools (some of whom might indeed be uncertified). That’s a more relevant comparison, isn’t it?

            The third study is great because it’s a randomized trial. You don’t get better than that.

            The second study is typical of any study done with North Carolina data. It’s a minor problem that doesn’t discredit the study, any more than it discredits Helen Ladd’s work.

            The objection you raise to the first study makes little sense to me, and I don’t see that the Kane/Rockoff/Staiger study even uses the term “transcript review” (which you put in quotes). Can you pinpoint a specific page from that study? (Not the union-funded analysis, the actual study?)

        • Demian

          To say that peer-review can be flawed does not mean it’s not a higher standard. While non-peer reviewed studies can still be valuable, they are also more suspect. And as the Heilig review (and other reviewers) note for the TFA ones, the non-peer reviewed ones have flaws.

          • The “flaws” are nitpicky and minor.

            Besides, even if you had a good point about the value of peer review, you’re wrong about the status of these particular papers.

            The first paper I cited was published in a peer-reviewed journal: Economics of Education Review. I linked to the non-peer-reviewed version, but here is the peer-reviewed version: http://www0.gsb.columbia.edu/faculty/jrockoff/certification-final.pdf Guess what the final version says: exactly the same thing about the superiority of TFA teachers. To wit, “classrooms of students assigned to TFA corps
            members scored 0.02 standard deviations higher relative to certified teachers.”

            The second paper I cited is under review at JPAM.

            The third paper was eventually published in JPAM in 2006 here: http://web.missouri.edu/~podgurskym/Econ_4345/syl_articles/glazerman_MPR_TFA_JPAM.pdfGuess what the final version says: “We found that TFA teachers had a positive impact on math achievement and no impact on reading achievement. The size of the impact on math scores was about 15 percent of a standard deviation, equivalent to about one month of instruction. The general conclusions did not differ substantially for subgroups of teachers, including novice teachers, or for subgroups of students.” Again, not different from the non-peer-reviewed version.

            So if that’s the best you can come up with, maybe you should just trust these studies as good evidence.

        • Demian

          > “That’s (comparing TFA to other poorly qualified teachers) a more relevant comparison” (regarding the third study).

          This only tells us that TFA are a tiny bit better than other poorly qualified teachers. At best, this is an argument for using TFA as a stop-gap measure – not a solution for addressing the achievement gap.

          > “It’s a minor problem that doesn’t discredit the study” (regarding the second study)

          No, it doesn’t discredit the study, but it makes its conclusions suspect. To quote from http://ies.ed.gov/ncee/wwc/publications/quickreviews/QRReport.aspx?QRID=53, “The WWC has reservations about these results because students’ ability in these subjects may have varied in ways not controlled for in the analysis and because study data do not identify students’ teachers with certainty.”

          > “The objection you raise to the first study makes little sense to me” (regarding the first study)

          This was a criticism noted in the Heilig review. I also do not find any clear source in the actual study. I have sent the review authors an email. Perhaps we’ll see.
          This study also seems to under-estimate the impact of high-TFA recuit turnover. It claims their very minor advantage in math somehow makes up for their high-turnover (never mind impact on reading). This assertion seems to bely physics

          It is a strong claim to say that five weeks of training will make someone a better teacher than certified teachers (or even than experienced teachers as is sometimes claimed). Such claims require strong evidence. If TFA has figured out some magic bullet, then why aren’t other schools of education adopting it? And, why are researcher showing the efficacy of TFA’s approach – independent of TFA?
          The three studies have at best modest support for TFA and with the mathematica study only against other poorly trained inexperienced teachers. However, other peer-reviewed studies, such as the one done by Darling-Hammond, find TFA recuits do worse.

          • It makes no sense to dismiss the Mathematica study while accepting Darling Hammond’s. Darling Hammond’s study was a regression analysis, which can certainly be useful, but the Mathematica study was a random assignment study that compared TFA teachers to the same sorts of teachers who would otherwise be in those same classrooms. You just can’t get any evidence that is better than that, and to cite Darling-Hammond as superior is jaw-droppingly backwards.

          • “If TFA has figured out some magic bullet, then why aren’t other schools of education adopting it?”

            Actually, there may be no magic bullet other than smarter candidates. If those same TFA candidates had more training, they might be far better.

        • Demian

          I didn’t say anything about the relative quality of the mathematica study vs the DH study. I compared their conclusions. The Mathematica study merely found that TFA was very slightly better than other very poorly qualified candidates. Whereas the DH study “found no instance where uncertified Teach for America teachers performed as well as standard certified teachers of comparable experience levels teaching in similar settings.” These are not necessarily contradictory findings. Neither support the conclusion that TFA training is excellent or that TFA is a viable solution to addressing the achievement gap.

          You acknowledge it may not be TFA’s training, but its recruitment of supposedly smarter candidates, that lead to what are at best meager results – especially for first year recruits. So, assuming TFA is able to recruit good candidates, why not have TFA switch to a student teaching model for the first year? This would actually stand a better chance of impacting the achievement gap, would provide better training for TFA recruits, and would avoid some of the burn-out of throwing recruits into the fire with insufficient training. TFA’s ability to recruit is certainly impressive – although not surprising given that a considerable amount of their hundreds of millions goes to marketing/recruitment. The fact that they don’t push this more responsible approach makes me suspicious of the organization’s (not the recruit’s) motives.

  4. C

    NIce work as always, Gary. I work in the building of the company that publishes this book. A few days ago they had a huge display touting its release. I hadn’t known of it beforehand (though I have read Brill before). Disheartening.

    Politics values repetition above almost all else, and the “reformers” are using this extremely effectively. It’s a scary time.

    Do you think there’s any way to sort of “unify” the Ravitch movement (for lack of a better term)? I suppose that’s what Save Our Schools is, right?

  5. Christopher

    Looking forward to reading the book…can’t respond intelligently without having done so. However, as a first time visitor to your blog your tone makes it difficult to trust your critique.

    • Gary Rubinstein

      Christopher, This probably isn’t the best post for a first-time reader. Click on the favorites category to see some other ones. This is a daunting task pointing out all the things I disagree with in a 500 page book. Look at some older posts, and hopefully you will understand my tone in a bit more context.

  6. Richard Munro

    very well said. Allowing for a sabbatical is quite reasonable it gives teachers a rare chance to concentrate on improving their skills. I took a sabbatical to teach and live in Spain; I learned a lot about teaching English and of course improved my Spanish. If my Spanish is as good as it is it is because I have had a lot of practice and the chance to live in and study in (not visit as a tourist) Spanish-speaking countries.

    • I’m surprised Brill goes to the trouble of denigrating sabbaticals. In my district, if you successfully apply for a sabbatical, you must be able to document how your time will benefit your employer and your students.

      I don’t know that sabbaticals are terribly rare in non-union professional jobs, either. My spouse’s company provides full-pay sabbaticals to its employees without requiring any kind of benefit to the company through the time away. It’s a reasonably common benefit where I live.

  7. K

    Wow, this is insanely detailed. Love it.

    I particularly liked your point about one role of unions being to prevent certain teachers having 3 preps versus 1 prep. I always find it hard to come up with a counterexample to the “adults vs. children” debate although in my gut I feel that there are few things, maybe no things, that I would advocate for as making my life as a teacher easier that would not also make kids’ educations better. Fewer preps, more grading/conference/planning time, smaller class sizes, aides in the classroom…I get that these things may not happen or may not be priorities for various reasons. But none of them would harm kids and I believe all of them would help kids. The prep example, though, I think is the clearest and easiest for anyone to understand.

  8. The “Dance of the Lemons” with teachers thing is a popular press meme in California, usually used to bash teachers’ unions. But the thing is, teachers wouldn’t be transferring from site to site because they have been made unwelcome at their old site if principals would do the paper-consuming, time-heavy work of documenting the issues they have with a teacher’s practice. I have heard principals complaining about teachers they don’t think are right for the site, but I rarely see them suggesting PAR or observing practice. It’s easier to convince a teacher to take a voluntary transfer than it is to move towards having the teacher fired.

    I also suspect that some principals don’t want to do the paperwork because they don’t necessarily think the teacher is not very good – it’s just that they don’t like the teacher personally, or don’t like the pedagogy that teacher uses.

    Another issue I have with the idea that classrooms are loaded with bad teachers destroying children for fun and profit is that it sees teaching as totally static. I teach at a high-needs school. All of the kids live in poverty, all are children of color and about half are English Language Learners. Working with children and families at a high-needs school is far different than working in a school serving middle class white children. People who can be successful teaching at a low-needs school aren’t necessarily equipped to excel at a high-needs one. Some teacher transfer works to put teachers in environments where they can succeed – if a teacher was a “lemon” at my school, they may not be on the west side of town.

About this Blog

By a somewhat frustrated 1991 alum

Region
Houston
Grade
High School
Subject
Math

Subscribe to this blog (feed)


Subscribe via RSS

”subscribe

Reluctant Disciplinarian on Amazon

Beyond Survival On Amazon

RSS Feed

Subscribe