Nov 19 2011

Peer Reviewed Research About Effectiveness (or lack thereof) Of TFA Teachers

This is a copy of a great post from Education Week.
The original can be found here

In the past few weeks, Philip Kovacs has taken a public stance in Huntsville, Alabama, raising questions about the school district’s decision to invest significant resources in bringing Teach For America interns to that city. This is a guest post by him, describing his concerns.

Guest post by Philip Kovacs.

When I first heard about Teach for America, I thought it sounded like a good program. Given a hard to staff school in a poverty stricken city, why not let enthusiastic college graduates with some training go in and put their hearts and souls into classrooms that would otherwise be staffed with less-prepared or unprepared individuals?

That line of thinking began to change as I read column after column problematizing the venture.

Over the past two years, my city (Huntsville, AL) laid of over 300 certified teachers, many of them graduates of my program. Then I found out our city had signed a $1.7 million deal for at least 170 TFA members. That figure is not firm, as the contract stipulates “at least” 170 TFAers over the next four years. Recent reports suggested it was $1.9 million.

When I learned that TFA members…They are not teachers, any more than a recent undergraduate with 5 weeks of medical training is a doctor, or a lawyer, or a police officer. (Given the choice between someone with 5 weeks of training in any of those 3, or someone trained via an extensive program, which would you choose?)

…when I learned that TFA members were coming to my city, that they were going to cost the District an additional $5,000 per year more than other new teachers cost, I became very interested in the research supporting the program. I bought and read Learning on Other People’s Kids, a book written by a former TFA trainer and a respected scholar. Have you read that book? It is dedicated to TFA members, and it is eye-popping to say the least.

Regarding the research supporting TFA, what I found was, well, nothing.
There is no peer-reviewed research that supports the program.

There are two paid-for studies, one which acknowledges its own flaw re: sample not reflective of population. The other was identified by the Department of Education as flawed for misidentifying causation. You can learn more about both by reading this excellent post, “Good” vs. “Poor” Studies of Teach For America. (In fact, please visit the site and tell Russell hello. I am in his debt for his help preparing me for the school board meeting this past Thursday.)

I have, however, seen two, peer reviewed pieces, one of which can be downloaded here: Teach for America: A Review of the Evidence (the other is forthcoming in the rather conservative Kansas Law Review) suggesting that TFA members are worse for elementary students in both reading and math, as determined by test scores. NOW, I don’t think test scores are the best indicator of teacher or student performance, so I’d be more than wiling to judge the success of both teachers and students using other data points. But test scores are the gods of the current educational reformers, and if their gods don’t support what they are doing, there is some sort of problem in the temple.

At best the empirical evidence is mixed, at worst, it is damning. Given that the organization has been around for 20 years, if it was so good, why aren’t there dozens of peer-reviewed reports proving it?

Furthermore, if I were a high level Teach for America employee or the founder (who makes the paltry sum of $350,000 a year heading her non-profit), I would commission study after study to prove the program’s success.

Goodness, I would probably go and buy a study from the Manhattan Institute or the Heritage Foundation, or Cato or any other report-producing think tank that hates teacher’s colleges and the teachers that come through our doors.

There are dozens of these organizations, if there weren’t, we wouldn’t be living in the most anti-teacher moment in history, save the early1600s when there was an active effort in the colonies to keep all people illiterate, not just the slaves.

Here is something else to consider. Market based reformers claim that competition works and that the market should determine success. If this was the case with TFA, the organization would not need $25,000,000 from the federal government or another $25,000,000 from the Broad Foundation to stay afloat. Its overwhelming success would earn enough dollars from the cities who use its graduates. In reality the “free” market can be ignored when you have enough money to buy yourself out of it.

If my city could not find qualified, certified teachers, I might support TFA recruits entering classrooms as part of a rigorous scientific study, if and only if parents were notified and given the choice to opt out, as is required by law.

The fact is Huntsville laid off 300 teachers over the past two years, many of whom were graduates of my university, and there is no teacher shortage in our city. In fact, competition for jobs is extreme with one principal telling me that she has 50 applications for every position…50!

I am also disturbed by the cronyism involved in all of this. Our new superintendent graduated from the Broad Foundation’s super-prep center. The Broad Foundation supports TFA. it doesn’t take an architect to connect these dots. Importantly, there are other non-traditional programs that prepare individuals for teaching. I went through one! There is an excellent program in North Carolina that has an amazing retention rate, so why not go with them given the overwhelming research that shows experience matters?

(For research challenging the argument that experience does not matter, see from the left leaning EPAA: Does Teacher Preparation Matter? Evidence about Teacher Certification, Teach for America, and Teacher Effectiveness. . See also, from the very right-of-center: Eric A. Hanushek & Steven G. Rivkin, Pay, Working Conditions, and Teacher Quality, 17 FUTURE CHILD. 69, 7778 (2007).

After consideration for my untenured self and the futures of my students, I decided to go and ask our Board of Education a few questions. They are included below in case you want to ask them to your board members, and I encourage you to do so. I suggest you practice them more than I did, so you can look up more often in order to make eye contact with the Board. I was, in fact, pretty nervous, which is silly because 1) they work for me and 2) there is not a more incredulous audience than 25 high school juniors!

Philip Kovacs is a former high school English teacher who teaches teachers at the University of Alabama, Huntsville. He is the editor of The Gates Foundation and The Future of U.S. Public Schools, priced to move at Amazon.com… His research interests includes education in and for democracy and critiques of neoliberal think tanks, institutes and foundations. You can follow him on twitter @philipkovacs.

Here’s a link to the YouTube clip of Philip talking to the school board, note the camera in the back, as he called the news and convinced them to come to the meeting.

Here are the questions he asked the school board:

1. You’ve claimed there is “overwhelmingly positive research” in support of Teach for America (TFA). This is demonstrably false. Why are you making this claim when there are only two, non-peer reviewed reports on TFA, both of which have been discredited by scholars?

Furthermore, given that TFA has been around for over 21 years, if they were so successful, shouldn’t there be dozens of peer-reviewed studies showing that success?

As there is no peer reviewed research on TFA, this is in fact an experiment, as such, will you give notice to parents whose children will take place in your experiment, as is required by law?

If your answer is “no,” are you in fact demanding that all families participate in your experiment, or will parents be allowed to place their children in classrooms with professional teachers?

2. Will you guarantee that TFA members will be equitably distributed across the district and not only placed in Title I schools, which would be in direct violation of the ongoing federal desegregation order re: Hereford v. Huntsville?

Furthermore, will you provide the media with the percentage of black teachers laid off and the percentage of white “new faces” replacing them, or will the media need to use the Freedom of Information Act to determine those figures?

3. I am aware of several alternative programs that have better retention rates than TFA (see, for example, North Carolina). Did you solicit competing bids from these other organizations? If so, where are those bids, if not, why not?

4. Dr. Robinson claims the $5,000 per year is for professional development, but TFA claims the money must be used towards paying off college loans. Who is incorrect on this point?

Are my tax dollars going to professional development, or am I paying off other people’s college debt because, quite frankly, I have plenty of my own college debt.

5. If, in two years, your $1.9 million experiment on Huntsville’s children has not produced “overwhelmingly positive results,” will you hold yourselves accountable and resign?

For the record, what is your metric for determining “overwhelmingly positive results?”

6. Will you provide members of the media with the exact amount of money you have given to the Broad Foundation since Dr. Wardinksy arrived, or will you make them use the Freedom of Information Act to determine to that figure?

7. Will you agree to stop outsourcing public education and to immediately end outside-of-district spending until the media has had time to determine, exactly, how much of our tax dollars you have given away?

What do you think of Philip Kovacs’ approach and the questions he has raised? Is Teach For America present in your schools? What do you think of their role?

12 Responses

  1. MeghanK

    In my city they are also laying off experienced teachers to hire TFA members. This article was recently in our local paper: http://www.commercialappeal.com/news/2011/nov/01/teach-america-memphis-produces-effective-teachers/
    The report was produced by the Tennessee Higher Education Commission. I wish someone would debunk it. I would, but I’m not completely confident of my knowledge of statistics. Maybe, if you’re not too busy, you could?

    I looked at the report, and I don’t understand T-scores. I looked up T-score, and I still don’t understand.

  2. Ms. Math

    Philip has some decent points, and some points that don’t seem very well argued when analyzed from a mathematical proof or research perspective. I understand why he feels like TFA deprofessionalizes education. As a Phd student in education I’m seeing how much we know as a field that TFA is not capitalizing on-there is more to learn about teaching than can possibly be learned in five weeks. However, I think he’s missing some important points when making the claim that TFA is not a good thing for our educational system.

    Research in education is a huge challenge-statistical verification of causal relationships in education seems almost impossible. I don’t think that I, or anyone else, can prove if TFA’s model is a net positive for our educational system. And when I say positive it’s not because I believe that all of our kids are making amazing progress and meeting all of our ambitious goals in our first two years. So, for all of you new teachers or applicants reading this post and wondering if you have made a horrible mistake by committing to serve what Philip seems to think is an ineffective and harmful organization, let me see if I can give you a different perspective on why that it is good you are here.

    The research reports I’ve read on TFA’s effectiveness(some mentioned above and a few more he missed) seem to support the idea that we are about as good as traditionally trained teachers and better than other brand new teachers. Our retention levels are about average when compared to others at schools we teach at(And yes, that does imply that retention levels across the board are atrocious).

    However, I think it is a mistake to end the analysis of a TFA teachers impact at the end of their second year. At the school I worked at many TFA stayed on and became some of the most respected and active teachers at the school. Over half of the school improvement team charged with addressing issues of school-wide failure were TFA teachers. Was it ideal that newer teachers were coming up with ideas to fix our school? No. But, I don’t think anyone else at the school was planning to do it if we weren’t there.

    And yes, lots of people leave their first schools, and I did too, but I don’t think that many of our stories end when we leave our placement school. I still follow my kids on facebook and they inspire me to work harder. The work that I do with all of my energy is focused on addressing an issue I first saw as a corps member. Those kids experiences in math classes are the reason that I’m studying what I am in graduate school. And I know stories are not proof, but the reason I’m in education for the rest of my life is because I experienced the inequity first hand. The statistics I read about in Savage Inequalities became stories about real kids whose hopes and dreams of being doctors and lawyers and teachers were being held up because they couldn’t make sense of the disaster of a math education they had received.

    So how does my story generalize? Because my story isn’t proof of much of anything by itself. I went to a statistical talk recently about how incredibly hard it is for a teacher to make huge dents in student achievement. Socio economic status, prior education, mother’s education level, organization of the school, etc. all played a huge role. This particular statistician who’d been doing research on accountability models felt like based on the data she had seen that teachers accounted for about 15 percent of student learning and the rest was due to outside factors. If you change those outside factors by changing the way math is taught, changing the way the school is run, the parents participate, etc. you can explain why some TFAers are able to make a lot of progress and become existence proofs that the model can do wonders.

    So, what is the point of putting these high achieving college graduates into the midst of a broken system and telling them for two years that they can change things that are probably beyond their control? It’s a learning experience. For the teacher. And it doesn’t bother me that we are learning on someone elses kids because we are doing about an average job of it. Yes, TFA is an experiment, but so is almost everything in education in every classroom. Embrace that it is an experiment instead of saying this is a problem. Teaching is always an experiment, even for people with PhD’s in education. It’s not like putting kids in traditional math courses has even shown that traditional teachers are not doing irreparable damage to the kids understandings of math. Look at our international scores. And if the TFA teacher thinks about their time as a scientist they are going to learn from it. They will try different solutions, see how they work out, and create models and hypothesize causal relationships. All teachers do this-I’d ask Philip what scientific proof he has for his favorite lesson and ask if he sent a note home asking permission for the kids to participate in it given that it might not result in learning as demonstrated by statistical models.

    So what do we learn? We are learning how hard it is for those kids to come to school every day and face what they do. We realize that for the first time in our life effort is not always enough. We push up against the restraints of the system and realize how hard it is to make change. We realize that quick fixes, fancy curriculum, and more professional development are not going to fix anything overnight. We try things out and see them fail and learn about more variables in the complicated education system.

    And yes, if we quit, and move on, and don’t use everything we’ve learned, then TFA is probably not a net positive. However the statistics also show that many TFAers stay in education. And I think that is where the true value of TFA lies and why it’s worth supporting. It can’t be argued that it convinced elite college grads to enter a field that has low status and pay. They made something that used to be easy to get into competitive and appealing to a entirely different audience. And this audience matters-because they often take their experiences and lessons and stories and try to fix something. And now all the shiny idealism that you see in the “I just got accepted to TFA and I’m going to change my kids lives in a week posts” turned into realism about the situation. These kids can learn but they have been given an awful place to do it.

    So, take your experiences from TFA and don’t feel bad if you tried your best and your kids are still about average. Learn from this time, write about this time, and don’t forget this time when you go on to figure out how you can best contribute to the world.

    • Ms. Math,

      Thank you for your post. I have asked my superintendent, who is mentioned in Dr. Kovacs’ post above, to provide me with the evidence that he claims proves that TFA teachers are more effective. He has not produced such data.

      You seem to claim that any claims to demonstrate a pattern or tendency is fundamentally flawed, and yet you also want to claim that the current system is “broken” and that for evidence of this we should “look at our international test scores.” Finally, you argue that you believe that TFA produces a net positive.

      So which is it? Can we use data to evaluate the program or not? And if not, why are you so willing to use data to evaluate a “traditionally trained” teacher’s performance?

      You state that if TFA teachers “quit, and move on” that TFA is probably not a net positive. You then claim that “statistics also show that many TFAers stay in education,” but like my superintendent you don’t offer any evidence to support your claim. Could you post some of these “statistics” that support your claim that TFAers stay in education? I would appreciate it.

      The study by Heilig and Jez from June 2010 shows that TFAers have an attrition rate of more than 80%. You may review that study for yourself at http://nepc.colorado.edu/publication/teach-for-america. There are many other studies cited in that study that have similar findings.

      Finally, you claim that TFAers should “learn from this time” and incorporate those things learned as they figure out how to “contribute to the world.”

      Could you share with us what insights these TFAers have had, and how their insights have improved our schools? TFA has been around since 1990. What evidence of changes can you cite for us that they have had upon our schools?

      The effect of Broad Foundation funded organizations on my children’s school system in just 5 months are as follows:

      1. Larger Classroom Sizes and Higher Student to Teacher ratios.
      2. Expanded Central Office expenditures and reduced classroom expenditures.
      3. Significant cuts in Special Education spending to fund significant increases in spending for Teach for America
      4. An increase in the number of schools closed or slated for closure, all while claiming that he doesn’t believe in closing schools.
      5. Teachers and Administrators operating in fear.

      You may see the evidence for these changes at my blog at http://www.geekpalaver.com if you wish.

      If these are the benefits they have gained, I would prefer that they go own about their chosen careers without inflicting such insight upon our kids, and their teachers.

      • I can’t find the stat-I was doing a research project on TFA and remembered a bunch of stats but didn’t take the time to cite them since posting was not counting for a grade!

        I believe it was in this dissertation study about TFA-though all I’m locating is info about how many leave teaching and not how many stay in education.
        http://digitalarchive.gsu.edu/msit_diss/86/

        I completely agree that many TFAers leave teaching.
        I don’t have any real evidence about TFA changing the landscape in the education system. The issues are too complicated. I do know it changed me and some of my friends and I believe we are doing good for education and might not have been otherwise-I believe my argument suggests but does not prove this possibility of why TFA is good.

    • Was it ideal that newer teachers were coming up with ideas to fix our school? No. But, I don’t think anyone else at the school was planning to do it if we weren’t there.

      Of the initiatives the new teachers planned, how many were they able to implement? How many were left as ideas to fix the school that would not be implemented? And of those implemented, how many continued after the TFAers involved left?

      One of the aspects of education that TFAers miss if they choose to leave after their two years (or after three, or during their two years) is how difficult it is to implement sweeping changes to a school culture. Those changes require buy-in from a heavy proportion of the staff, not just the people on the leadership team, and they have to be planned, with many small goals and ways to see if they’re working or not.

      New teachers also don’t have the institutional memory of the school to know what’s been tried before and what happen, so there’s often replication of the same old plans. For us veterans, it’s hard to see how this time will be any different.

      It’s not the size or the brilliance of the idea, it’s what happens when that idea is implemented. This is something of a fundamental issue with Teach for America: a lot of focus on new and big and innovative, and not so much on what happens to those new, big, innovative plans.

      For the record, “statistics show” TFAers stay in education only if you use a very wide definition of “education”. Those of us who stay in the classroom for years are few.

  3. April

    Gary,

    Just wondering if you have done any articles about TFA’s tendency to place teachers in Special Ed positions. As a special educator, I am horrified by this.

    April

    • Gary, I second April’s request. I’ve heard that something like 20% of TFA CMs are placed in Sped positions. As a Special Ed teacher myself, this is terrifying. The learning curve for working with students with special needs is even steeper than Gen ed classrooms. Those kids can’t afford completely untrained teachers. What kind of additional training does TFA provide these teachers BEFORE day one? Do they have extensive practice in writing IEPs and implementing BIPs? What kind of learning interventions have they been taught? Are they given a chance to work directly with students with disabilities during Institute?

      Also, I am pretty sure that parents of a student with an IEP would have just cause to sue a district for putting any uncertified teacher in a Sped position. Really would love to hear more about Special Ed and TFA!

  4. Andrew

    Does anybody have any really good peer reviewed studies that shows that ANY teacher training works? What I have seen has been weak.

    My guess is that experience trumps training by miles. Most who haven’t taught will be weak the first year or two – TFA trained, traditionally trained, or untrained – and will improve over time. If I’m right (I may not be – show me the GOOD studies), then TFA is lousy because of the retention problem, not because of the condensed training.

    One more thing. One needs to control for student teaching – student teaching, for most IS teaching, and hence experience. One shouldn’t assume that if classically trained teachers do better than TFA teachers it’s because of training per-se – to call student teaching training instead of experience seems a bit disengenuous.

  5. I hadn’t thought about it before, but it’s weird that TFA is still being funded like a start-up after two decades. Sure, funders are buying expansion now, but it’s still taking in a lot of education dollars that could be directed elsewhere. (Not that they would be, of course.)

    I’m increasingly concerned about TFA’s apparent interest in ECE. Their particular training model and focus strikes me as inappropriate for very young children. There’s also a lot of problematic discourse on “family background” and education outcomes within TFA (and education reform generally). In ECE programs, you’re going to have a lot closer contact with those families, and if you come in with strong ideas about what they need, you’re not coming in with respect and the willingness to learn about them and their children.

  6. Wess

    I, for one, joined TFA in part because I wholly, foolishly, believed all TFA teachers were outstanding teachers. I would LOVE to see more research done on TFA (or any teacher training program! Or anything in education!) that convinces me whether we are or aren’t making a difference–in the classroom, in the country, anywhere.

    The moral of the story: let’s all go into education research so we can settle this debate once and for all!

About this Blog

By a somewhat frustrated 1991 alum

Region
Houston
Grade
High School
Subject
Math

Subscribe to this blog (feed)


Subscribe via RSS

”subscribe

Reluctant Disciplinarian on Amazon

Beyond Survival On Amazon

RSS Feed

Subscribe