May 07 2011

Two Out Of Three Ain’t Bad — but is it true?

In response to critics that TFA teachers don’t have enough long-term impact, TFA replies with the statement from their annual survey “Nearly two-thirds of Teach For America alumni work in the field of education, and half of those in education are teachers.  Teaching remains the most common profession among our alumni.”

Now a statement like this is pretty strong and probably shuts up those critics, though it also probably leaves them scratching their heads.  How could this statement possibly be true?

I decided to investigate a bit to see how much of this was fact and how much was PR spin.  Before I get started, though, I’d like to explain something about myself to people who don’t know where I’m coming from with my occasional criticisms of TFA.  It’s not that I want TFA to fail.  There are two things I don’t like about misleading PR:  1)  Politicians often believe it and then make policies based on it, like “those TFA teachers keep teaching so we can solve the education crisis by firing all the overpaid veterans and replacing them with these TFA teachers.” and 2)  TFA seems to believe the PR themselves and don’t seem to feel the need to improve their ability to get more alumni to teach beyond the two years.

The first thing that you should notice when looking at a stat like “one third of alumni are still teaching” is the careful choice of words.  You’re only an alum if you don’t quit before  you become an alum.  And since somewhere between ten and fifteen percent of people who start in TFA do not complete their commitment and therefore become alumni, they are not considered in that stat.  That alone would move the number down to 30%, but that’s just the beginning.

The first report I looked at was the 2007 report.  At the bottom there was some fine print that said “The information in this report is based on self-reported data as of April 2007 and represents more than 57% of our alumni network.”

Well, 57% is pretty low, considering that it’s self-selecting.  This is not a random sampling by any means since those who are in education are more likely, I think, to respond to the alumni survey.

The 2008 report says “Percentages that reflect current data — as opposed to cumulative data — are drawn from our 2007 alumni survey, which received 4,097 responses, and represent 35 percent of our alumni network at the time.”  This doesn’t sound very reliable statistically.

The 2009 report says “Except where noted, percentages that reflect current data — as opposed to cumulative data — are drawn from our 2008 alumni survey, which received a 57 percent response rate and went out to our alumni from corps years 1990-2006.”  So now it’s back to 57 percent.

Finally, the recent 2010 report says “Data is self-reported and reflects 72 percent of our total alumni population.”  This is quite a jump, so I wanted to examine it.  Of course the size of the new corps is rising so if they were able to get 100% of the new CMs to respond, that would help.  There were 17,000 alumni the year before and 3,000 new ones that year.  But if they kept the 57% of the original 17,000 and got all the new 3,000 to respond, that would still be only 63%.  To get the 72% they claim, they would have to not only get all the new 3000 to respond, but they’d have to get 67% of the 17,000 people who they tried to get before.  I don’t think this is feasible since as the years go on, it gets harder to track people who you’ve lost.  I do admit that it’s possible that they made a huge effort to get those numbers up to answer criticisms about the low self-reported turnout.

Just under the ‘two thirds in education and one third in teaching’ stat in the 2010 report it also says ‘Nearly half of our corps members stay in their initial low-income placement schools for more than two years.’  Now, this, I believe.  CMs do often stay for a third year.  I did, though it was mainly because I needed to make up for the damage I had done my first year.  Still, this is a true stat.  But it also skews the ‘one-third of alumni are teachers’ statistic.  If you take the 3,000 new ‘alumni’ (there were 17,000 alumni from first 18 years, so 3,000 is a big percent of alumni) of which 1,500 are teaching a third year, that accounts for a large percent of the alumni who are still teaching.  I think of those third-year people as part of their own category which is definitely worth reporting and even bragging about.  However, if you don’t count them among the alumni who are still teachers (I know they are officially alumni, but it does cause a misleading stat), you would be down to about 25% alumni teaching, which still doesn’t account for the people who quit and never became alumni.

The one-third stat makes it seem like one out of every three alumni have chosen to become career teachers, while I’d say the number is more like ten or fifteen percent (of which I’m one of them.)  I think that ten or fifteen percent is actually pretty impressive considering that most of them (like me) weren’t planning to become career teachers before doing TFA.  They should be proud of the people who are still teaching, but be honest about their retention rate.

11 Responses

  1. MavorW

    Interesting blog. Although I was not a TFAer I was a Orleans Parish Teaching Fellow from the New Teacher Project in New Orleans in 2003.. And before that a teacher in the Los Angeles Unified School District. I worked with and socialized with TFA members in both cities. Will I appreciated many of the TFA members I was, as you blog suggest, struck with how poorly prepared many of the TFA members were. So many did not last long. I realize how difficult New Orleans Schools were before Katrina, but it was, in many cases like sending lambs to slaughter. I don’t think any of them are still in teaching. My overall opinion of TFA is that in the long run it will hurt the profession and learning. 10-15% doesn’t seem to good to me. Sound a bit inefficient and wasteful. What about taking the same resources and improving the teaching done my those more committed to the profession?

  2. MavorW

    I do know of two TFAers from Louisiana or Mississippi who are still teaching or did teach for a decade or so. However they both move to expensive private schools right after or soon after there two years were up. Doesn’t really support the TFA spin. One of the schools charges $19,000 a year for high school and has a class size of 15. Wouldn’t it be nice?

  3. Excellent post – I read awhile back that this statistic TFA loves to throw out to combat valid criticism of their ‘revolving door of inexperienced teachers effect’ is absolute garbage when examined honestly and critically (unfortunately, the mainstream media eats up this statistic and other TFA garbage uncritically all the time – e.g. George Will’s recent atrocious TFA PR puff piece).

    As you stated, this statistic is based on a low-response-percentage, self-selected alumni survey (so also, TFA members who have a horrible experience in the program – of which there are many, including myself – and drop out will never see this survey nor be a part of that statistic and presumably most of them are far, far away from education). Also, I would guess that the majority of TFA alumni who choose to respond to this survey had a relatively decent TFA experience and many of those who didn’t respond had a more awful experience and are thus also disproportionately far, far away from education as well (this is just a guess, but it makes sense). Finally, TFA defines “involved in education” in this regard remarkably broadly from what I’ve read/heard…all together, it adds up to an absolutely worthless, misleading statistic that they should be embarassed about presenting to the public…but they’re not…ah well.

  4. CarolineSF

    This is EXACTLY the technique that Edison Schools used to use — “84% of our schools are achieving positive gains” (but if you checked, they had excluded more than half of their schools before calculating the 84%). TFA founder Wendy Kopp is married to former Edison huckster, I mean executive, Richard Barth. Gosh, what a coincidence that they use the SAME dishonest techniques to come up with phony figures purporting to show success.

    http://www.pasasf.org/edison/edison.html

  5. Something else I think they do is very loosely define the field of “education.” I’m working as an admin at a university now, but when I filled out my survey this year, they only have higher education categorized as under “education.” I don’t consider myself to be working in education any longer, but TFA does.

  6. Gary,
    My pleasure meeting you in DC. Keep writing and presenting the facts and I’ll do the same.
    B

  7. Ed

    Actually, 57% response rate for a survey is decent in the real world, especially with voluntary surveys. It’s not stellar, but it’s not too bad. The concern you point out about the sample not being representative is the bigger issue–those who responded are probably skewed toward those who are still in education and those who had an overall positive TFA experience, and it doesn’t account for people who did not finish their 2 year stint and thus aren’t alumni. Also, the “in education” category is too broad and can encompass a lot (including working with high SES populations, in higher education, etc., not that there is anything wrong with that per se).

  8. Daniel

    A TFA teacher at my Bronx high school setting “high expectations” failed 20% of her kids then left for law school. I was one of the teachers helping those kids complete credit recovery work on Saturdays in their senior year so they could graduate, although I’m sure she is still taking credit for their graduation.

  9. Nora

    Gary. These are not facts. They are how you feel about methods. “To get the 72% they claim, they would have to not only get all the new 3000 to respond, but they’d have to get 67% of the 17,000 people who they tried to get before. I don’t think this is feasible since as the years go on, it gets harder to track people who you’ve lost. I do admit that it’s possible that they made a huge effort to get those numbers up to answer criticisms about the low self-reported turnout.” –> seriously?? First you complain about the response rate not ‘feeling’ significant. Then when the number goes up, you’re skeptical … Where’s the actual grounding on which your deconstruction of this ‘great deception’ is built?

    Your belief or lack thereof in survey design does not a lie make, nor does your issue with semantics. The whole point in choosing words that reflect the reality is to maintain truth, not distort it. Congratulations on your critical thinking, but you might want to apply the same yardstick when it comes to your own reasoning making unfounded, blanket statements, as you have done here.

Post a comment

About this Blog

By a somewhat frustrated 1991 alum

Region
Houston
Grade
High School
Subject
Math

Subscribe to this blog (feed)


Subscribe via RSS

”subscribe

Reluctant Disciplinarian on Amazon

Beyond Survival On Amazon

RSS Feed

Subscribe