Aug 24 2012

What do TFA, KIPP, and Lance Armstrong have in common?

Answer:  They have all been recently caught in huge lies.

Last week I wrote a blog post about a Reuters article in which a high level TFA staffer admitted that one of their statistics about the effectiveness of the first year corps members (that 41% of first years get 1.5 or more years of ‘gains’) “is not a particularly rigorous statistic”  TFA responded to the Reuters article on Alexander Russo’s blog.  Here is the response:

We were glad to see that the recent Reuters article and sidebar article on Teach For America highlighted the mission and impact of our organization over the past two decades. As the sidebar story rightly notes, Wendy Kopp always envisioned Teach For America as a “lifechanging” organization. We were especially pleased to see several alumni recognized for their impact including KIPP cofounders Mike Feinberg and Dave Levin, Louisiana schools chief John White, and Newark schools superintendent Cami Anderson. Their leadership is helping to open doors of opportunity for thousands of underserved students.

However, we were troubled that the article did not accurately characterize the research base around Teach For America. In describing the research, the reporter downplays the rigorousness of both internal and external reviews of Teach For America, and cites data from independent studies to give the impression that Teach For America is inconsistent at best.

The reporter uses an out-of-context quote from our former research director, Heather Harding, to inaccurately imply that internal reviews of Teach For America teachers are unreliable. While internal reviews are not as rigorous as external ones, they still have plenty of merit. In fact, Teach For America relies on a wide range of research—both internal and external—to help us constantly improve our methods of preparing teachers for the classroom.

While the article highlights the most recent report on teacher preparation programs by the Tennessee Higher Education Commission, it neglects to mention the most striking finding: the commission named Teach For America the top-performing program in the state. This echoes findings from similar studies done in North Carolina and Louisiana, both of which also named Teach For America as the most effective teacher preparation program in the state. A growing body of independent research has shown that Teach For America corps members are effective across school subjects and age groups.

The reporter also makes only minimal acknowledgement of the contributions of our alumni highlighted in the sidebar article. There are now over 28,000 Teach For America alumni nationwide; over two-thirds of them are still in education, despite the fact that only one in six corps members say they considered a career in education before joining Teach For America.

Drawing on their experience as corps members, our alumni are having a significant effect on the direction of public education in the United States. We wish that this article would have more accurately represented not only Teach For America’s research base, but the impact our corps members and alumni have had on American public schools over the past 20-plus years.

So TFA is upset that this reporter did not ‘spin’ the numbers they way TFA would have liked them to.  Yes, there have been some studies that have concluded that TFA first years add a bit more ‘value’ than other first years.  But value-added is not the same thing as learning, and even if it was, they get such a small amount more of it, and first year teachers are so bad anyway, that what difference does it really make.  The fact is that the other teachers will have longer careers so they will be able to easily make up for that one month of ‘gains’ that the TFAer had in the first year.

TFA got caught in a huge lie when they claimed that 41% of the first years get at least 1.5 years of gains, and once you are outed as a liar it does cast a shadow over your other claims, even if some of those other claims happened to be true.  This is why it is best not to lie.  Even in this statement, they cite a favorite statistic that 2/3 of alumni are still ‘in education.’  This is also a stretch.  Can any TFAer out there even claim that 2/3 of the people that they know from TFA are still ‘in education’?  This is a crazy statistic.  Does ‘in education’ mean to TFA what ‘all beef’ means to McDonalds?  I would love to see a list of the professions of the 22,000 alumni that make up those 2/3 and what it is they do that qualifies as ‘in education.’  If TFA would be more honest, I would stop writing about those lies.

KIPP also got outed recently.  Today a report came out by Ed Fuller that had been in the works for a couple of years.  He was hired to analyze charter schools in Texas and was given access to student level data so he could check, once and for all, whether or not charters truly serve the ‘same kids’ and get better results.  This is a very important report, and you can read it here.  Here are some of the conclusions, quoted from the report:

Contrary to the profile often portrayed in the media, by some policymakers, and by some charter school proponents (including some charter CEOs), the high-profile/high-enrollment CMOs in Texas enrolled groups of students that would arguably be easier to teach and would be more likely to exhibit high levels of achievement and greater growth on state achievement tests. Indeed, the above analyses showed that, relative to comparison schools, CMOs had:

  • Entering students with greater prior TAKS scores in both mathematics and reading;
  • Entering economically disadvantaged students with substantially greater prior TAKS scores in both mathematics and reading;
  • Lower percentages of incoming students designated as ELL;
  • Lower percentages of incoming students identified as special needs; and,
  • Only slightly greater percentages of incoming students identified as economically disadvantaged.

In other words, rather than serving more disadvantaged students, the findings of this study suggest that the high-profile/high-enrollment CMOs actually served a more advantaged clientele relative to comparison schools—especially as compared to schools in the same zip code as the CMO schools. This is often referred to as the “skimming” of more advantaged students from other schools. While CMOs may not intentionally skim, the skimming of students may simply be an artifact of the policies and procedures surrounding entrance into these CMOs.

Thus, the comparisons that have been made between these CMOs and traditional public schools—especially traditional public schools in the same neighborhoods as the CMO schools—have been “apples-to-oranges” comparisons rather than “apples-to-apples” comparisons. The public and policymakers need to look past the percentages of economically disadvantaged students and disabuse themselves of the notion that enrolling a high percentage of economically disadvantaged students is the same as having a large percentage of lower-performing students. In fact, despite a large majority of students entering the CMOs identified as economically disadvantaged, students at the selected CMOs tended to have average or above average TAKS achievement and certainly greater achievement levels than comparison schools. This was particularly true when comparing economically disadvantaged students in CMOs and traditional public schools—the economically disadvantaged students in CMOs had substantially greater academic performance than the economically disadvantaged students in the comparison traditional public schools.

In the words of Billy Joel, “Honesty is such a lonely word.  Everyone is so untrue”.  Schools like KIPP and also YES Prep which TFA likes to hold up as models of what amazing work their alumni are doing and how they are proving that teachers in regular public schools must be lazy bums, are really cheating.  They may not be doing this intentionally, but it is happening and it is causing other schools that are not cheating to be closed down to make room for more liars.  How is that fair?

And, in case you haven’t heard yet, Lance Armstrong who won 7 Tour de France titles has given up his fight against the charges that he used performance enhancing substances.  I don’t know a lot of details about the case, but it seems like this is another example of a ‘winner’ who is truly a ‘loser.’

If you lie enough, you’re eventually going to get caught.  Whether these recent articles about TFA and about Texas charter schools reduce the prestige of these places or not, I don’t know.  It might take more work to expose these lies.  The ball, though, seems to be really rolling in this direction.  I’m glad that there are so many people, now, who are onto the trail.  I think the reporter or researcher who truly ‘breaks’ the story about all the lying that is going on in the ed reform movement will win a Pulitzer Prize.

School is starting up for me so I’ll be slowing down on the blogging for a bit, but I encourage everyone who is working toward spreading ‘truth’ to keep up the good work.  The children in this country are counting on you.

9 Responses

  1. Maybe I am too much of a cynic, but I don’t think the lying matters much to many of the people making decisions. Anyone who looks at charter schools’ stats for more than a minute should have been able to predict the results of this report. I really think there is an underlying thought in America that less privileged families are less deserving and therefore it’s only right that more privileged kids get better schools, even among the “reformer” crowd.

    • Michael Fiorillo

      I think you are correct: this organization is based on a mountain of lies, but can breezily continue making them with little concern for being called out, since those lies serve the interests of very powerful people who’ve set their sights on the public schools.

      In an Age of Impunity, when, for example, the greatest financial swindle in history – which had the involvement of companies represented on the Board of TFA – results in not one management indictment, facts truly are stupid things, as Ronald Reagan so eloquently said.

  2. Two cents

    I might be able to shed some light on the “two-thirds of alumni stay in education” statistic.

    That figure is drawn from the annual alumni survey. TFA defines the “education” field pretty reasonably, in my opinion, including K-12 teachers, school-based professionals, administrators, college professors, and people in education-related advocacy. The majority are classroom teachers, with large contingents of professors (all higher ed counts as “education,” which is somewhat problematic) and administrators.

    As I see it, there are two main problems with the survey. The first is that selection bias is an issue, because while all alumni receive the survey, only about 60 percent of them return it. It seems safe to assume that people who are dissatisfied or disengaged are less likely to spend time filling out a lengthy survey, while those who feel positive about TFA are more likely to do so.

    In addition, the survey results are weighted toward younger alumni (those more likely to be in their third or fourth year of teaching) because of the program’s rapid expansion in recent years. It will be interesting to see if the survey results change in any way once the organization stops increasing the corps size each year and the age distribution of the alumni base becomes more regular.

    Everyone I’ve met at Teach For America has been very well intentioned, and I’ve never known any of them to lie in the course of doing their job. However, there’s a lot of institutional pressure to go along with the party line, and when a really striking figure that seems to support it comes up (like “two-thirds of alumni stay in education”), you’re not supposed to raise any difficult questions about validity.

    This is despite the fact that they employ really talented statisticians and people to create the surveys in the first place. The organization (and many of the people within it) genuinely want to be objective and transparent truth-seekers. But they really, REALLY want the truths they find to correspond to their beliefs. I would say it’s more a matter of cognitive dissonance more than anything else.

    (This is an important distinction, because if people you disagree with are conflicted, rather than malevolent, then you have a better chance of winning them to your side .)

    • Frederika

      It’s those people in education advocacy that are problematic in the statistic. Staying in education means STAYING IN EDUCATION–IN SCHOOLS, WITH STUDENTS. Doing ed policy for the mayor or a congress person is not being in education. The majority of TFA alum are not classroom teachers. Few corps members stay in teaching–make teaching their career. They may start as “teachers” but they don’t stay as teachers. WHY NOT?

      • Terry

        Because that is a common occupation and they are too important for such a lowly position….a big ego needs to be fed by other bloviators.

  3. Dufrense

    It’s funny that Russo takes issue with the Reuter’s writer citing data from an independent study: “How dare you use third-party research that’s not tainted by confirmation bias!”

  4. Since TFA’s statement does not cover how precisely Heather Harding’s comment was taken out of context, I’m reluctant to accept that it was.

    When I was in the Corps, the method used to collect data about significant gains was to ask CMs to self-report – no assessment necessary or even suggested. Given both the enormous pressure for huge gains and the lack of evidence required, every Corps member I knew reported incredible gains whether or not they felt that their class had made progress.

    TFA has to know these data are unreliable. That they disseminate and defend them is one of the things that disturbs me the most about the organization.

  5. JDM

    Once someone starts lying and is outed as a liar, how do you ever know when they’re NOT lying? At this point, every TFA representation falls into one of two categories: either it’s arrant bullshit and doesn’t matter OR it has to be checked.

  6. Andy

    A few thoughts on the KIPP data points Gary. I think you are right to highlight important studies that compare the characteristics of entering students between charters and traditional schools. As you mention, too often we take the “% free/reduced lunch” as sacrosanct and assume if this number matches then the comparison is appropriate. It is right to look at the starting achievement characteristics, special education population, etc. Additionally, I do hope future studies can find the nuance in the free reduced number-there is a huge difference in the profile of a student from household with less than $10,000 income versus $39,000-both of which I believe fall in the free/reduced category.
    I do want to pose some additional thoughts/questions:
    1) We should be cautious about generalizing one study about KIPP schools in Texas to all KIPP schools or charter schools. Some KIPP schools have massive waitlists, others do not and that makes a difference. So maybe we can say “KIPP schools in Texas are liars”
    2) Just because a a relationship is “statistically significant’ does not say anything about the size of the relationship. As the study’s author noted, the difference in starting scores for 5th graders at KIPP and comparison schools is relatively small and is only .051 standard deviations above the mean. My point here is that any studies’ conclusions must be taken with a grain of salt. He also didn’t report the “r-squared” statistic number which helps understand the size of the relationship.
    3) Is cheating the right word to use because some schools have students with higher test scores than others? I don’t think so. The more responsible comment is probably something like: “What the study shows is that we must be careful when drawing conclusions about the successes of KIPP because the profiles of their students is not a perfect match to comparison traditional schools” You are right that their success is overstated, but I don’t think the study implies that we should dismiss everything they do.

About this Blog

By a somewhat frustrated 1991 alum

Region
Houston
Grade
High School
Subject
Math

Subscribe to this blog (feed)


Subscribe via RSS

”subscribe

Reluctant Disciplinarian on Amazon

Beyond Survival On Amazon

RSS Feed

Subscribe