Newark, New Jersey, is an excellent example of a city where charter schools have flourished over the years.  A TFA alum is even the chancellor of schools in Newark.  KIPP schools are the gold standard of charter schools and have been in Newark for some time.  According to their website they have five schools there serving around 1,800 students.  Two schools are elementary, two are middle schools, and there is one high school.  One of the middle schools, the RISE academy, if often hailed as a true ‘miracle school’ that defies the odds with its amazing test scores.

One thing that KIPP does which I respect is publish an annual report summarizing the statistics of its over 125 schools.  Though they leave out certain information which I think is relevant, they leave a lot in which paints a more accurate picture of their successes, which in some cases are quite limited.

Looking at page 75 of the report, we see what sort of achievement in high school was accomplished by the students who graduated from the miracle two KIPP Newark middle schools.  Their SAT score was 1250, which is extremely low, only 416 per section.  When it comes to AP tests, only 31% took AP courses and only 2% passed at least one AP test.  With their numbers this means that just one student in the entire school passed an AP test.  If the amazing KIPP Newark middle schoolers are kids who peak in 8th grade, what good is that?

But this only tells a small part of the story.  I found two other sources for information about this school’s performance.  The first is on the KIPP Newark website where they report the unusual demographics of this school.  At the bottom of this page we learn that this school has nearly 60% girls.

There have been studies about how high attrition is for black boys at KIPP and this is further evidence about this.  Combine this with some facts from their New Jersey school report card where we see that in addition to this unequal balance of boys and girls, there is, for some tests, an incredible ‘gender gap.’  For example, in 8th grade language arts, 71% of boys scored proficient or better compared to 89% of girls.

Another thing I found on the state report card is that the Newark KIPP network does suffer from attrition.  Notice how the recent graduation class of 55 students was 71 students three years earlier.

Two years ago, when KIPP released their 2010 annual report, I wrote about how they admit their student retention rate was only 88%.  When you lose 12% of your students a year, that amounts to losing about 40% of the students who begin a KIPP middle school in 5th grade by the time they are supposed to complete 8th grade.  Two years later, that figure has not changed at all.

While I do appreciate that they are willing to admit this statistic, it is amazing to me that education writers don’t write about this more often.  I know that the attrition isn’t solely from students who have been ‘counseled out’ (unofficially expelled).  Sometimes families have to move for reasons out of their control, but I’d think that if KIPP were so great many families would find a way to have their child continue there.

KIPP has received a lot of money on the facade that they have the secret to getting amazing results from the ‘same kids’ with the ‘same resources.’  Their own reports and publicly available data from New Jersey clearly show that their success is extremely limited.  Yet, they continue to expand and to be used by politicians as evidence that ‘reformers’ know what they are talking about.  How long this will continue, I don’t know, but I have to believe that it won’t be for very long.

I’ve often heard teachers complain about the latest reform “fad.” It’s understandable insofar as veteran teachers have been around for many rounds of “reform,” only to see each and every one swept abashedly into a locked closet in the back of the class (right next to where I surreptitiously put those pre-tests I never got around to grading).

I think there’s a lot of truth to that, and I suspect that you agree. Where we’d probably part ways is the takeaway from this insight. You might say that this goes to show how temporal the current regime of reform is; I would say that the faddish nature of past reform shows that we need to stay the course.

You write that Michelle Rhee and Kaya Henderson’s reforms in Washington DC have not worked.  (Disclosure: I previously spent a couple months working as an intern at StudentsFirst. And I had one very brief, very pleasant conversation with Michelle Rhee. I have no affiliation with StudentsFirst now, though, and my views are my own.)

I think it’s misleading to say that based on a few years of inconclusive data we can determine that a reform has “failed.” There is mixed evidence regarding the results of the changes made in D.C., but as Matt DiCarlo pointed out in a takedown of some of Rhee’s overstatements, it’s inappropriate to draw broad conclusions, pro or con, based on just a few years of observational data.

The goal, by the way, of some of these changes – albeit not always articulated well by reformers – is to raise overall teaching quality by attracting some people into the profession who might not otherwise become teachers. It is, to use your baseball analogy, to try to make all hitters bat .300 (or more to my liking, have an on-base percentage of .400). Baseball of course, is a zero-sum game – a batter’s success is a pitcher’s failure – but in education it’s at least theoretically possible for all teachers to be excellent. That’s why many reformers believe in increasing teacher compensation in order to attract top talent, and this is what D.C. is trying to do. This sort of reform, though, will by its nature take time.

I know you believe that these new-fangled evaluation systems are doing far more harm than good, but I can’t accept that after a few districts have implemented them for a few years, they can be trashed as complete failures. If we’re not seeing improved results in another couple decades, then I will be the first one to say that these reforms need to go.

The completely understandable response is, well, what happens to the students who are (in your view and many others’) getting worse results because of such reforms? I have no glib answer to that. What I can say is that students in D.C. were struggling greatly before Rhee’s reforms. The spending per student sits at $30,000 and the results were among the worst in the country.

Gary, I know that you believe that schools can only do so much for their students, but certainly DC schools had and have a lot of room to improve. Was there not space to try something new, even if it didn’t align with many educators’ intuitions about what will work? Isn’t this especially true considering the shaky (though mixed) evidence for more traditional reforms, such as class-size reductions and Head Start?

What I feel disillusioned about is that many teachers that I worked with were so negative towards our pay-for-performance system that they sought to tear it down rather than build it up. You’re upset that StudentsFirst spends a great deal of money on elections, rather than trying to improve teacher preparation; well, I hope you’re equally upset that NEA and AFT have spent huge amounts of money backing candidates who oppose teacher evaluations. What if that money were spent on making teacher evaluation systems be as effective as possible?

Few alternatives have been suggested. Yet, I don’t think even you would agree with the notion that a decade ago teachers were being fairly and effectively evaluated. As you know, many “evaluations” consisted of an annual pro forma visit by the principal into the teacher’s classroom; meaningful feedback was not consistently given. I know you think the pendulum has swung too far, but perhaps you could acknowledge that reformers were responding to some bad circumstances.

Am I wrong? I know that the AFT has backed peer review programs, which I think could be great, but it also seems like a blip on the radar. Perhaps reformers have to account for the failure to work with unions on this – but isn’t always a two-way street?

Yours in dialogue,

Matt

PS I hope you saw Matt DiCarlo’s recent piece on VAM. I agree that it’s not junk science or a “sham.” I also think that some districts have a way to go in using it effectively – hence my comments above about teachers being partners in creating such evaluation systems.

My response:

Dear Matt,

“another couple of decades”?  No, I don’t think this would be a good idea.  I do appreciate your desire to apply the scientific method to education, however.  And yes it can be premature to label an experiment as a failure before it has been given time to run its course.  But if we’re going to act like scientists, we have to do it right.  For education, it is a bit like medicine.  There is a ‘disease’ — poor kids getting low test scores — which we want to ‘cure.’  The vital first step is to identify the cause of the disease.  If you misdiagnose this, any remedy based on fixing the wrong cause is doomed to fail.  Even if ‘bad teachers’ were a big cause for low student test scores, I’m not confident that corporate reforms strategies would work, but treating the wrong cause is definitely counterproductive.  In this case it is like trying to cure a strained knee with chemotherapy.

To treat the problem scientifically, the people controlling the experiment should have some idea about what ‘success’ should look like at various stages of it.  Yes, I suppose there could be experimental drugs where they seem to be not working at all and then at the last minute they kick in and cure the disease, but in this case we should be seeing some benefits, particularly in places that have embraced the reforms most like Washington D.C..  Their lack of progress so far suggests to me that I am correct that they are fixated on the wrong root cause.  The problem with waiting for a few decades, from my perspective, is that there is too great of a risk that these remedies come with severe side-effects.

This may seem non-scientific to you, but my very traumatic first year created in me a keen ability to tell when something in education just isn’t going to work.  My first year I was very naive and ideas about how to teach or about how to manage a classroom would sound so good to me until I tried them only to learn, the hard way, that they had serious loopholes.  So since that first year I’ve developed an innate sense for these sorts of things.  When I hear new ideas about how to teach or about how my school’s bell schedule is going to be altered or, at a macro level, how to ‘turnaround’ certain schools, I just ‘know,’ even sometimes when I haven’t been able to verbalize exactly why, it isn’t going to work.

In the case of the ‘bad teacher’ cause I also have some strong first hand evidence that this is not the major issue.  For one, I taught at three failing schools that all had what I’d call ‘above average’ teachers.  Also, two of the smartest friends and TFA alum that I know have been principals of un-miraculous schools.  I feel like if they weren’t able to turn around their schools (at least from a test score perspective) that it can’t be done without some kind of cheating.

You are correct that many schools and districts, pre-reform movement, were not fulfilling their potential.  Though I think there is a limit to what a school can accomplish, I do think it is worthwhile to try to achieve that limit and, yes, schools did have a lot of problems:  money being used unwisely, terrible organization where on the first day of school some classes have 50 students and others have 10 students, bad curricula, etc.  I’m not surprised that ‘reformers’ have been given an opportunity to show that they could do things better.  My hope is that once this wave of ‘reform’ is shaken off, a new movement led by people who actually know something about students, teachers, and schools take over and guide us forward.  I look forward to the day when I can stop playing defense and participate in that endeavor.

I mentioned my ‘sixth sense’ which I’m sure could invite some ridicule, but here is what I believe will happen as a result of this misguided ‘reform’ movement.  Within five or ten years there will be a massive teacher shortage.  Already, I’ve read that in California the number of new teacher applicants is way down this year.  The promise of big bonuses to young superstars will just not be enough to get people to replace all the teachers retiring or otherwise fleeing the profession.  Maybe TFA and other alternative certification will be expanded to fill the void and maybe we may even be able to get test scores to the same, or even marginally higher, level with enough focus on that goal.  But the other metrics, the more important ones that are so tough to measure, will suffer.  Maybe we will drop from the top of the Nobel prize winners category.  Or, like we see in New Orleans, we will have more crime.  We may win the ‘test score’ battle, yet lose the war.  I know that this might sound extremely speculative, but that is where I see things going, at this rate.

As far as the unions spending money to back candidates to go against the ‘reformers’ I think that is exactly what they should be doing right now.  When you’re under attack, you have to defend yourself as your first priority.

You know, the basic premises of ‘reformers’ are not crazy to me.  Certainly we want schools and teachers to be as effective and efficient as possible.  And yes, a good teacher gets kids to learn.  It may be tough sometimes to measure that learning, but still, a teacher should be able to present some evidence that students have progressed.  But these modern measures of progress for students and schools are being misused.  Maybe they are not complete ‘junk’ but they certainly have a large margin of error so using them as a way to rank teachers is not very fair.  Matt Di Carlo does seem determined to keep the possibility that one day these measures will be improved and used appropriately.

But the big problem is that the ‘reformers’ are not as wise as Di Carlo.  The whole ‘reform’ movement is led by people who I really don’t admire.  I haven’t figured out if they are well-meaning but just not wise about things or if they are devious and excellent liars.  I can see the case for either argument.  As a litmus test, compare some of the writers and bloggers on both sides.  A good starting place that will hit home with you is to look at the pieces that you wrote and appeared in The Answer Sheet and in Dropout Nation.  I can’t think of another person besides you who has had pieces in such different forums.  In your Dropout Nation ‘Testing is good for teachers and children‘, editor RiShawn Biddle is so insecure that he constantly interrupts your essay (I counted four such intrusions).  Compare to your ‘It’s time for TFA to fold‘ where Valerie Strauss, after the introduction, allows you to say what you want without interruption.

Anyway, I hope I have not seemed to ‘dodge’ any of your questions.  I’m wondering what you think about the mythical highly-effective teachers and high-performing charter schools.  Do you think they prove that ‘poverty is not destiny’?  Also, what do you think that TFA makes the right decision when they try to convince the new CMs that they are very capable of achieving ‘transformational’ teaching where they change the life trajectory of their students?  Realistically, I don’t think that my impact on students, even in my ‘teacher of the year’ year, was so great that it changed any life trajectories.  Do you think that you did?  Do you think that the alum who trained you did?  If this is unrealistic, is it still OK to train people with that mindset since it will keep them optimistic, like telling someone that a diet and exercise program is likely to work, thus motivating them to stick to it?  Or is is more like telling someone that they can climb mount Everest if they just have high expectations for themselves and little practical training?

Look forward to continuing the dialogue.

Gary

The biggest distortion of their numbers is their claim that they have 1.3 million members.  Many of these ‘members’ are people who are completely opposed to StudentsFirst’s agenda, but have accidentally signed up by clicking on a button on a form that said something like “Click here if you think great teachers should be rewarded!”  When they held a rally in Connecticut earlier this year, only about ten people showed up despite the thousands of ‘members’ they claimed they had there.

The most recent example of StudentsFirst playing with numbers to advance their own agenda was something bizarre I read on their blog by teacher Bhavini Bhakta called ‘A Teacher’s Two Cents’ in which she begins “I’ve been a teacher for 9 years and I had to go through being laid off for 8 of those years.”  This seems a little too extreme to be true.  I’m not sure what her situation was and why she was, supposedly, laid off so often, but I do know that there must be much more to the story.

[READ] 9-yr, 8-time LIFO’d CA teacher @missbhakta shares her story: “A Teacher’s Two Cents” studentsfirst.org/blog/entry/a-t… #edchat #edreform

— StudentsFirst (@StudentsFirst) May 15, 2013

So I did a few seconds of research and found that five months ago Bhavini wrote a similar anti-LIFO article in The Los Angeles Times entitled ‘California’s pink-slip shuffle’.  In this one it says at the end, “Bhavini Bhakta lost teaching positions in four schools over eight years because she lacked seniority.”

So which one is it?  Has she been LIFOed eight times or four times?  When StudentsFirst lies so much, they really can’t be trusted.

Incidentally, in case you’re wondering, I fully support LIFO layoffs.  I believe that layoffs are rarely necessary.  There is so much turnover in teaching anyway that just having a hiring freeze would make most layoffs unnecessary.  I would, however, support something where an experienced teacher who is in her first year teaching in a new district would somehow get credit for her previous years to avoid LIFO layoffs.  Most teachers laid off through LIFO, I think, are true first year teachers and not eight year veterans.

So of course it is pretty ironic that out of all the perspective teachers in the country who could have been chosen from various education programs, they chose to have TFAers who really have not ‘decided to become a teacher’ but who have decided to teach for two years and, most likely, go onto something else.  Statistically speaking, at least one of those ten teachers will not make it through the first year.  Some of these ten (I won’t say which) will have a lot of work to overcome how young and soft-spoken they seem.

These were the top ten reasons:

10.  I hope to live up to the teachers who inspired me, like Ms. What’s-her-name

9.  It’s no fun saying the pledge of allegiance every day by myself.

8.  Honestly, I didn’t pay much attention the first time through school.

7.  Kids need to know the Moon landing was faked.

6.  If I could make a difference in just one student’s life — well, that wouldn’t be a very good average.

5.  The glamour.

4.  You work long hours, but at least the pay is bad.

3.  Hoping to teach in an all song-and-dance high school, like on ‘Glee’.

2.  In the summer, I can watch all you losers go to the office.

1.  I want to help kids talk good.
I don’t want to seem like a guy who has lost his sense of humor about things.  I started watching Letterman back in the late 1980s and though I haven’t really watched him in about ten years, I think he is a great talent.  Reasons 8 and 1 do have a bit of teacher bashing in them, the myth that most teachers come from the bottom of their graduating class (those who can’t do, teach).  Reason number 5 (the glamour) is also somewhat ironic as people do join TFA because of the glamour associated with it and the opportunities it opens up afterwards.

Anyway, I’m not overly offended by this, but I thought it would still be worth writing about.

The premise of Harlem Children’s Zone is a good one.  It serves to provide school and complete wrap-around services (health, mental health, nutrition, etc.) to kids from Harlem beginning at birth and supporting them all the way through college.  The program, you might imagine, is very costly — hundreds of millions of dollars — and Canada, himself, pulls in about $400,000 a year.

During the question and answer section of the talk, Canada called on musician and TFA board member John Legend who asked the very relevant question:

John Legend: So what is the high school dropout rate at Harlem Children’s Zone?

Geoffrey Canada: Well, you know, John,100 percent of our kids graduated high school last year in my school.  A hundred percent of them went to college.This year’s seniors will have 100 percent graduating high school.Last I heard we had 93 percent accepted to college.We’d better get that other seven percent.So that’s just how this goes. (Applause)

Now I know that generally when a ‘reformer’ brags about a 100% graduation rate, he means that 100% of the students who made it to senior year also graduated while neglecting to mention what percent of the cohort had not made it to senior year.  This is the statistic that is quite easy to find by looking at the New York State report cards.  Here are seven consecutive years of enrollment statistics I got from there:

So the 62 graduates in 2012 had been the 97 6th graders in 2006.  This does not represent a 0% dropout rate, as Canada implied to John Legend, but a 36% dropout rate.

But looking at these numbers also reveals two large scandals of which I had only been aware of one before looking into this:  First, notice how there are 68 8th graders in 2007 (down from 100 6th graders in 2005) but no 9th graders in 2008.  This is because Canada ‘fired’ the entire group of what would have been their first 9th graders and first graduating class.  This story is not a big secret anymore, though Canada doesn’t seem to have lost much of his credibility for it.  For the second scandal, notice that in the 2007-2008 school year there were 88 6th graders but in 2008-2009 there are no seventh graders.  This is because they also rid themselves of an entire class of 6th graders that year.  The next year they decided to only take new 5th graders which is why we see they had 12th graders and 11th graders in 2011-2012 but no 9th or 10th graders.

Regardless of how poor the performance of those two lost classes were, Harlem Children’s Zone could have easily kept those students and have been able to answer critics of their test scores by saying that those two classes of students were students who had started as 6th graders in the school and that HCZ should only be judged by the performance of the classes of 2016 and beyond since those students will have completed their entire schooling through the program.  I guess that this could have been construed as an ‘excuse’ which would have gone against the ‘no excuses’ ideology.

As far a performance goes, the HCZ Promise Academy high school may have decent state test scores, but when it comes to national tests they only had on the ACT a 20 in Math, a 15 in English, a 15 in Reading, and a 17 in Science.  Aside from Math, this puts them in under the 20th percentile for those tests.

I’ve spoken to two teachers who have worked at Promise Academy, one is a former elementary school teacher there and another is a current middle school teacher.  The middle school teacher said that the school had recently had almost a complete turnover in staff.  When a place is supposedly so great, teachers would want to spend some time there, in my experience.  The elementary teacher described a horrible abusive working condition where uncertified co-teachers often taught test prep and where there was little support from the administration for teachers and nearly zero interaction with Canada, despite his nearly half a million dollar a year salary.

To see Canada get publicly humiliated in a debate with Diane Ravitch from September 2011, watch the live version of Meet The Press from Education Nation here.

As someone who is, I suppose, a big “friendly critic” (an expression TFA coined as the need to describe the growing number of frustrated alumni) of TFA, I think the biggest problem with TFA is all the lying.  Though the individual people I’ve known on staff aren’t huge liars, themselves, the sum of all the lies add up to an organization whose lying is pathological.  Really, they’ve elevated the art of lying to new heights, much the way Mozart elevated the concerto.  Even people like Bernie Madoff who thought they were great liars can’t help but marvel at TFAs techniques.

The lies began innocuous enough.  They were just part of their PR, part of their advertising and fund raising efforts.  Other lies they didn’t even realize were lies until they were too embroiled in them and still other lies they still don’t seem to realize are untrue.  If I could change one thing about the organization, this would be it.

Of all the lies, I can easily identify the biggest three.  Over the years I’ve written multiple posts about each of thes lies, but thought it would be useful for people just learning about, or just joining, TFA to get a summary of them in one place:

Biggest TFA Lie #1:  The training is adequate.

This is a lie that I have been trying to expose for the past seventeen years.  Back then I felt that by offering too rosy a picture of the first year, new TFAers were overconfident and not able to take their preparation as seriously as they would if they knew how difficult it is to have a good start to the first year and how nearly impossible it is to recover from a bad start.

For many years this was my big issue with TFA.  By sugar coating the first year they mislead the corps members into a dangerous overconfidence.  From 1995 to 2006 I volunteered at the institute presenting a workshop (which you can see here) about the realities of many first years of teaching (using my own as a case study).  In the latest incarnation of the TFA pre-institute reading they have a whopping 5 minutes dedicated to this in a reading called “250 minutes.”

In it the teacher describes a very bad day, but later concludes that:

And also:

So the only five minutes of thinking about how grueling the first year can be is from a teacher who had at most a bad month.  For a more realistic view of the first year, many of the bloggers on this site do a nice job at this.

In recent years a new problem emerged in the training model.  As the size of the corps grew exponentially (the first few corps were around 500 people, then it was around 1000 for a while, but now it is 6000 a year), TFA did not figure out a way to give all those trainees enough summer school students to practice teaching.  Now we routinely see people training for less than 12 hours in front of a class for the entire summer with less than 12 students in each class.

In the pre-institute reading that new CMs got this year, they explain why the readings are focused on big ideas surrounding education rather than much about how to teach:

If any trainees actually empower their “summer school students to make incredible academic strides,” I’m sure that it will have a lot more to do with the tiny class sizes of often single digit numbers of students than any “nuts and bolts” (maybe thumb tacks) that the teachers picked up at institute.

Though the corps is twelve times bigger than it used to be twenty years ago, the amount of money TFA has is around 200 times more.  With $200 million a year, they need to find a way to get people more classroom experience.  I’d also like to see the placement procedure fixed so that the corps members can all practice with the age group that they are going to teach.  It seems to me that if principals are so enamored by TFA, as TFA claims, there could be a way for new CMs to replace all the CMs who are leaving their schools.  This way they would know the placement way ahead of time and train accordingly.

TFA seems to be in denial about their training being ‘good enough,’ based on how slow they have been to improve it.  Maybe they think that since the standardized test scores from teachers who trained with TFA (at least the 91% of TFA teachers who make it through the first year and get a chance to administer those tests) are not all that different than the test scores of ‘traditionally certified’ (TFA speak for ‘dumb’) teachers, that this is some kind of proof of the validity of their model.  But this seems to go against one of the goals of TFA to have teachers who are ‘transformational.’  If TFA teachers are about the same (some studies have them, at least in math, a little bit better at raising standardized test scores) as traditionally certified teachers, that must either mean, at least by TFA’s logic, that those other teachers are also transformational or that neither truly are.

Biggest TFA Lie #2:  The magical power of high expectations.

If I were to summarize TFA’s philosophy to teacher training in a few words I’d say “Students always rise to meet the expectations of their teacher.  A large part of the reason that poor kids don’t have the same academic achievement as wealthy kids is that the teachers of the poor kids have low expectations.”  How great it would be if this were true.  While I do believe that setting expectations extraordinarily low isn’t a good idea either, expectations that are too high are likely to backfire on the naive teacher.

I haven’t figured out it TFA is purposely lying to new CMs about this or whether TFA, itself, actually believes this.  The motivation behind lying would be, I suppose, that it would ‘trick’ new corps members into getting the confidence they need to take on this responsibility.  Having high expectations, after all, is something that new teachers can choose to have, even if they don’t have the skills to get students to those expectations.  The scary thing to me is that I’ve talked with different TFA staff members, and my sense is that this is not supposed to be a trick to psyche out the corps members.  They seem to really believe that low expectations is a large culprit for the problems in American education.  A good demonstration of how TFA leads new corps members to embrace ‘high expectations’ as the primary weapon for fighting educational inequity is in this corps member produced video last summer.

Honestly, if I were to make my list of reasons why poor students struggle to ace standardized tests, low expectations from teachers would not crack my top 10.  Yet, the first 35 pages of TFA’s guidebook ‘Teaching as Leadership” is all about the power of high expectations until they reluctantly admit in one sentence on page 36 (and then never again) “Yet setting a goal that is impossible for students to reach even with extraordinarily hard work might further undermine students’ shaky confidence, cementing their impression that effort does not lead to achievement and that they are ‘not smart’ enough to achieve in school.”

Biggest TFA Lie #3:  The existence of miracle TFA teachers/schools/districts.

As evidence that the training model is good (lie #1) we often get to meet ‘miracle’ TFA teachers who wield their all powerful high expectations (lie #2) to lead their classes to amazing ‘gains’ of up to two years.  As an example of this, see the latest pre-institute reading where the accomplishments of Jeremy Beard (husband of new TFA co-CEO Elisa Villanueva-Beard) are described:

This is about all the proof that a new corps member needs.  100% of the first three graduating classes of his school went to college?  Wow.  But what they neglect to say, and what I’ve learned by investigating hundreds of these miracle claims, is that there is always more to the story.  When people hear this 100% statistic, they assume that this means that 100% of the students who entered the school in 9th grade eventually graduated and went to college.  But all that happened is that 100% of the students who actually made it to graduation got accepted into college.  More relevant is what percent of 6th graders eventually graduated and got into college.  Fortunately, this is very easy to find out.  In Texas they have an excellent public data system called AEIS which you can access here.  Within a few minutes of searching for this IDEA college prep in Hidalgo county I learned that the first graduating class of this school was only 27 students in 2007.  Six years earlier in 2000-2001, there were 69 sixth graders.  What happened to the other students, I’m not sure.  For their second graduating class in 2008, 32 student of whom 100% also got into college, well there were 85 sixth graders six years earlier.  I encourage readers to double check these numbers for themselves.  And my goal is not to take down Jeremy Beard who seems to be a nice enough guy.  The point is that TFA just can’t resist including some kind of bizarre miracle story in their literature.  In this case they could have chosen any success story from their entire 20 year history and the one they chose was so easily revealed as an inflated claim of success.  This is not to say that they didn’t do good work over there, just that it wasn’t enough to prove the unlimited power of enthusiasm, hard work, and high expectations.

This type of attrition is true of all the ‘high performing’ ‘no excuses’ charters that you hear about, and which I have written about for a few years beginning with my very first investigative post around two years ago.  College acceptance rate for graduating seniors is a meaningless statistic that is often thrown around recklessly.

This type of lie extends to TFA alumni who have gone on to lead charter networks (like KIPP), cities (like D.C., Newark, and New Orleans) and even states (Tennessee and Louisiana).  Under scrutiny I’ve found that their results are definitely exaggerated.  Here is something I wrote about New Orleans and here is a popular blog about what is (and isn’t) going on in Washington D.C.

I should make it clear to newcomers to this blog that I hope people don’t take my realism and exposing of blatant lies and half-truths as my believing that teachers can’t make ‘a difference.’  I’ve been teaching for fifteen years so I certainly try every day to do my best and some days I’m better than other days.  Knowing that teachers are not superheros does not mean that I don’t think that they are still heros, but maybe more like an action hero like Indiana Jones than like Superman.  We find a way to use the limited power that we have to make as big of a difference as we can.

When I taught in Houston for four years, I put in a lot of hours and really got to help my students and to know them very well.  Twenty years later, I am still in contact with many of my old students who have friended me on Facebook.  I don’t know that I was ‘transformative’ in the sense that TFA claims that many alumni were.  I don’t know that I was ‘the reason’ that some of my students that did eventually graduate college did so.  I do know that I got many of my students to like and appreciate math, which was really my goal.  Likewise, teachers all over make small differences each day.  It is tough to know what our individual impact is.  I feel confident that I enhanced the lives of many of my students.  Others hated my guts, but maybe those students were inspired by a different teacher.

Conclusions

The new co-CEOs have been going around the country on a ‘listening tour’ since taking over recently.  I continue to wait for my invitation to a private meeting, though I’m not expecting one.  If TFA wants to listen, then they can read this, I guess.

TFA does not have to lie so much.  I know that they mainly do it because if they stop lying they might not be able to get as much public and private money.  But there is a lot of truth that they can empathize and be proud of.  Like the fact that even if few of the TFA teachers are changing life trajectories, there are some excellent teachers they have trained who are making small differences each day.  Some of these teachers (though not as many as they say) teach well past the two year commitment.  There are also some great school leaders — not the ones that we hear about from them — but honest ones who are also making small but genuine differences each day.

Lies will not help America’s children.  Lies might make some charter operators rich which, I suppose, is good for the charter operators.  But these lies are causing, around the country, schools to be shut down, teachers to be fired, and students to be scattered around looking for a new school after knowing that they got their old school shut down — and all because these schools, teachers, and students were not able to match the things that TFA has been lying about.

TFA, to quote Jack Nicholson in ‘A Few Good Men’, can’t handle the truth.  They just can’t stop lying.  And as fast as they lie, I will continue to reveal those lies.  (It is tough keeping up, sometimes!)

Runners Up

Though these are the biggest three TFA lies, there are so many more that deserve honorable mentions.  I’ve already blogged about some of these.  Here is a partial list:

2/3 of alumni are still in education, half of them as teachers

90% of principals are very satisfied with their TFA teachers

50,000 people genuinely apply to TFA (You count as an ‘applicant’ if you complete stage one which only takes about a minute)

Charter schools don’t have easier to teach kids and more involved parents and don’t expel kids who bring down their test scores.  (Finally admitted here, but myth still persists)

The average career length of a TFA teacher is 8 years

39% of first year TFAers get a year and a half of learning in one year from their students (finally admitted that this was a lie here)

Independent studies show that TFAers get between a half year and a year of gains above non-TFA teachers

Feel free to add your own …

Matt wrote a second letter and sent it to me over a week ago.  What follows is his letter and my response:

Gary,

Thanks for your response. I hope I didn’t put you on the defensive too much. In fact, the reason I wrote to you is because of how much I respect your writing, as well as the fact that I respect that you’ve chosen a career as a teacher. I did it for just two years – I fully realize the challenges of the job, and some of the insights teachers understand about education policy that non-educators don’t.

That being said, I think you miss something fundamental in your response: the goal of teacher evaluation systems is not to make teachers “try harder,” a common straw-man argument. Evaluations systems are designed to reward and retain effective teachers, and support and dismiss ineffective teachers. That’s the theory; practice of it is much more difficult, of course.

The recent Times article, I think, should be a wake-up call to reformers that they must pair teacher evaluations with efforts to sustainably improve the teacher talent pool.

It’s not particularly helpful to think about getting rid of ‘bad’ teachers in the abstract. At my school, at least, there were perhaps one or two teachers who I thought were probably awful and should not be in a classroom with children. Likely, you would agree that they should be fired. And likely we’d both agree that firing those one or two teachers would not dramatically improve education quality. (Though I do think the gains would be meaningful and important, and while reformers surely overstate this value at times, traditionalists also understate how difficult it is to dismiss a bad teacher.)

A much larger group of teachers at my school, perhaps a quarter or a third, were truly fantastic. They were exactly what many imagine of a great teacher: dedicated, hard-working, inspiring, and life-changing for some students. These were teachers who I would love to have teaching my own (hypothetical) child.

The final group of teachers – which I counted myself as a part of – could neither be classified as ‘good’ or ‘bad.’ Almost all of us were hard-working, and cared about children, but weren’t so good that I think most parents would actively want to send their kids into our classrooms. (A note: Obviously my assessment of the proportion of awful, ‘good enough,’ and great teachers is purely anecdotal. I’d be curious your thoughts on this, Gary.)

This is why I say that we can’t think of such teachers as ‘bad’ in a metaphysical sense. We need some point of comparison, some better option. That, I think, explains why so few teachers are being rated as ineffective – because principals may realize that they can’t get a better teacher, so they go ahead and rank a good-enough teacher as effective.

(Another potential explanation – and one that I saw play out at my school – is an observer effect: teachers and students changed how they acted when a principal was in the classroom, sometimes leaving the evaluator with a mistaken impression of how effective the teacher was.)

I’m disappointed that reformers are not doing a better job focusing on increasing the number of teachers who are great rather than just good enough. Many really want to address this problem, but I don’t think there has been enough energy around creating a sustainable pipeline of excellent teachers. This is understandable, insofar, as this goal is difficult, and might only lead to results far down the line. It’s easier to focus on the small number of truly awful teachers than to address the larger, more important goal of supporting and attracting exceptional career teachers.

I guess my question for you, Gary, is whether you think it’s possible to recruit a teaching force that includes a larger number of great teachers? Do you believe, as I do, that great teachers can ameliorate poverty and change students’ lives? Do you accept my distinction between excellent teachers and ‘good enough’ ones?

Before I end, I want to briefly address a few other points.

o   No, the Chetty study has not been ‘debunked.’ Reasonable people can of course disagree about what the policy implications of the study are and can raise some legitimate methodological question (though of course an ideal methodology is likely impossible in such context). Consider Bruce Baker’s reaction and Matt DiCarlo’s reaction – both raise some interesting questions but neither suggests that the study is patently invalid.

o   I do not think a 1% increase in salary for a given year is small at all. Let’s say a teacher has 20 students in a class and those students, over a 40 year working career would average a salary of $30,000/year. If that teacher increases the average salary by 1% to an average of $30,300, that means the teacher has added a monetary value of $240,000 for each year teaching.

o   There’s now very strong psychological evidence that testing doesn’t just assess learning – it promotes learning. Though reasonable people can disagree about whether this logic applies to standardized tests, this phenomenon is certainly something that those who oppose high-stake test should engage with a bit more.

o   I believe that you’re making a false dichotomy in suggesting that a teacher can do well on standardized tests by avoiding critical thinking exercises. Indeed, I think critical thinking exercises will likely improve high-stakes test results

o   I absolutely agree with your point about ‘bubble’ kids. My school was obsessed with proficiency rates rather than overall averages, and I always argued against this view. However, I don’t believe this serves as an indictment of high-stakes tests; instead, it’s an important implementation issue that should be considered and can be overcome.

o   I think I have to point out that one of my chief complaints was how rarely research is cited in opposition to standardized testing…and then you didn’t cite a shred of research for any of your opinions regarding testing. You say, ‘In my research I’ve found that often there is not much of a difference between the two schools.’ I know you have a lot of experience in schools, but I hope you can understand why I don’t simply trust your intuitions and experience.

o   I myself do not see school closing as some panacea, and I realize that many students will just end up going to an equally bad, but farther away school. I do hope and believe that some may end up in a better school, and I don’t think that, here in Chicago, the city can sustain paying for a huge number of schools that under-enrolled. (I do realize that closing schools will not save money in the short-term, but will in the long term – to its credit CPS has been upfront about this. Another interesting note is that CPS says that it is not using test data to determine school closing, and is only looking at utilization.)

Alright. I think that’s more than enough for one letter. I’m definitely enjoying having a dialogue rather than a monologue. Thanks, and I look forward to reading your thoughts.

~Matt

Dear Matt,

No, you don’t ever have to worry about putting me on the defensive.  Some of my thoughts are tough for me to explain, sometimes, so I might tend to over-explain so they can’t be misinterpreted, but other than that I’m completely comfortable answering any questions, no matter how tough.

I agree with your percentages about how many teachers are average, below average, and above average.  You seem to say that about a third are truly excellent, two thirds are just ‘average,’ and there are a small number, perhaps two or three percent, who are ‘bad.’  But this does depend on how exactly these categories are defined.  I think we’d agree on what constitutes one of those ‘bad’ teachers (and also agree that there are so few of those that a policy focused on identifying and terminating them isn’t going to ‘fix’ education), I think that the ‘average’ teacher is doing an admirable job and would have no problem having my two actual kids being taught by one of those.  Sure, I’d like them to have a few that I would consider ‘great’ from time to time, but I’m fine with the fact that most of my children’s teachers will fall into that ‘average’ category.

Like in Baseball where a ‘great’ hitter bats 300 and an ‘average’ hitter bats about 260, I think that great teachers are not as different from average ones as the reformer crowd claims.  When discussion about the importance of class size comes up, reformers often talk about how they’d rather have a great teacher with 40 kids in a class than an average teacher with 20.  Knowing what I do about how difficult teaching is, I’d put my kids in the small class with the average teacher over the ‘great’ teacher with the 40.

This is not to say that I don’t try to be a ‘great’ teacher each day that I go to work.  I’ve won various awards and written various books and articles about teaching, yet too often I’m up in front of my class, generally when I’m trying out a new lesson idea for the first time, and humming to myself to the tune of “We Are The Champions” something like “I am a failure, my friends, and I’ll keep on sucking till the end …”  My hope is that I have more ‘great’ days than ‘bad’ days that on average I’m considered ‘good.’

When I was younger I suppose I would have agreed with your belief that “great teachers can ameliorate poverty and change students’ lives.”  I don’t know that ‘great’ teachers (at least the mythical ones we hear about from TFA) are that much better at ameliorating poverty and changing students’ lives than ‘average’ teachers.  Maybe the difference is that I think ‘average’ teachers are better than you do, or that I don’t think that ‘great’ teachers are as good as you do.  Part of my belief has been shaped by my experience during my 2nd, 3rd, and 4th years of teaching when I was in TFA and at E. L. Furr High School in Houston.  This was the school that was recently on the front page of the New York Times in an article about how the school has finally curbed some of their gang related violence problems.

During the three years I worked there, I found the staff to be very impressive.  Yes, there were a few clunkers.  One was a man who was a year away from retirement and was, I felt, going senile.  Another was a woman who was an extremely hard working science teacher, but who had never really learned classroom management so all her hard work was wasted as students did not take her seriously.  But there were a lot of excellent teachers including the best English teacher I’ve ever seen.  I taught many eleventh and twelfth graders there and these were the top students in the school as many students never made it out of ninth grade.  Many of these upper classmen were, to use a TFA term, on or near a ‘trajectory’ to college.  If great teaching is supposed to help nudge them onto that college track, I was certainly in an excellent position to be someone who could have done that last push.  And despite my efforts and those of my co-workers I’ve come to learn, by keeping in touch with, or by recently reaching out to, my best students that few of them truly ‘overcame poverty’ or graduated college.  I don’t think that this means that we were not good teachers, or even great teachers.

And just because I don’t think that teachers have the power to do all that reformers think they do, this does not mean that I don’t think that teaching is a noble profession or that it is a very important one.  I guess a good analogy would be that I see teaching math not unlike teaching someone how to play a musical instrument.  The music teacher is not a failure if the student never becomes a professional musician.  Hopefully the music teacher will encourage the student to enjoy music and to want to practice and get better.  Also there’s the very important relationship that the teacher has with the students in which other life lessons can be taught, aside from music.  I was fortunate to have such a music teacher who I took private trumpet lessons with from 4th to 12th grade.

But if you want me to say it more directly, yes, ‘great’ teachers are better than ‘average’ ones who are, in turn, better than ‘bad’ ones.  Everyone knows that.

The issue, though, is how this should drive policy in such a way that we maximize whatever the purpose of schooling is?  Of course this ‘purpose’ would have to be defined first and though my definition might be a lot more holistic than yours, I would imagine that you are opposed to one that relies too much on standardized test score gains.  Then again, everyone nowadays claims to agree with this which is why they always stress ‘multiple measures’ in teacher evaluation.  But when I hear numbers being thrown around like 50% for value added or even 30% (no reformer I’ve read about dares to suggest a number less than 30%) I get very uncomfortable.

I do not think that putting this much weight on very inaccurate metrics will make schools better.  My belief is that if this pattern continues, it will make education actually worse, which is why I spend so much time reading and writing about this.  It hasn’t worked in D.C. because it is based on a false premise.  It seems like reformers are banking on the idea mentioned in Waiting For Superman that if we could just fire the bottom 5% of teachers each year, achievement would soar.  The only way that this would work is if there were a significant number of other teachers who need to fear for their jobs in order to get motivated to do their best teaching.  I just don’t think that this is the case.  Treating the wrong disease can be very dangerous.  It would be like a doctor prescribing chemotherapy for the flu.  It is misguided, painful, and will likely make the patient sicker.

I very much agree with your point that more effort should be dedicated to making the vast number of ‘average’ and even ‘great’ teachers even better.  This is where organizations like StudentsFirst have completely missed an opportunity to use all that money to truly improve education.  Teachers can definitely benefit from having more resources, professional development, truly usable lesson plans and activities online.  I personally just attended the NCTM (National Council of Teachers of Mathematics) conference where I attended workshops that I hope will make me a better teacher.  There were 20,000 math teachers there, which is a small percent of the math teachers in the country.  Most people paid over $1,000 to go to the conference, but maybe if there were more much cheaper opportunities to get good professional development, more people would participate.  Teachers want to improve.

I do not see ‘teacher evaluation’ reform, in the direction that it is currently going, as a way, as you describe, of helping teachers improve.  In New York teachers don’t even get to see the tests so how is that supposed to help them improve.  I personally have learned very little about how I can improve by analyzing the results of my own students on standardized tests.  Generally what I learn is that when someone else writes a very bad question about some topics, then even students who understand that topic quite well could get the question wrong.  These tests are just not good enough and to make them good enough, I think, would not be an efficient use of scarce resources.  As newspapers gush about the common core standards and assessments, all I see is hundreds of millions of dollars going to Dell and Apple to get the schools tech up to date to administer these tests.  As you wrote about the lavish cost of TFA and how this is not a good use of funds, I feel that way about the trend to keep teachers honest by making them ‘accountable’ for their standardized test ‘gains.’

Yes, teachers can learn to teach better.  I certainly don’t think all teachers (myself included) are perfect.  But the witch hunt for teachers ‘bad’ teachers is, in the long run, going to dissuade people from becoming teachers.  This will, in time, lower the quality of education in this country.  That’s what I’m afraid of.

Gary

Over the years I’ve worked, from time to time, as a teacher trainer for both TFA and for The New York City Teaching Fellows.  I think my favorite part of those jobs was watching the trainees teach and then providing detailed feedback of the strengths and weaknesses of their practice.  I spent the entire day at KIPP and popped in and out of classes at will, getting to see about twenty different teachers in my visit.

The KIPP teachers only teach four classes a day with class sizes between ten and twenty students.  This is a very light schedule and it certainly costs a lot of  money to have such a low student teacher ratio.  I don’t know if this is common for KIPP schools, but this was a pretty nice perk, I thought.  I also don’t know what their salaries are, but I’ve heard stories about KIPP teachers making extra money.

One thing that I see in schools a lot is the most experienced teachers getting to teach the high level class while the newbies have to teach the ninth grade remedial classes.  I suppose that this is some kind of reward for longevity, but it really is unfair.  If it really is about ‘the kids’ the new teachers should teach some of those easier classes.  I’m sorry to report that at KIPP they seemed to have the same sort of totem pole going on.  In the ninth grade wing is where I saw the most first year TFAers.

I saw a first year TFA guy who was trying very hard to teach 9th grade math, but his lesson, like most of the math lessons I saw there, was completely uninspiring.  Really just a mechanical activity which the students were most certainly not getting.  A few doors down I watched the beginning of a lesson on world religions where the new TFA teacher was struggling to get the YouTube video he was planning to show from ‘Little Buddha.’  The clip was not running properly and students were completely disengaged.

I think the worst teaching I saw was a TFA-type English teacher who had a class with about 9 tenth graders.  This teacher had no control whatsoever.  Students were ignoring his instructions and he finally got his question out and one student answered it while the others chattered away.  He had a nervous smile all the time and revealed his inability to carefully choose his words when one student tattled on a girl in the class and another student started telling the girl to be quiet until the teacher said “Don’t worry.  I’ve got this.  I’m on her,” which got a big laugh from the entire class.

I was told that I had to see another young teacher, maybe a TFAer too, who taught Social Studies.  There were, again, about ten kids in the class, and there was a discussion question about whether or not Napoleon was ‘enlightened.’  The responses were from just two students back and forth and it really seemed like the didn’t understand the question.  (I wasn’t really sure what it meant either.)  This teacher definitely had a lot of charisma and some humor.  Any time a student said the word ‘French’ they had to do a French ‘Ha Ha Ha,’ which was a nice touch, but I don’t think I saw any real ‘learning’ going on in that class.

Most of my time was spent in math classes.  I didn’t see much innovation going on in any of them.  In one class the teacher taught a rule for solving a problem and then told the class “If you want to see a proof of this, come after school.”  In another I watched a teacher try desperately to get some kind of inquiry into his lesson.  He showed his class how a theorem involving triangles worked for an acute triangle.  He then asked “If I wanted to check to see if this was not just lucky, what other types of triangles might I try this with?”  It was a good question but the students were unable to come up with the answer.  It went on for a few minutes before he simply told them.

I saw a class that was called ‘precalculus’ but the material seemed to me to be basic Algebra, something with compound interest.  On the wall of the room I saw some student work displayed, mostly recent tests, and saw that the tests making the wall had scores in the low 80s.  Achievement was not as high there as KIPP PR would have us believe.  In that precalculus class the teacher mentioned that most people had failed the last test and that she would like for at least 75% of the students to pass.

An ‘English’ class I watched was merely students on laptops doing test prep.  I left that room as quickly as I could.

I did see a few things I liked there.  Like a computer programming class which was hands-on and also most definitely not ‘test prep.’  I also like an engineering class.  In that engineering class, students were working in groups trying to design cars they were going to build that would be powered by mousetraps.  I like activities like that and even though the students were making very little progress on the project, they were trying and I’m all for students doing something interesting and getting to exercise their creativity.

The most pathetic thing I experienced, though, and which I wrote about a little in my original post about my visit was a ‘study skills’ class.  KIPP has recently been priding itself on its ‘grit’ training program.  Just passing the tests is meaningless if the students have not developed the kind of attitude that will help them overcome the challenges they would face in college.  I actually like the idea of having a class where students can learn to develop these skills.  But what I saw revealed something that I suspected about KIPP all the time:  They don’t really have any special expertise in some of the things they claim to do well.

So this class had about 5 kids and one teacher and the teacher was asking the students to make a list of things that distract them when they try to work at home.  Rather than write the things down the students started either calling out things that distract them or complaining that nothing distracts them.  Later the teacher announced that as an incentive for these students doing their homework, they would get some candy if they would do one of their homework assignments that night.  Kids started calling out “That’s it, just one homework?” and then the teacher explained that the next week they would need two assignments and then it would increase from there.  The last fifteen minutes of class were spent with the teacher answering questions about what the minimum amount of homework would need to be done to qualify as one assignment.

I don’t know.  Maybe I came on a bad day to that class, but it really just seemed like a teacher winging it and not anything based on research or child psychology or anything.

Most of the teaching, I should stress, was not ‘bad.’  The issue I have is that the teaching would need to be spectacular to satisfy me.  Schools are getting closed all around the country, 20 in New York City, 20 in Washington D.C., 30 in Philadelphia, 50 in Chicago.  When politicians are asked why they are doing this they generally point to a charter school, often a KIPP, which ‘proves’ that all you need are ‘highly effective’ teachers and all students will excel.  This KIPP is the only KIPP High School in New York City so all the students from their middle schools filter into this school.  They will have their own multimillion dollar new building next year, paid for with money that I believe could have been better spent elsewhere.

What was strange was that the teachers and administrators who I spoke to, who were quite nice to me, were completely oblivious to the ed reform context in which they teach.  Maybe they were in denial about it, but it is pretty clear that kids, teachers, and schools are being punished all over the place for failing to live up to how great a school like this KIPP is supposed to be.  I’d love it if a KIPP teacher would come out and say “Please stop shutting down schools and using me as the justification for it.”  It seems to me that the whole charter school movement, at which KIPP is at the forefront, has benefited the small percent of students who make it through the KIPP program — they have a lot of attrition — and also benefits ‘the adults’ like the teachers and the administrators there, but that benefit has come at a much much larger cost, the destruction of neighborhood schools and displacement of unwanted students.  All in all, it is a large net negative, though it needn’t be.  With more honesty about what they are, and are not, accomplishing at KIPP and other charter schools, we could improve public schools rather than obliterate them.

My first thought was that since we have so much in common, these discussions would not have enough conflict to make them very interesting reading.  Matt says that he is generally on the ‘reform’ side of these discussions, though, so I asked him to ‘start’ the exchange.  It is interesting to be on the receiving end of one of these ‘open letters’ as I’ve initiated so many of them.  I can see why so many people didn’t respond to mine.  By the nature of the process, the recipient is likely to feel and seem like he is on the defensive.

Here is his letter, followed by my response:

3/29/2013

Dear Gary,

It was about two and a half years ago that I started reading your book, ‘Reluctant Disciplinarian’. I had just finished TFA’s institute and was a couple weeks away from my first day teaching English at a middle school in Colorado. I remember thinking your book was a bit cynical and a bit negative.

Fast forward a couple months. I wished I’d taken your book more seriously, and I realized the wisdom of many your views. In retrospect, I had been inculcated, by Institute, in the TFA culture of  ‘high expectations’ and I believed that I was an excellent classroom manager because I could control eight or nine summer-school students when multiple other adults were present. Not as easy in a real classroom, as I soon found out. TFA’s faulty training model is perhaps a discussion for another day.

Anyway, I discovered your blog several months ago, having recently finished up my stint as a TFA corps member, and beginning to get into education policy and then some education writing. I regularly visit your blog for what I consider to be the most clear-eyed, fair-minded traditionalist view. (A note on terminology: I tend to like ‘traditionalist’ rather than ‘anti-reformer’ for obvious reasons.) Unfortunately, though I respect much of what you write, I can’t say I agree with most of it.

So why am I writing you? Because I’m interested dialogue rather than monologues. Because I do fear that each side of the education debate has become an echo chamber. Because I’m hoping to have a meaningful discussion between TFA alumni who have divergent views on education policy.

It might be useful to start with what seems increasingly the biggest fault line between reformers and traditionalists: testing. Diane Ravitch recently said that the “most damaging things happening today stem from high-stakes testing.” I’ve got to admit that I’m baffled by the belief that high-stakes testing is that destructive.

I taught in a school district that was very reform-minded: we were regularly evaluated by principals and half of our evaluation was based on student tests, of which there were four or five high-stakes standardized assessments per year for each core subject. (The district’s former superintendent even wrote a report for the Fordham Institute on the evaluation system used.) I’ve written about some issues with my district’s evaluation system, but the amount of testing was, by and large, a good thing in my view.

What’s surprised me most about the anti-testing backlash is how little research is cited to support traditionalists’ arguments. (As a point of comparison, I’m more sympathetic to the anti-charter movement because there’s a solid research base for opposing charters. Though I don’t – a topic for another day, certainly.) Gary, you’ve long been critical of standardized tests, saying that you don’t ‘put a lot of stake into standardized tests.’ But what is your evidence for this, beyond your intuition and (admittedly extensive) experience?

Take a look at the research. The well-known Chetty study found that increased standardized test scores correlated with better life outcomes. The SAT has been a powerful predictor [pdf] of students’ first-year college GPA, correlating about as well as high-school GPA, which is pretty impressive in my view. There’s also strong evidence that the SAT is a good predictor of a student’s likelihood of graduating college. Yes, it’s a reformer talking point but I happen to think it’s true: standardized tests aren’t perfect metrics, but they are useful ones.

Traditionalists sometimes act as if preparing for a standardized test is a useless activity. Not so. Whether you like it or not, to be successful in many professions students will have to be successful at taking standardized tests. I had to pass a test to become certified as a teacher in Colorado; I’m now in law school and had to take the LSAT and will have to take the bar. Sure, there’s an argument that test prep has gone too far – and I would guess that that’s true in some schools and districts – but there should also be an acknowledgement that the ability to take a test has many real-world uses.

Finally, I think Paul Bruno has made the point well over at the Scholastic Administrator blog: it’s bizarre that many teachers are so opposed to testing when in fact they give high-stakes tests (high-stakes for their students, at least) all the time in their own classrooms. Why are standardized tests so fundamentally different than classroom tests?

I saw Karen Lewis speak at the University of Chicago a couple months ago, and she argued against standardized tests, saying, ‘My students aren’t standardized!’ That got some applause and head nods, but does that really make sense? Presumably what she meant was that all students can’t be measured by the same test – but when she was a teacher did she not give all students the same tests? Don’t you, Gary?

Say you go next door to a colleague, and design a test to give to all ninth grade algebra students, with the intent to assess them all equally, judge your own performances as instructors, and find areas of weakness, both yours and your students’. You’ve just created a standardized test, and yet you’d probably agree that doing so is pedagogically sound. I see standardized tests as scaling this idea.

Yes, we need to be careful with incentives when linking pay to test scores, we need to make sure tests are fair and accurate, we need to avoid narrowing the curriculum, and we need to ensure that teachers have a part in designing the tests (which my old district did to its credit). Of course there’s a lot of work still to be done on these matters, but it is work that can be done. These are issues that can be solved, not ones that warrant trashing high-stakes, standardized tests altogether.

Looking forward to hearing your thoughts,

Matt

My response:

4/6/2013

Dear Matt,

Thanks for taking the time to write to me.

No real educator, certainly not me, is opposed to tests.  As a math teacher I give tests all the time.  Over the years I’ve slowly increased the value of these tests, and now they are nearly 90% of my students’ grades for my eleventh graders (I also teach an elective called math research which is more project based and quizzes only count for 15% of that grade).  I take great pride in my ability to make a great test — one that is not too hard or too easy, one that has just the right amount of ‘skills’ but also challenges my students to prove that they understand the underlying ideas enough that they can apply them to an unfamiliar setting.

During my summer training for TFA in 1991 I remember reading somewhere that in an education utopia there would not be any grades.  Students would study and learn because they were intrinsically motivated and interested in the material.  I remember thinking, back then, that I agreed with this.  But I changed my mind pretty quickly once I became an actual teacher.  If you look through both of my books I wrote about teaching, you will see that tests and students succeeding on these tests are a big component of my philosophy of teaching.  Tests serve a lot of purposes.  For one thing, tests serve as a way to encourage students to study.  Also, tests are a way to reward students who have been diligently doing all their homeworks and punish, indirectly, students who have been cheating by copying homeworks from others.  In my own class it is not uncommon for me to say something like “Last year there was a ten point test question on this topic which a lot of people lost points on for making this mistake, so please be aware of this as you do your homework and study.”  I’d say that I refer to ‘the test’ in one way or another, on average, a few times each period.

At my school I’ve been the chairperson for the eleventh grade math final exam for the past three years.  In this capacity I have to collect questions from a team of teachers and compose a test by sorting out the ‘good’ questions from the ‘bad’ ones.  I teach at Stuyvesant High School which is, by many standards, one of the best schools in the country.  I compose a final which, if I did it right, should produce an average of a little under 90.  If it is way off of that, I have made the test too easy or too hard.  There was a time at my school where the teachers who taught the ‘honors’ eleventh graders didn’t want their students to have to take the ‘regular’ final.  They argued that it was too easy for them and would inflate their grades so they wanted to make their own test.  In the debate over this, which I was on the winning side, it was agreed that the honors students should have to take the regular final.  The main argument that was sold was that it is OK if the honors students ace the final since getting a good grade in honors will help them get into college.  In the back of everyone’s minds, though, was the suspicion that the honors teachers were concerned that their students would not do so well on the ‘regular’ final which would make those teachers look like they were not covering the material they were supposed to.  So, in that sense, I do like ‘standardized’ tests for many purposes.

Like the two people you mentioned, Karen Lewis and Diane Ravitch, I am not opposed to tests — or even standardized tests.  The issue I have is that the scores on these tests are being misused in a way that, in my opinion, will ultimately make education in this country (and even achievement) worse.  This is why I scoff at ‘high performing’ charters who beef up their scores in various ways (and STILL can’t get them to be very good, in most cases).  This is why I oppose value-added being a component of teacher evaluation.

I suppose the biggest misuse of testing data is in teacher evaluation.  I do not believe that ‘bad teachers’ are the problem in this country.  So I really don’t think that fixing the ‘broken’ teacher evaluation system will raise ‘achievement’ in this country.  Instead what will happen is that a lot of money will be wasted (not to mention instructional time) in creating, preparing for, administering, grading, and interpreting the scores.  Despite what we hear from ‘reformers’ from time to time, money does matter and giving more of it to a school, provided they don’t waste it (which is another issue …) is a good thing.  So I don’t like to see billions of dollars wasted on educational alchemy, in this case turning test scores into golden teacher effectiveness metrics which, in theory, would result in teachers learning how to improve and, if that doesn’t work out, how to be fired.

Already we are seeing that even under these new evaluation systems most teachers are being rated ‘effective.’  This is very frustrating to reformers who think that this just means that the new evaluation systems are too lenient.  Never would they think that this might mean that they were wrong about how many bad teachers there were.  This is bad science.  You should conduct an experiment to test a hypotheses and if the experiment disproves the hypothesis you don’t reject the experiment.

I do agree that if there are two teachers at the same school who have generally similar students for a bunch of years and for one of them the test scores on a uniform end of the year test are substantially higher than the other then it is probable that that person is a better teacher.  This is particularly true if that teacher was not ‘teaching to the test’ and, worse, choosing to not expose students to activities that promote critical thinking since that would take time away from preparing for the convergent thinking tests.

But when ‘high stakes’ are attached to these tests, a whole new dynamic opens up that, I think, hurts schools, teachers, and students.  Teachers would scan their class rosters in the beginning of each year wincing when they see that have some students who are disruptive.  Maybe some teachers have an ‘in’ with an administrator who will transfer those students to another teacher.  I’ve read about schools that have a term called ‘bubble kids.’  This means that they have done a ‘triage’ on their students where some will pass the test without little help and some won’t pass the test, even with a lot of help.  Teachers are instructed to focus on the bubble kids who are on the border of passing and failing.  This is an unethical gaming of a system that puts too much stake in ‘percent proficient.’

Value-added based on test scores is too inaccurate for it to really be any part of a teacher’s evaluation.  How can a system which I’ve learned can rate the same teacher in the same year as a highly effective 7th grade teacher yet an ineffective 8th grade teacher.  It makes no sense.  The tests, which I have analyzed on my blog, are not good enough.  For enough money it might be possible to create different assessments for students that might be able to isolate teacher quality.  These tests would be quite expensive to make and to grade, so I don’t see why we should spend all those resources trying to identify the few ‘bad’ teachers that every principal already knows who they are.

You mention the Chetty report that students with teachers who had high valued added had better lives.  But have you seen how easily that report has been refuted?  For one thing, the students with the ‘good’ teachers only made about $250 a year more, on average, and this was after they decided not to count the students who were making the most money — throwing them out as outliers.  Perhaps this is why they didn’t submit it for peer review.

I think the very worst misuse of testing, though, is when it is used to label schools, through a value-added type calculation, as ‘failing’ or ‘high achieving.’  In my research I’ve found that often there is not much of a difference between the two schools, and when there is a difference it is likely because the ‘high achieving’ school (often a charter school) has ‘better’ kids.  Standardized test scores are being used as a weapon to demolish schools, most recently over 50 in Chicago, across the country.  Whatever benefit there might be from high-stakes use of standardized test scores (They motivate the miniscule fraction of teachers who aren’t otherwise motivated to try?  They make some charter operators very rich?) this is far outweighed by the damage they have done.  I’d be very happy if schools stopped getting closed down over standardized test scores and if teacher pay and job security were not linked to it.

My belief is that the misuse of testing is beginning to backfire on the reformers.  With the new systems being rolled out around the country, as part of Race To The Top, it will soon be clear to everyone how far we are from being able to use these test scores any anything more than a very rough idea of where some students are in their knowledge.

Thanks again for writing.  I hope I’ve answered your questions and feel free to write back as often as you’d like.

Gary