In a bizarre statement, somewhat akin to Arne Duncan’s famous utterance about Katrina being the ‘best’ thing that ever happened to New Orleans schools, is something that NY city Chief Academic Officer, Shael Polakow-Suransky, said about the latest round of New York City school closings.
Last year 116 elementary and middle schools had gotten an ‘F’, a ‘D’, or three consecutive ‘C’s which made them candidates for closure. Ultimately 14 of those schools were closed, though it required a court order to rescue a bunch of others. This year there are 217 schools that have passed the threshold. With regard to which of those 217 will be targeted for closure, Polakow-Suransky in the Daily News said things will be looked at on a case by case basis because “This is an art. It’s not a science.” An art? I don’t know if that was the best choice of words. Ballet is an art. Oil painting is an art. Ventriloquism is even an art. But shutting down schools based on flawed metrics? That’s just cruelty.
The way the grading system goes in New York City, 30% of the schools get ‘A’s, 35% get ‘B’s, 25% get ‘C’s, 7% get ‘D’s, and 3% get ‘F’s. No matter what the scores are, even if everyone does well, 10% will get a ‘D’ or an ‘F’. I’ve found the rating system to be very unscientific and have written about it extensively. To show how unreliable the statistic is, I compared the percentile rankings for schools in 2011 against the same school’s percentile ranking in 2012.
In fairness, this does reveal a slight correlation. The correlation coefficient is about .4, which for some situations borders on ‘moderate’. But even if there were a very strong correlation, this would not necessarily mean that this is an accurate measure of school quality. it could just be biases that are favoring certain schools and will favor them, or hurt others, in the same way in two consecutive years.
The NYC DOE offers a statistic to claim the stability these scores. In the DOE press release they said “Grades remained stable across the city and for individual schools, as 86 percent of schools did not change more than one grade from 2011; 97 percent of schools were within two grades.” This is another example of ‘how to lie with statistics.’ The number 86% is only meaningful when compared to what would be ‘expected’ if the scores were completely random. Even if the scores were completely random, there would still be about 70% of schools either staying in the same category or moving just one category up or down.
Here’s why: Suppose there are 100 schools and in 2011 30 got an A, 35 got a B, 25 got a C, 7 got a D, and 3 got an F. For the 30 A schools in 2011, even in a randomized system, 60% of those 30, or 18, would get an A or a B in 2012. For the 35 who got B in 2011, 90% of those 35 would get an A, B, or C in 2012, which is 31.5. Of the 25 who got a C in 2011, 67%, or 17, would get a B, C, or D. Of the 7 who got a D in 2011, 35% of those, or about 3, would get a C, D, or F. Finally, of the 3 who got an F, 10%, or about .5 would get a D or F in 2012. When you add those numbers up, 18+31.5+17+3+.5=70.
For moving up to two grades away, a pure randomized method would still yield 92% staying the same or moving at most two categories. This is the kind of thing that can happen when there are really just three categories that dominate the five altogether. For example, everyone who gets a C will move at most two categories since there are only two above and two below it. Also 97% of people who get a B will have to stay within two categories, since only the 3% F would be beyond two moves.
Perhaps more informative is the fact that only 41% of the schools stayed in the same category from 2011 to 2012, which is better, but not by that much, than the 28% that would if the grades were assigned randomly.
The six schools that had the biggest drop from 2011 to 2012:
P.S. 041 Gun Hill Road 94% to 12.5%
P.S. 108 Assemblyman Angelo Del Toro Educational 93% to 17.5%
The 47 American Sign Language & English Lower School 91% to 20%
P.S. 183 Dr. Richard R. Green 85% to 19%
Eagle Academy for Young Men 81% to 14%
Goldie Maple Academy 80% to 10%
Today’s Daily News Reported that the first 36 schools have been named and have been in ‘discussion’ with Marc Sternberg as the process for trying to close them down gets underway. Apparently 5 out of the 7 schools that were targeted for closure before a judge prevented it were not on the list again. Also, 7 of the schools on the list are schools that had replaced other schools that had been shut down previously.
Will this be Bloomberg and Polakow-Suransky’s Requiem? We can only hope it is a symphony that remains unfinished.
Gary Rubinstein
test
Your blog has provided a great articulation of all of the issues I have had with TFA throughout my time as a corps member. As a second year corps member, I appreciate your honest critique of TFA, and you so clearly put into words all of the things that I struggle with by being part of this organization. From TFA’s problematic relationship with charter schools (I am not working in a charter school but over half of my corps does) to their inadequate preparation of teachers like me in a specialized position (bilingual elementary education), I feel conflicted every day that I am part of this organization. On the one hand, I am thankful to TFA because it allowed find my passion for teaching, which I plan to make my career, but more often than not I feel frustrated by the direction in which the organization is going. Your blog reassures me that I am not just a bad teacher or a bad fit for TFA (which I am often made to feel for not strictly following the TFA line). It reminds me that these issues with TFA I have discovered for myself as a corps members are not only real but are having a serious impact on the future of education reform in our country. Thank you for lending a very public voice to these issues. Your blog has made me realize that I am not alone in feeling the way I do.