Somewhere, Woodward and Bernstein are weeping.
I entered the fight against the ‘reformers’ back in February after hearing Duncan claim that a school in Chicago got dramatic results by shutting down and replacing with a charter school in the same building with the same kids, but with different adults. It was important for Duncan to have at least one ‘miracle school’ to prove that his style of reform was reaping results.
Knowing this couldn’t possibly be true, I investigated and found him to be using statistics in a very misleading way. This spurred my contacting the ‘leader’ of the other side (are they ‘anti-reformers’ or just ‘pro-research’?), Diane Ravitch who then featured my investigation in a New York Times OpEd which generated a lot of attention.
To clarify: A ‘miracle school’ is one that is getting amazing graduation rates and standardized test scores while serving the same populations with the same budgets as the failing school around the corner. The only difference is that the ‘adults’ (teachers and administrators) are working harder and smarter with higher expectations.
Some read this OpEd as a claim that poor kids can’t learn so we should just give up trying. I knew that the main point was that since even the ‘miracle’ schools weren’t getting such dramatic results in a short period of time, there is no reason for us to be shutting down schools and to be firing teachers at schools whose results aren’t all that different from the supposed miracles.
This OpEd was so influential that ‘reformers’ stopped touting imaginary miracle schools and have switched their attention, for the most part, to finding schools that are exhibiting a lot of ‘growth.’
Still, though, education reporters seem to have a short memory. A few months pass and the reporters begin, again, to write stories about these miracle schools which then requires me to get my ‘google’ going to ‘debunk’ these claims.
I should make it clear that I don’t believe that standardized test scores correlate with how good a school is, how good the teachers are, or how much the students have actually learned. So when I show that a ‘miracle’ school actually does not have the test scores it claims or that the test scores are misleading, I am not saying that the school is not a good school or that kids are not learning there. There are good schools with good test scores, good schools with bad test scores, bad schools with good test scores, and bad schools with bad test scores. The things just don’t correlate. I debunk to make this point with the hope that this country gives up the standardized testing obsession and using computers to calculate how much ‘value’ a teacher or school ‘adds.’ Good administrators at all levels can tell if a school is trying or if teachers are trying by observing those schools or teachers for short periods of time.
Another criticism that I’d like to dispel about the so called ‘anti-reformers’ is that we think that since teachers can’t be expected to overcome all the distractions of poverty in a short period of time with diminishing resources, that we think that teacher quality or school quality doesn’t really matter since no matter how good you are, you can’t get these results without some kind of deception.
As someone who likes to think that I’m an above average teacher, I believe that I make more of a difference on my best days than I do on my worst. Whether these ‘differences’ result in standardized test gains, I don’t know. If I teach to the test, I can probably get score increases at the expense of real learning, so I prefer not to teach to the test. It is better to have good teachers than bad teachers. That’s obvious to everyone.
But there is a disagreement about how much the quality of teaching varies. I think there is a bell curve with regard to teacher quality with a small percent being very bad and a small percent being truly exceptional. Most are near the middle. Holding teachers more ‘accountable’ is not going to turn average teachers into exceptional ones, nor will it turn bad teachers into average one. Supporting teachers will make everyone that much better.
So it is with the above disclaimer that I want to share my latest two miracle school debunkings.
This first school I learned about Harvest Preparatory Charter in Minneapolis, MN. It was reported in the newspaper that they scored 77% proficient in reading and 82% proficient in math. In the article, they describe that the school does a lot of “drill and kill” which indicates a lot of test prep.
I downloaded the NCLB Data Report to see what I could learn about this school that wasn’t reported in the article.
Here are some things I learned about the school:
1) Though the school has 343 students, their test scores are only based on 120 students. This is a small sample size.
2) The school had a lot of attrition, 121 total 3rd, 4th, and 5th graders in 2010 became 82 total 4th, 5th, and 6th graders in 2011.
3) At least one of the tests they don’t do “kill and drill” for is the 5th grade science test. In that they had only 7% proficiency.
4) 4th grade math went from 53% in 2010 to 88% in 2011. It will be interesting to see what the scores are next year.
Now, this school may be a good school, or might not be. Those test scores, my point is, don’t tell the full story of this school.
The next school I learned about is a high school in Kentucky that is in the process of being ‘turned around.’ They got extra money and had to replace half their staff under the model they chose. The school, Academy@Shawnee, as reported by Education Week (you need to register to read this ) reported that they went from 5% math proficiency in 2010 to 25% in 2011 and from 22% reading proficiency in 2010 to 45% in 2011.
Now, this does sound pretty impressive if the numbers mean what we are intended to think they mean. But I went to the state education website for Kentucky where I learned that the math score is just based on their 100 eleventh graders, while their reading was just based on their 100 tenth graders, NOT the entire school as the article suggests.
All we can say is that 20 more 11th graders met the cutoff in math and 20 more 10th graders made the cutoff in reading from 2010 to 2011. Nothing more. No miracle. Might be a fine school, but good test scores don’t prove that just as bad test scores don’t prove a school is not good.
True they had 5% passing math in 2010 and 25% passing in 2011, but over past five years, the scores tell a different story.
Math proficient percent
The 5% was a serious outlier, so should not be used to hail improvement. They are back to 2007 level.
Reading proficient percent
The scores started dropping when the current principal took over and now he’s gotten them back to their 2007 levels.
So we have to really look at what the statistics mean before we declare a school to be ‘turned around.’ Knowing that they had to get those two metrics up, they could have invested a lot of energy into helping the 100 tenth graders prepare for reading and their 100 eleventh graders prepare for math.
Education reporters need to do their jobs. And they have access to so much more information than I do with my primitive Google searches. They have lawyers on staff who can help them obtain information through the ‘Freedom Of Information’ act, which I don’t think I have the ability to navigate.
What is disappointing about these ‘miracle schools’ stories is that education reporters are chasing after the wrong scoop. I can see why they want to do a feel good story about a school beating the odds. Certainly “School Still Stinks” doesn’t make a great headline, but the story doesn’t have to be that.
Education reporters have the BIG story right in front of their noses. Rather than praise a school for meaningless stats, the big story is that other schools are being shut down and their teachers fired for the same meaningless stats.
It’s like if you were a religion reporter. What is more likely to earn you a Pulitzer prize, a story about how The Virgin Mary was spotted on a grilled cheese sandwich or a story with proof that millions of people are praying to a false idol?