New Orleans was the home of a very important battle in The War of 1812.  Exactly two hundred years later, 2012, New Orleans is again the focus of another type of battle, one of how to best help our schools.

About a year ago I analyzed a press release by TFA alum and then Recovery District Superintendent (RSD) John White about how the New Orleans RSD had scored first in the state on a growth metric.  In my analysis I demonstrated that this was a statistic that was meaningless as it is inflated for districts with low starting scores.

Since that time, John White has been promoted to commissioner of education in Louisiana, after just a few months in his role as superintendent of the RSD.

In a press release from yesterday, White and the new Superintendent, Patrick Dobard, spoke about how the new ACT results prove what progress New Orleans is making since the majority of students attend charter schools there.

The average composite score on the ACT for students in the Recovery School District (RSD) New Orleans rose four-tenths of a point from 16.4 to 16.8 from 2011 to 2012 – representing an increase four times the statewide average which rose one-tenth of a point from 20.2 to 20.3 during the same timeframe.  In the last two years, the average composite score for RSD New Orleans has increased a total of 1.2 points, up from 15.6 in 2010, ranking fifth in the state for overall progress.

There’s a famous book called ‘How To Lie With Statistics’.  This is a good example of inventing statistics that sound good, which I’ll explain three examples here.

1)  First of all, a score of 16.8, is very bad.  It is somewhere around the 20th to 30th percentile.  Notice we’re not hearing very much about the 90-90-90 schools anymore?

2)  The new thing, though, is ‘growth,’ and since they say they got “an increase four times the statewide average” that does seem pretty good to someone who has not studied a lot of math.  The flaw with this statistic is that in this case it is not appropriate to measure how many times bigger one gain is over another because the statewide gain was just a tenth of a point.  When you are comparing a number to something so close to zero, it is very easy to get something that is many ‘times’ bigger.  So if the state’s gain was a hundredth of a point then the RSD’s gain of four tenths would be 40 times as big.  But it would still be pretty meaningless.  Basically, zero times anything is still zero.

3)  But they got ‘growth’ for two consecutive years.  Isn’t that a trend that will continue until they eventually overtake the state scores?  No.  More likely the gains will level off, as they have already started doing.  So in 2010 they had a score of 15.6.  Then in 2011 they went up by .8 to be 16.4.  Now in 2012 they went up by .4 to reach 16.8.  Notice that their rate of increase has slowed down (for you math people, the second derivative is negative!)  So what will happen next?  Well, nobody can really say, but one possibility is that they have hit diminishing returns so they .8 increase got cut in half to become a .4 increase.  If that continues, they will only go up by .2 next year and .1 the next and never get much higher than 17.

In summary, I’m not impressed by the latest New Orleans ACT results.  And now they are expanding the program to have all 8th, 9th, 10th, and 11th graders take it so they can measure their growth more objectively.  While I do like the idea of using a test that they don’t make or grade, I have to question whether or not it is a good use of money to give to The College Board for all this testing.  After it is all done, all they’re going to learn is that it would have been better to put that money into the schools instead of The College Board.