Is Success Academy the Climate Change of K-12 Education? (Part 1 of 3)
Former Vice President Al Gore released An Inconvenient Truth in 2006, making the case that the Earth had experienced climate change resulting from rising carbon dioxide emissions. Eight years later, a Gallup poll showed that forty-three percent of Americans still do not believe effects of pollution have caused a rise in the Earth’s temperatures.
In the face of climate models which predict that adding carbon dioxide to the atmosphere would lead to higher temperatures, and strong evidence that both temperatures and the levels of carbon dioxide in the atmosphere have actually risen, a surprising number remain convinced that climate change is a farce, a hoax manufactured by leftist scientists.
The same year that Al Gore released his documentary, former New York City Councilwoman Eva Moskowitz opened the doors of her first charter school, Success Academy - Harlem 1. In the eight years that followed, her schools have also faced a large amount of skepticism.
Despite mounting evidence to the contrary, a large and vocal group of observers dismiss the performance of Success Academy as unremarkable, believing it to flow from bad actors and incessant test-prep, rather than legitimate strategies which might be implemented elsewhere.
In this three-part series, we analyze Success Academy’s performance on the just-released 2014 New York state assessments, attempt to evaluate claims that Success Academy’s achievement may be the result of biased inputs, and finally look at some aspects of the model which make the school unique.
The crudest (and unfortunately most widely used) way of analyzing school performance is to simply compare schools based on their average scores. Below are the schools ranking in the top 25 among 1,252 NYC tested schools. Five of the 25 are Success Academy schools.
As most readers will be quick to observe, this sort of approach leaves much to be desired. With no consideration of the population served by each school, comparisons are often unfair. Instead, schools are best evaluated when they can be benchmarked against others who serve similar students. Consider the graph below which plots the scores of all NYC schools against the Economic Needs Index developed by the Department of Education.
Remarkably, 62 percent of the variation in school scores is explained by the economic needs of the population served. By considering the population served, we are able to reach a much more reasonable expectation for each school’s achievement and can identify peer schools to serve as benchmarks.
The graph below highlights Success Academy schools in this context, and using the selection tools to the right, readers can also highlight performance of other groups including the citywide gifted program and other charter networks.
At every single Success Academy school, from the relatively low needs Success Academy - Upper West to the relatively high needs Success Academy - Bronx 1, performance is remarkable. On average, Success Academy schools are scoring 39 points above what would be expected based on the economic needs of the students served.
To put these 39 points into context, education researchers often convert points to the corresponding number of years of education they represent. In this context, we see that Success Academy students are on average 3.0 years ahead of the students who attend schools with similar economic needs.
This achievement is not only remarkable; it is also unique. No other charter network overachieves at a similar rate. Even the citywide gifted and talented programs (which typically require students to score in the 99th percentile for admission) do not show similar overachievement.
After Success Academy, the two groups which show the greatest level of overachievement are the citywide gifted schools and the Ichan Charter Schools. While the achievement at both of these is also impressive, the overachievement at Success Academy is double these groups.
It is clear from the above analysis that Success Academy students significantly outperform their peers on state exams; yet, a number of important questions remain. For example, can comparisons really be made between traditional schools and charters where parents must opt-in? What about attrition? What about English Language Learners and Students with Disabilities? These questions and more are the subject of part two of our analysis.
by John Keltz and Jarod Apperson.
John Keltz (@keltz_) is a researcher for the Atlanta Public Schools. He graduated from Case Western Reserve with a bachelors degree in Economics and Math. He earned a Masters of Economics from The University of Wisconsin - Madison.
Jarod Apperson (@gradingatlanta) is a Graduate Research Assistant at Georgia State University. He graduated from New York University with a degree in Finance and Accounting.
 The standard deviation of New York state exams is 35, so Success Academy has an effect size of 1.11 standard deviations. The estimated effect size of a year of schooling in grades 3-8 is about 0.37. Therefore, the overachievement at Success Academy corresponds to approximately 3.0 years of schooling.