Aaron Regunberg: The Importance of Student Attrition Rates
Friday, December 30, 2011
Question: if a doctor were given a month to treat a hospital ward of 47 people, and over the course of her time there 23 of her patients regained their health but 24 of them transferred to other wards before they had been healed, is it right for that doctor to say that she cured 100% of her patients?
Answer: I guess it depends. On the one hand, I suppose you could make an argument that since the remaining 23 patients all were healthy, then her 100% claim was valid. But I think most people who looked at this situation would have a problem with that characterization, particularly if they learned that the doctor had a great deal of discretion to determine who stayed in her hospital wing as well as a clear incentive to retain only those who would be the easiest to cure.
Under these conditions and incentive structures, it would be reasonable to question whether the doctor did anything to push her really hard to treat cases out of her ward. And even if our doctor were made of better stuff than that—let’s say she had simply devised a treatment regimen that caused more than half of her patients to leave of their own volition—I don’t think many of us would buy the assertion that this treatment could help turn around the whole hospital, which as an entity has a responsibility to treat every patient who walks in its doors. In fact, I believe most readers would find it very problematic if this doctor’s claims of a 100% cure rate were being used to convince the chief of medicine to redirect money from the other wards to her own division.
Misrepresentation
In such a case as this, I would argue that our doctor’s attrition rate matters a great deal in evaluating her efficacy as a medical practitioner, and certainly should be used to question the value of giving her more of the hospital’s scarce resources. I’d also argue that what seems like clear common sense in this clumsy allegory should likewise apply to other situations, like—and I’m sure most readers already know where I’m going with this, since I’m not that sneaky—the privatization of public education.
To make the example more specific, for the past few months lobbyists from Achievement First have been leaning heavily on their claim that 100% of AF high school students get accepted into four-year colleges or universities. I haven’t been to a single hearing or presentation on this issue (and believe me, I’ve been to a lot of them) at which they didn’t use this statistic as their heavy-hitting closing argument. And I’ve seen how effective this line can be, and the way policy-makers’ faces change after it’s delivered—it wins them over, plain and simple, and why shouldn’t it? If it were true, it would be an eye-catching accomplishment.
The only problem is that the claim is just as much of a misrepresentation as our allegorical doctor’s was. Achievement First has only had two graduating high school classes thus far, in 2010 and 2011. The 2011 graduating class began in 9th grade with 47 students. By 10th grade it had been winnowed down to 37 students, by 11th grade to 29 students, and when their last year began only 23 students remained. The same is true of their 2010 graduating class, which fell from 45 students in 9th grade to 22 in 12th. So, sure, 100% of their 12th grade classes may have graduated and been accepted to college, but that’s not 100% of their high school students—it’s less than 50%.
Achievement First has claimed these steep attrition rates occur because of students’ desires for additional extra-curricular activities that are not offered by AF. At a recent City Council Education Sub-Committee hearing at which AF advocates presented, they mentioned a student who had been having a great time at Achievement First but decided to leave because another school had a lacrosse team. Of course, I can see these kinds of decisions accounting for a few transfers every year, but I find it hard to swallow that more than 50% of multiple classes left because they wanted to play lacrosse. (Incidentally, I’m also a bit disturbed by how plainly this argument contradicts an oft-repeated AF talking point stressing all the extended-day sports, arts, and extra-curricular activities their schools are able to offer).
An Enormous Advantage
We can find a more likely explanation for their attrition rates in our medical allegory from above. Clearly if Achievement First wants to ensure that 100% of its 12th grade class graduates and gets accepted to college, it has a huge incentive to retain only those students who are more likely to graduate and go to college. And clearly their schools have a great deal of discretion to create just such a situation, either through explicit push-out measures or by devising a system that causes harder-to-serve students to leave of their own volition.
Of course, students in Providence and other low-income communities tend to move around a lot in general. But as Richard Kahlenberg pointed out in a Washington Post article earlier this year, the big difference between schools like Achievement First and their residing districts is that whereas struggling students come and go at non-charter schools, at Achievement First students leave but very few new students enter. Having few new entering students is an enormous advantage not only because low-scoring transfer students are kept out but also because in the later grades, AF students are surrounded only by their peers who are the most committed to the program--a luxury that other high schools cannot share.
There are many Providence community members who have argued that none of this matters, that even if all the hype and spin about AF were true and they did get 100% of their high school students to graduate and go to college, the Mayoral Academies proposal would still be a bad idea because, in the end, it means taking a significant amount of money out of a school district that is already starved for resources. And I probably agree with that to a degree. But it is significant that so many of AF’s claims don’t seem to hold up when you look past the shiny veneer. I just hope that enough of Rhode Island’s policymakers are willing to search deep enough to do so.
If you valued this article, please LIKE GoLocalProv.com on Facebook by clicking HERE.
