Bad Arguments

WCPSS recently responded to the United States Office of Civil Rights in connection with an ongoing investigation of reassignment decisions from Spring 2010. Some shoddy arguments were made in that response. Because the reassignment process has slowed, I decided to postpone my final post on the Wake School Choice Plan to address this topic.

I will not address every misstatement and bad argument recounted in the response. Because of the way the law works, the response recounts some misstatements and bad arguments by individual board members to show through their public statements that they did not intend to discriminate against black and hispanic students. For this purpose, it is legitimate to recount misstatements and bad arguments. So while I have discussed some of these before, I will not focus on them here. Further, most of the arguments in the response, like the argument that the challenged actions have not disproportionately impacted black and hispanic students, are micro-level arguments against something called “disparate impact” that take advantage of the fact that the complaint has not been amended to include the latest round of 2011 reassignments, which almost certainly had a disparate impact on black and hispanic students. These arguments, right or wrong, fall into an “overtaken by events” category that makes them not so interesting outside the legal arena. 

Beginning on page twenty-four of the response, however, are a series of arguments designed to show that “a majority of the Board has reasonably concluded that the district’s use of SES as a student achievement factor has not resulted in marked educational benefits but has imposed unfair burdens on poor and minority students.” Whatever the Board majority may have concluded, and however reasonable it may or may not have been, the arguments themselves are not very good. It is these arguments that I address below.

1. No studies show that SES diversity has improved academic achievement for poor and minority students in WCPSS.

“No studies show that the absence of a meltdown at Shearon Harris Nuclear Plant has markedly improved public health in Wake County.” I’m not a big fan of sensationalist argument, but sometimes it gets the point across. The case against high poverty schools is not as strong as the case against nuclear meltdowns, but the fallacy of relying on the absence of a local study is the same in each case: there is no good reason to believe that things will turn out differently in Wake County than they have elsewhere. The diversity policy retards the growth of high poverty schools. High poverty schools have been studied extensively, throughout the nation, from a variety of angles and approaches, at different levels of rigor, and the answer is clear: avoiding them is a good idea. Individual high poverty schools succeed, but as a group, they are systematically less likely to produce good results for all demographic groups than their low poverty counterparts. Further, most successful high poverty schools require greater resources than their low poverty counterparts and/or rely on peculiarities of situation that cannot readily be replicated or scaled. 

2. Comparison to other North Carolina School Districts shows that the SES diversity policy has failed.

Some fairly comparable school districts, like Charlotte Mecklenburg Schools and Guilford County, are now outperforming WCPSS in important ways. (NB: The WCPSS response overstates the case by highlighting only systems and metrics for which this is true.)

This does not mean the diversity policy has failed. While the comparison districts achieve better results in certain areas despite the absence of a diversity policy, you can’t use this to demonstrate that the diversity policy has failed. It may just show that whatever benefits the diversity policy has provided have, in the last few years and in certain respects, been eclipsed by some combination of things these other districts are doing right combined with things WCPSS is doing wrong.

To make a decent argument that the diversity policy has failed, you would have to show that some aspect of the diversity policy (or, conversely, the neighborhood schools policies of these other districts) actually caused WCPSS’s results to decline or the other districts’ results to improve. The paper does not do that, and it would be difficult to come up with an argument of this kind that is supported by data.

3. WCPSS student achievement data show that the diversity policy has failed.

WCPSS makes two arguments here. First, it points out that the proficiency of students receiving a free or reduced lunch on standardized tests is not “clearly” and “inexorably” correlated with the poverty level of the school. Second, it notes that minority students who attend magnet schools fare worse than minority students who attend nonmagnets.

The first argument means you cannot prove the diversity policy is working via the correlation between school poverty and the educational results for poor children. The correlations are too low, because many things, not just the poverty level of a school, determine how well it educates poor children. But it does not prove the opposite proposition, that the diversity policy provides no benefit. On that score, there is some suggestive evidence to the contrary. If you plot the reading proficiency of white, black, hispanic, and poor children against the increasing poverty levels of their schools, for example, you will see a consistent downward trend for all groups and a notable drop in the results achieved by the best schools at any given level of poverty.

The second argument, comparing the performance of minority students at magnets to minority students elsewhere, reflects the fact that the minority students in our magnet schools are more likely to be poor students—indeed, particularly poor students—than minority students elsewhere. This arises because most magnet schools are located in neighborhoods that are both poor and majority-minority, while most other schools are not. There could be something here, but unless you extract the Asian students from the “minority” category (because they perform as well as white students in WCPSS), then control for levels of poverty in the minority populations, the argument is worthless.

4. The SES diversity policy unfairly burdens poor and minority students.

Two distinct claims are made here. First, WCPSS points out that poor and minority students are more likely to endure long bus rides than their nonpoor, white counterparts and were more likely to have their magnet applications rejected. Second, it argues that longer bus rides show a “troubling correlation” with weaker academic performance.

The first claim is true. Certain poor and minority students pay a price in lost time and choice so that those same students and others can attend lower poverty, less racially isolated schools than they otherwise would. It is always legitimate to ask whether this is fair. It is hard to find an objectively right answer, but I would weigh most heavily the opinions of the communities most affected.

As for the second claim, there is a “correlation” between distance to base assignment and achievement, but it is not as “troubling” as the WCPSS response hopes you will think. Students who travel farther to their base assignments tend to perform less well because they tend to be poorer, and our poor students—like the poor students in every public school system—tend to perform worse, as a group, than the nonpoor.

There is a misperception here, often shared by both sides, that a long bus ride, by itself, could or should improve academic performance. Busing a child away from a neighborhood magnet school maintained at 40% FRL to another, more distant school at 40% FRL will not do this, because the child’s performance should be similar in both schools. What the bus ride does is permit the existence of a system of schools where poor and minority children do not have to attend a poor, racially isolated school.

5. Eliminating the SES diversity policy is unlikely to harm students because the Board majority has established new programs that will close the achievement gap “in other, more effective, ways.”

The Board—the whole Board, but led in some of its efforts by the “majority” identified in the response—has proposed changes in policies and programs that may help close the achievement gap. Some of them, like using objective and documented criteria for math and other program placements and using effectiveness data to place leadership teams and teachers, are likely to reduce the achievement gap. Others, like merit pay for individual teachers, are likely to be a waste of money (see here and here). Still others may be the right thing to do (e.g. reducing suspensions), but their likely effect on the achievement gap is unclear.

None of them has been shown to be more likely to reduce the “achievement gap” than a policy retarding the growth of high poverty schools. More importantly, however, these ideas are in no way inconsistent with the diversity policy. It is entirely possible to implement them without promoting the growth of high poverty schools, so they are not a valid argument against the policy.

Conclusion

In the WCPSS response to the Office of Civil Rights, crafted by lawyers, the above-mentioned arguments are carefully hedged. They are presented not to show their correctness, but to show that they were plausible enough for Board members to reasonably believe them. As you might expect, this subtlety has been lost on or ignored by some members of the Board and the press, who have presented these suspect arguments as conclusive proof of something they do not show.

I have attacked these arguments because they are bad arguments in support of a worse idea: the elimination of a policy that has retarded the growth of high poverty schools. Those who purport to be data-driven have a duty to use data to inform, not deceive. The response fails to do this, and we should condemn the use of these particular arguments by anyone who claims to be data-driven.

At the same time, WCPSS has lost ground in recent years when compared to fairly comparable systems. If the diversity policy did not fail us, other policies necessarily did. It is no answer to blame “growth” for this failing. Growth is a good thing for schools. Just ask Detroit. More importantly, growth is something that will one day return, whether it is good or not. When it does, we must have have a solid understanding of what we did wrong and how to do it better—despite growth—or we will be left in the dust. Some efforts of the ED Task Force are solid steps in that direction.

The Wake School Choice Plan (Part 3)

This post continues my exploration of the Wake School Choice Plan (WSCP), a student assignment proposal advanced by the Wake Education Partnership. This third post focuses on whether the WSCP would use our schools efficiently. I tackle this question from three angles: facilities use, transportation cost, and “other.”

Facilities Use

The WSCP makes a strong effort to match the population of its three “areas” to the capacities of their schools. Generally speaking, this sort of approach should promote efficient facilities use, though it is not much different than what we try to do now.

The WSCP also assigns kindergarten students and other “choosers” one at a time. This does differ from the node-at-a-time approach we now use. Further, this approach should permit the WSCP to better use capacity limits to ensure that there are few, if any, crowded schools in 2012. Although the plan would allow this, it does not appear to do it. It appears to favor giving people their choices instead, contemplating school crowding of up to 125%. This level of crowding is extreme by today’s standards. This problem could easily be fixed by restricting crowding to lower levels.

In comparison to our current node-based assignment approach, I think the WSCP will have a harder time using middle schools and high schools efficiently. The difficulty arises from the plan’s “feeder patterns.” Students in a given elementary school will automatically be assigned to a particular middle school. Students in a given middle school will automatically be assigned to a particular high school. Feeding one entire school into another—rather than particular nodes, as we do now—does not provide much flexibility to adjust enrollment at the middle and high school levels.

While the plan specifies which schools are in each of its three “areas,” it does not identify the feeder patterns for each area. This makes it hard to demonstrate the problem convincingly, but there will almost certain be one, particularly at the high school level, where our high schools already operate at 112% of their optimum capacity. This must be watched carefully.

Transportation Cost

Despite the many choices it provides, the WSCP posits significantly lower transportation costs than now exist based on a projected annual reduction in student transport distances of approximately 12.2 million miles. It achieves these projected savings in the following ways: (1) it assumes that students in a choice regime will overwhelmingly attend their nearby schools, and that far more of them will be able to walk; and (2) despite indications that the WSCP would preserve the current systemwide magnet program with transportation, the plan seems to ignore the associated transportation costs.

With respect to the first point, the WSCP’s transportation cost estimates claim to assume something called a Reverse Fibonacci distribution of students. (I don’t think it adheres to the assumption for middle and high schools, but this is not discussed and cannot be demonstrated from the limited data provided.) This distribution posits that 38% of elementary students will attend their closest school (lower than the 53% who do so now), but 77% will attend one of the three closest (much higher than current conditions) and only 3% will attend one of the three farthest options.

The WSCP provides no empirical support for the assumption, which is said to be based on experience. I doubt it’s accurate. To me, it seems likely that an even higher number will select their closest school, because it is closest and because the WSCP strongly incentivizes you to choose it by giving you priorities if you do. After that, however, I think the choices of many will focus on the magnets among the choices or on the schools with the fewest low achieving, or poor, or nonwhite students, whether they should focus on these considerations or not.

As to the second point, there is simply no discussion of systemwide magnet transportation, from which I deduce that its cost has not been factored in. Nor is there any discussion of the cost of providing transportation to grandfathered students, which is likely to be substantial in the short run.

On the plus side, the WSCP transportation analysis may underestimate the efficiencies that could be generated through the above-discussed feeder patterns. These might—if students strongly select their neighborhood schools, and if the feeder patterns can actually be implemented—streamline current transportation patterns even further than the plan anticipates.

On the whole, the WSCP’s transportation cost proof posits savings of over 12 million student miles traveled. This sounds like a lot. But students currently travel 223 million miles, so it really isn’t. It’s about 5% of current miles. And this assumes that the plan’s projections are accurate rather than optimistic.

It was politically necessary to try to show that the WSCP could be implemented without increasing transportation costs. It may even be true. But it is important to keep in mind that whatever the difference in cost between the old assignment plan and the new one, it is unlikely to be material if the new plan works better. Total transportation costs are about 5-7% of our overall and local spending, so any difference between two reasonable plans is likely to be small.

Other Costs

This plan has two other potential costs that are not discussed: the cost of “outreach” and the cost of enhancing underchosen schools. I will not take these up here, because I have no way to estimate the cost of “outreach” designed to induce folks to make good choices, and the cost of enhancing underchosen schools is one we will likely bear in some form or fashion regardless of the assignment mechanism.

Conclusion

If the WSCP did not worry too much about whether folks were actually granted their wishes, and if it did not incorporate guaranteed feeder patterns, it would not be difficult for it to ensure near perfect utilization of every school. Because the WSCP does try to grant wishes, and does incorporate guaranteed feeder patterns, there will be some significant inefficiencies, just as there are now.

The WSCP’s transportation cost estimates depends heavily on the assumptions it makes about where children will choose to attend school. While I am not convinced that transportation costs will be lower under the WSCP, I doubt they will be materially higher.

In a nutshell, I would not focus too much on the efficiency and cost factors in considering whether the WSCP is a good idea. I would focus more on whether it will promote achievement and whether it would increase parent satisfaction. 

If you have any thoughts on things I missed in this post, please use the feedback form or email me at nriemann@bellsouth.net. In my next and final post, I will look at whether there are any alternatives to the WSCP that might provide better results.

The Wake School Choice Plan

This post takes a first look at the Wake School Choice Plan (WSCP).

The WSCP is a student assignment proposal by the Wake Education Partnership and its consultant, Michael Alves. The WSCP offers the Wake County Board of Education a compromise approach that might achieve some goals of the new Board majority while preventing the proliferation of schools where many kids are not proficient.

According to the plan document, the WSCP is built on four “pillars”: stability, choice, proximity, and student achievement. If you are not familiar with the plan already, you can find an overview of the plan here (and its details here).

My discussion will assume that you have read the above-cited executive summary, if not the whole plan. For that reason, I will not summarize the WSCP in the post but will instead look at the following four issues of concern to me (and, I hope, to you):

(1) Would it promote student achievement?

(2) Would it increase parent satisfaction?

(3) Would it use schools efficiently?

(4) Are there any superior alternatives?

In this first post, I take up only the issue of student achievement.

Would the WSCP promote student achievement?

I can think of two ways an assignment system might promote student achievement. First, it might create better learning environments for students by limiting the number of low achieving students at any one school. Second, it might provoke schools to compete for students in ways that promote academic achievement. I look at the WSCP from each of these angles below.

Student Mix

I will assume here that—all else being equal—a high number of low achieving students in a given school makes it more difficult for that school to educate its students. The point can be, and has been, argued, but I will not revisit those arguments here.

The WSCP rejects assignment decisions based on race or socioeconomic status, but it does include “student achievement” as one of its four “pillars.” According to the Executive Summary, designing schools where 70% of the children are at or above grade level is a “good target,” but “[a] specific achievement quota is not educationally necessary or operationally practical.” Systemwide, 71.9% of WCPSS students passed both the 3-8 reading and math EOGs in 2009-10, so it would be ambitious to set a 2012 goal of 70% student body proficiency for each school if the Board intended to allow any variance in achievement at all.

It may not matter. Despite the plan’s talk of proficiency targets and ranges, the plan does not appear to implement any. Instead, the WSCP focuses on the identification of “at risk” kindergarten students and assumes that distributing these students “proportionally” across elementary schools will be sufficient to distribute low achieving students, since predictable feeder patterns will distribute these kids all the way through high school.

The success of this approach depends on at least three contestable assumptions: first, that kindergarteners labelled “at risk” are highly correlated with low achievers; second, that the “at risk” kindergarteners can be efficiently distributed across schools; and third, that each feeder pattern educates the “at risk” populations with similar rates of success. Otherwise, things will get out of whack rather quickly, achievement-wise.

Given the importance of these three assumptions, the WSCP should have addressed them more carefully.

With respect to the first, the WSCP avoids a political “hot potato” by declining to specify how the “at risk” kindergarten students should be identified, calling only for “an informed and research-based determination” as to the school readiness of each child. The plan implies—but does not state—that this determination should look at the parent’s education, whether the child is being raised by a single parent, whether the child is poor, whether the child has limited English proficiency, and whether the child has a learning disability. Unfortunately, the plan does not discuss the correlations between this laundry list of factors and student achievement. Nor does it propose any answers to the hard choices that must be made about how to use this data (or any other data) to determine who is “at risk.”

With respect to the second assumption mentioned above, the plan does not discuss how difficult it will be to allocate what it calls a “proportionate number of seats” for at risk students at each school in an assignment area, given the other priorities of the plan and the way its lottery system works. Will choice and the lottery be overridden for low achieving students, once a school exceeds its “proportionate number” of low achieving students? Will choice and the lottery be overridden for low achieving students when a high achieving school is not chosen by enough low achieving students? These are hard questions for which I found no answers in the plan document.

Further, while each of the three large achievement area has a relatively equivalent number of low achieving students, the typical assignment choice—ten elementary schools closest to any given address, including at least two magnets and two year-round schools—do not. This seems to imply that achievement distribution will work only if some are not offered a choice of their ten closest elementary schools. If that is going to happen, or must happen to make the plan work, it should be discussed more directly.

Instead, it is addressed obliquely. The plan’s executive summary notes, without explaining, that the plan’s achievement goal “means the district’s current magnet school program must be retained. The measure of a magnet school’s success, however, will change significantly. Wake County’s magnet schools were created primarily to fill seats and create diversity. This plan suggests magnet schools be used to increase parental choice and improve student achievement.” Despite the claim, I don’t think it is really changing all that much. I think the WSCP needs magnets for the same reason the current system needs them—to make sure that you can fully utilize certain school facilities without overloading them with low achieving students. Without magnet student set-asides in most of the current locations, it would be hard to limit the proportion of low achieving students who attend those schools.

As for the third and final assumption identified above, the plan does not discuss how stable these achievement distributions might be as the kids progress through different feeder patterns. Even if each pattern is equally effective, educationally, the outputs may vary due to differential access to other educational options. Some feeder patterns may see more good students (or poor students) opt out for private schools, charter schools, or magnets due to the location of these alternatives.

If a plan like this were to be implemented, the Board should consider whether it will be sufficient to focus on achievement distribution at kindergarten and then forget about it, as the WSCP’s actual mechanics seem to propose. As far as I can tell, the plan includes only one mechanism for recalibrating the achievement distribution after the initial kindergarten assignments: the atypical student who rejects her current assignment or her “feeder pattern” to middle or high school and enters the lottery would receive a priority if she promoted “achievement level-diversity in [her] first-choice school.” But this priority does not come into play until the school has accommodated all feeder pattern students, all sibling applicants, all applicants who live within a mile and a half, all applicants applying to their nearest school, all applicants applying to their nearest uncrowded school, and all applicants applying to their nearest nonmagnet school. It is hard to see how this would have much impact.

Because this plan incorporates achievement interventions, it would produce fewer low achieving schools than a strict neighborhood schools plan. Because the achievement interventions are minimal, it would likely produce more low achieving schools than we have had in the past. Still, it might better address the problem of low achieving schools in areas like Garner, East Wake, and the “Rim,” where low achieving schools are not adequately addressed by the magnet program. Rigorously applied, it could even produce fewer low achieving schools than we have now, but this would require much more aggressive achievement sorting than it appears to contemplate.

Competition for Students

The nature of the plan requires students to choose a school, so schools are, in a sense, competing for students. Will this competition encourage schools to raise achievement? The plan assumes so. It argues that “nothing makes a school more attractive than high student achievement.” For that reason, "schools that are consistently under-chosen by parents will be targeted for school improvement measures that would make these schools more academically attractive."

I am skeptical of the claim that “nothing makes a school more attractive than high student achievement.” I think most people—not all, but most—avoid high poverty schools, low achieving schools, and racially isolated schools. Otherwise, I think they choose among reasonably good schools based on factors like programs, convenience, student peers, staff personalities, and facilities. I asked Mr. Alves about this, but he did not really answer me—he just said that folks expressed lots of reasons for choice that did not differ by race. But the reasons for choice are important, because if I am right, the competition may not center on achievement.

Furthermore, the competition for students under this plan seems like the proverbial pie eating contest where the prize is more pie. If you bring in more students, you get a somewhat proportionate increase in teaching resources and a more crowded school. The school that failed to bring in students is the one that gets the program enhancements.

Even that possibility seems suspect. WCPSS is not a system with 30% excess capacity, like those found in many urban settings. It is a system that will soon be able to fully utilize every school whether it is underchosen or not. Program enhancements may not be necessary to fill underchosen schools. Further, where will the money to make program enhancements come from? The proposal includes no price tag for these enhancements, implying that the enhancements would be minimal, would come at the expense of existing programs, or would constitute an undiscussed cost of the WSCP approach.

For these reasons, I think it unlikely that the type of competition promoted by the WSCP will promote (or undermine) systemwide student achievement.

***

To wrap up, the achievement “pillar” of the WSCP raises more questions than it answers. I do not think the choice aspect of the plan, by itself, will do much for achievement, for reasons just mentioned. But a plan that gives achievement any attention at all would, by definition, produce fewer low achieving schools than the strict neighborhood schools plan advocated by many. Much turns on how the Board would implement the fourth pillar of the WSCP, and the plan intentionally or inadvertently leaves that very vague. I do not think it was a service to do so, and vigilance is called for.

If you have any thoughts about the WSCP and its student achievement “pillar,” or the ideas expressed in this post, please contact me via the feedback link or email me at nriemann@bellsouth.net. If there are enough interesting points raised, I will post some follow up discussion.

In my next post, I will look at the WSCP’s potential impact on parent satisfaction with assignment.

Achievement and the Concentration of Poverty

I have compiled at this link 2009-10 achievement data for entering kindergarteners, third graders, entering sixth graders, and entering ninth graders who would, if the plan had been in effect in 2009, have attended school in each zone of the now-defunct zone plan crafted by the Student Assignment Committee. This data shows, unsurprisingly, that the zones with the highest levels of poverty and racial isolation also have dramatically lower achievement levels.

While the zone plan is defunct, the figures remain relevant, because four members of the Board have indicated that their vision of "neighborhood schools" would produce the same result. In fact, proposals now on the table from representatives of those Board members would result in a more impoverished Southeast Raleigh / Enloe "zone" than the one earlier proposed, because Garner's representative has proposed to send away more poor students and take back fewer poor students than contemplated by the zone plan.

Tremendous resources are required to effectively build achievement when academic deficits have been concentrated in one place like this. In Charlotte, for example, some schools are spending approximately three times as much as others per capita. While disparities in per capita spending also exist here, they are not nearly as stark. Realigning our resources in a similar fashion, in an environment where the school budget is shrinking dramatically, would sharply reduce the resources available to lower poverty schools. Alternatively, the Board might not realign those resources, in which case the academic deficits of those in our high poverty schools are unlikely to be addressed successfully.

Debra Goldman Interviewed by Rick Martinez

I listened to this interview and thought it was interesting. For those lacking the patience, here's a summary:

Goldman said she is for “community-based schools.” She says she is for “proximity,” “stability,” “a real calendar option,” “some program choice through our magnets,” “no mandatory year round,” and—most importantly—a base assignment, such that when you choose where you live, you know where your neighborhood goes to school. She also wants a plan that has a positive impact on the achievement of ALL students (she capitalized ALL herself).

As for the process of developing a reassignment plan, she said the process should involve gathering input, gathering feedback, and being transparent. She didn’t like the process that was being used because it wasn’t fair. There weren’t enough voices.

She didn’t like the zone plan itself because it lacked the base assignments called for by revised Policy 6200 and was dividing communities when it was supposed to be about community-based schools. It was going to be a lottery, and "rings of proximity" didn’t address her concern because elementary schools are clustered and some would be in ring for multiple schools while others would not be in the ring for any.

Asked whether she’s gone over to the other side, she said something to the effect of, “That’s not entirely correct—my view is my view, and it’s the same as it ever was.”

She did not indulge several of Martinez’s efforts to engage her along party lines on issues like who really suffers funding inequities and the wisdom of sending poor kids away from Title I schools.