Milwaukee and Racine DPI Report Card scores and poverty – and about those “growth scores”

In the weeks since Forward Institute released our Wisconsin Report Card Study 2012, charter school advocates and school privatization advocates have pointed to Milwaukee and Racine as examples of charter schools outperforming traditional public schools. The data for both districts show the overwhelming impact of poverty on Report Card scores. The need for public policy that effectively addresses poverty and education policies together should be the highest priority. The data also consistently demonstrates the inherent problems with tying school funding and teacher evaluation to current standard measures of educational outcome.

Figure 1 Mil-Racine graphs

Figure 2 Mil-Racine graphs

As demonstrated in the first two graphs (figure 1 is Milwaukee schools only, figure 2 is statewide schools), Milwaukee and statewide school districts show a significant correlation between the level of Economic Disadvantage and Report Card scores. The higher the level of poverty, the lower the Report Card scores. The plot also shows charter schools at the lowest income levels having lower scores than their public school counterparts – consistent with the statewide data. There is a difference in the data, however – one not addressed by charter school advocates.

In the statewide data, charter schools have a significantly higher percentage of low-income enrollment than public schools (43.6% Charters, 32.7% Public). In Milwaukee, public schools have a greater percentage of low-income enrollment than charter schools (88.5% Charters, 95% Public). (Low income is defined in this study as schools with ED enrollment higher than 48.9%. Middle income is ED enrollment of 30.4% to less than 48.9%. High income is ED enrollment less than 30.4%).

Based on the statewide outcome, we would have expected Milwaukee charter schools to perform better on the Report Cards in the lowest income group than public schools – having a lower percentage of high poverty schools. That is not the case. Figure 3 shows that in the middle and low-income groups, charter schools scored lower than public schools in Milwaukee.

Figure 3 Mil-Racine graphs

At the very least, based on the standard deviation, charter schools scored no better than public schools in Milwaukee. This would suggest that in spite of MPS traditional public schools having more schools with high ED enrollment than charter schools, they are still scoring no worse than their non-traditional charter school counterparts.

A histogram in figure 4 shows the Report Card score distribution for Milwaukee public and charter schools. The distribution is expressed as a percentage of the total to compensate for a larger number of public schools. This graph clearly illustrates that a greater percentage of charter schools had lower Report Card scores than public schools in Milwaukee.

Figure 4 Mil-Racine graphs

In the Racine Unified School District, there is insufficient data on charter schools to effectively draw a comparison between public and non-traditional charter schools.There is also currently no performance data on private voucher schools to draw any valid analysis. It is possible, however, to consider RUSD schools in comparison to the statewide data. Figure 5 shows the overall effect of poverty on Report Card scores is similar to the statewide and Milwaukee models.

Figure 5 Mil-Racine graphs

In the RUSD, 80.6% of all schools have low-income enrollment (ED enrollment >48.9%). Again, this is significantly higher than the statewide low-income enrollment (43.6% Charters, 32.7% Public), yet lower than Milwaukee schools (88.5% Charters, 95% Public). Figure 6 compares the scores stratified by Milwaukee, Racine, and statewide scores, charters and public schools. No MPS schools fall into the “high income” category. (Low income is defined in this study as schools with ED enrollment higher than 48.9%. Middle income is ED enrollment of 30.4% to less than 48.9%. High income is ED enrollment less than 30.4%).

Figure 6 Mil-Racine graphs

The high and middle-income groups show no significant difference in the Report Card scores, consistent with the statewide analysis, although the Milwaukee Charters middle-income group is close to being significantly lower than the Milwaukee public middle-income group.

Of greatest significance is the Racine low-income scores. RUSD low-income schools scored statistically equal to the statewide charter schools score – and higher than the Milwaukee Charter schools of low-income.

The data does not support the claim that Milwaukee Charter schools outperform traditional public schools. At the very least, the difference is not statistically significant. At the most, the mean Report Card scores indicate that Milwaukee Public Schools are outperforming their Charter School counterparts – particularly in the schools of highest poverty. In Racine, the highest poverty RUSD schools are performing on a par with statewide Charter Schools, and only slightly lower than statewide Public Schools.

While we all want better outcome for children of poverty, the data show that continued calls for expansion of non-traditional charter schools and private voucher programs (which have no accountability data to analyze) are nothing more than partisan politics; having demonstrated no evidence of improving educational outcome, particularly for children of poverty.

The Milwaukee and Racine data confirm a consistent link with poverty and educational outcome; and that in the highest areas of poverty, public schools are doing a better job educating children in the most challenging situations. 

A word about “Growth Scores”

There has been a significant push to compare public and charter schools based on the “Growth Score” portion of the DPI Report Card scores, as well as other school-to-school comparisons based on growth scores. DPI assessment experts have pointed out that this is an invalid comparison. Growth scores measure growth within a given school, based on measures within that school. For that reason, each school has a different baseline from previous years, making growth score comparison invalid. An illustration…

A majority of the “Growth Score” index relies on students moving toward proficient or advanced in reading and math tests, from previous years’ performance in that school. The mean growth score statewide is 61.4 for public schools, 62.4 for charter schools. As Milwaukee schools have been the focus in this study and the media, figure 7 shows the significant difference in starting points for public, charter schools, and test subject.

Figure 7 Mil-Racine graphs

Notice that charter school students in Milwaukee had a significantly higher percentage of proficient math and reading scores at the start of the measurement in 2008. With the exception of Charters Reading proficiency declining from 2008, the other three groups demonstrate a pattern of modest improvement with decline beginning in 2010 Public Math scores, and all scores declining from 2011-2012.

Further evidence of school-to-school differences can be seen at the highest and lowest growth score schools in Table 1 (RC score = Report Card score, ED enroll = Economic Disadvantage enrollment %, Reading and Math numbers expressed as percent of students scoring proficient):

Table 1 Mil-Racine

First, note the difference in starting points for each school. In both the top and lowest growth schools, charter schools have lower ED enrollment and higher test scores. While the highest growth score for a public school showed an improvement from 2008-2012 (much of it coming in 2011-2012), the other three schools have shown decline or no improvement. Much of the decline has come in the last year, 2011-2012.

While the growth score is useful in looking at growth within a school, it is certainly not an acceptable measure in comparing different schools. Moreover, the growth scores are telling us more about poverty and educational outcome…but that is for another time.

Scott Wittkopf, Chair

Forward Institute

2 thoughts on “Milwaukee and Racine DPI Report Card scores and poverty – and about those “growth scores”

    • If you look at the scatterplot in Fig 1, there are so many non-linear points of data suggesting, I think, that there are charter schools scoring poorly even at lower levels of ED – confounding the expected distribution. This suggests that there are other factors affecting low scores beyond those existing in public schools…teacher experience??

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s