Flawed Research: Vouchers and Florida Schools

REPORT OVERSTATES CLAIMS FOR FLORIDA’S A+ ACCOUNTABILITY SYSTEM

Reviewer says report presents strong analysis of Florida’s system, but wrongly implies vouchers can be singled out as a cause of school improvement

EAST LANSING, Mich. (Jan. 15, 2008) – A new study from the Urban Institute’s CALDER center examines whether Florida’s system of sanctions and incentives for its poorest performing schools led to improved student achievement. The report concludes that the system, including the threat of vouchers, appears to have spurred schools to improve practices and thereby improve performance. A new review of the study praises its comprehensive analysis of data, but identifies several key instances where the report overstates its case.

The CALDER report,“Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure,” was authored by Cecilia E. Rouse, Jane Hannaway, Dan Goldhaber, and David Figlio.

The report was reviewed for the Think Twice project by Damian Betebenner of the National Center for the Improvement of Educational Assessment.

In their report, the authors seek to understand the extent to which accountability sanctions and incentives impact school practices and student achievement. Based on statistical analysis of the lowest performing schools, the authors conclude that, in Florida, schools receiving a grade of “F” in the state’s A+ Plan for Education actually did alter their teaching practices, leading to higher test scores.

Though Betebenner commends the report for its thorough analysis of the data, he observes that the most prominent shortcoming of the report is its overstatement regarding the relationship between the pressure of accountability sanctions and improvements in school achievement. He notes that the report’s title and some of the statements in the body of the report suggest that vouchers and other accountability measures are the cause of the achievement gains though there is no supporting evidence that such a causal connection exists.

Betebenner adds, “Moreover, even if it is true that the Florida policy of vouchers plus other accountability provisions did lead to the changes in policy and practice, nothing in this new research allows a policy maker to single out either vouchers or other accountability provisions (or a combination) as having such an effect.”

Betebenner concludes that the Urban Institute report does offer reason to believe that one or more elements of Florida’s accountability systems “may offer … a lever” to improve student achievement. But he continues, “because changes in school policy and practice can occur for many reasons, this research should not be read to show that the accountability system ‘led to’ or ‘caused’ the student achievement increases. Nor does the new report consider whether the accountability levers in Florida are the most effective means of quickly and beneficially transforming the policies and practices of schools in a way that leads to increased student achievement.”

Find the complete review by Damian Betebenner.