Flawed Research:Reason Report

Reason Report Found to be Reckless and Irresponsible

New review concludes report is “a major step backwards”

EAST LANSING, MI (May 13, 2009)—Two weeks ago, the Reason Foundation released a report titled Weighted Student Formula Yearbook 2009, which advocates for a package of reforms concerning funding, governance and school choice. A new review of that report finds that it cherry-picks evidence, lumps many different strategies under a single reform umbrella, ignores contradictory findings, and in one third of its examples credits the reforms for outcomes that actually preceded the reforms.

The Yearbook was reviewed for the Think Twice think tank review project by Bruce Baker, a school finance expert who is an associate professor at Rutgers University.

Drawing from 15 case studies, the Yearbook relies on two underlying premises: (1) budgets should be allocated directly to schools within a district, with the amount based on each child’s needs; and (2) school principals should have full discretion on how to allocate those funds. The report examines 14 city school systems and one statewide one—Hawaii—that the report presents as reflecting “best practices” in implementing what the report calls Weighted Student Funding (WSF) reforms.

For many, the principle underlying WSF is appealing and common sense. The strategy is arguably intended to ensure that education funding adequately and fairly reflects the needs of students. In practice, however, the strategy has been found to be complex and its results much more ambiguous—and very much dependant on how it is implemented.

Baker finds that the Yearbook ignores all these complexities. Instead, the report mixes the basic WSF funding reform with other reforms ranging from site-based management and budgeting to school choice programs, including pilot, magnet and charter schools. In Baker’s words, the report “selects a hodge-podge of district reform strategies.” Some of those directly employ WSF but others “have little to do” with the funding strategy itself, or with district-wide reforms, Baker observes.

Baker’s review also finds that the Yearbook neglects “large bodies of relevant literature” and ignores “disagreeable findings in the literature it does cite.”

However, according to Baker, the most egregious flaw in the Yearbook is that in one-third of the examples it cites—five of the 15 case studies— “outcome successes mentioned actually occurred prior to the implementation” of the touted reforms. This is illustrated by the Reason press release promoting Yearbook, which points to impressive 2007-08 test score gains in Hartford, Conn., and attributes the gains to a change in policy directing 70% of resources to the classroom. Yet as the report itself notes, that WSF policy only began a year later, in 2008-09. Baker notes that it is difficult to conceive of a defense for such a claim.

“The report haphazardly aggregates a multitude of discrete policy issues under an umbrella labeled as WSF and deceptively suggests that all related policies are necessarily good—even going so far as to credit those policies for improvements that took place before the policies were implemented,” Baker writes. “The report then irresponsibly recommends untested, cherry-picked policy elements, some of which may substantially undermine equity for children in the highest-need schools within major urban districts.”

Instead of adding any serious information to the body of knowledge on WSF, Baker concludes, Reason’s Yearbook is “a major step backwards.”

Find Bruce Baker’s review on the web at: http://www.greatlakescenter.org.