Scientific Principles for Education Research:
What Counts as Evidence?

Debates about the nature of knowledge in education have characterized the field of education research since its inception. Fundamental questions about epistemology, methodology, and purpose have evolved significantly but remain thorny within research circles. In the meantime, enthusiasm for evidence-based education as a key driver of promoting school reform has risen dramatically in recent years, engaging policymakers and practitioners more fully around these crucial and complex issues. This trend has its roots in the standards and accountability movement that introduced new incentives for educators and administrators at all levels to seek out research "proven" strategies for raising student achievement for all. Emphasis on research-based programs began creeping into federal education law in the mid-1990s with the Comprehensive School Reform Demonstration Act and the Reading Excellence Act. Then came the sweeping reauthorization of the 1965 Elementary and Secondary Education Act-the No Child Left Behind Act-passed earlier this year with 111 references to "scientifically based research."

Somewhat ironically, the demands for education research that can improve policy and practice have come hand in hand with criticism of the existing knowledge base and the capacity of current investigators to develop it over time. Policymakers' skepticism most visibly manifested in a bill to reauthorize the U.S. Office of Educational Research and Improvement (OERI) that included legislatively determined definitions of what constitutes accepted research methodology (see http://edworkforce.house.gov/press/press107/oeri43002.htm for a press release regarding House passage of the bill and http://edworkforce.house.gov/markups/107th/edr/hr3801/313main.htm for the full text of the bill and amendments, including definitions regarding research quality).

This bill, coupled with a long history of debate that preceded it, led the U.S. National Educational Research Policy and Priorities Board (NERPPB, the policy arm of OERI) to ask the National Academy of Sciences/National Research Council to convene a committee of experts to articulate the nature of scientific research in education. The committee represented a diverse cross-section of disciplinary and content expertise, including psychology, sociology, mathematics education, reading, cultural anthropology, statistics, economics, cell biology, chemistry, demography, philosophy (of science), and history. The group had to confront tough questions about how education research is similar to, and different from, other fields while taking on the specific complex and politically charged issues associated with the field (e.g., are randomized field trials the gold standard?). The final publication can be viewed for free online at http://www.nap.edu/catalog/10236.html. Hard copies can be purchased at the same address.

NSF principal investigators are no strangers to these issues, but the era of evidence-based education involves an unmistakable, and essentially new, political dimension. The purpose of this session is to engage in familiar discussions about evidence, inference, and inquiry in education by explicitly exploring the issues in this modern environment. Three clusters of discussion questions include:

1. The Big Picture. What are the opportunities and challenges for the field associated with this rising political interest? Does the education research community have a responsibility to engage directly with the new push for "turning education into an evidence-based field" (see the U.S. Department of Education's Strategic Plan, Goal 4 at http://www.ed.gov/pubs/stratplan2002-07/index.html)? If so, how, particularly given the fact that the research community itself is quite diverse? If not, why not?
2. Lessons in Linguistics. Some of the language and jargon of science has been transformed into political watchwords (in education and more broadly). What do concepts like evidence, empirical, experimental, rigorous, and scientific mean to the research communities? What do they mean to the political communities? What are the tensions that arise as a result of any disconnects?
3. Method Matters. Most of the political interest in promoting high quality research centers on methodology, oftentimes equating particular methods (random assignment designs in particular) with "quality" or "scientific." How does the Administration's push for randomized field trials (again, see the U.S. Department of Education's Strategic Plan, Goal 4 at http://www.ed.gov/pubs/stratplan2002-07/index.html) square with the broader principle of choosing and implementing methods as appropriate to particular questions? What are the strengths and weaknesses of this and other methods (e.g., design experiments) in supporting knowledge claims of various sorts? What impact will this have on the field and on the future knowledge base?