Dissemination and the Integration of Research into Practice

This will be an interactive session whose goal is to gain clarity about the different types of research questions that arise when considering the integration of research into practice. These research questions will drive in turn the methodologies used in the research, and thus the session may help clarify for the audience and for NSF how to look at valid methodologies in an ecology of education research. Reading material: http://www.aera.net/gov/rpn/n-06.htm and links therein

A fundamental problem in the principled reform of education is the lack of direct connection between the general nature of research results and the highly localized nature of education practice. Fields such as medicine and agriculture have developed strategies for coping with the putative disconnect between generalized research and localized practice, but these strategies do not transfer to education without adaptation, evolution, and testing. It is the large pool of research-savvy non-researchers, and the expenditures on research by the private sector, which ensure transfer to practice in technology writ large. These elements are missing in education.

By analyzing how to effectively promote the integrated co-development of scholarship, practice, and policy we may be able to articulate the types of studies that populate an ecology of education research.

There are several reasons why this is a useful analysis at this point in time. One led to the recent NRC report on "Scientific Research in Education", and arises from attempts to define the field from the outside. Related to the need to define education research, but now from within it, is the strong interest to define a "scholarship of teaching" which differs from "action research". Additionally, there are many, widely differing attempts at justifying new and/or non-traditional methodologies as part of "education research". These methodologies run the gamut from experimental designs to design experiments to narratives as research to reflections on one's own practice.

In this context, rather than argue for or against particular methodologies, it is productive to understand what are the questions that will help integrate education research and practice. Linear models of transfer from research to practice, be it via artifacts or via publications, are not sufficient to sustain a research-based transformation of education. In this model the researchers, the object of study, and the subjects and users are all seen as distinct entities. The linear model implies that we can establish artifacts (materials and rules) useful to all, and does not in practice acknowledge how education improvement is a local process, heavily dependent on local conditions, and thus unique in the view of those who must implement it. Sequencing reform activities—separating the closely coupled ones from others more loosely coupled, and the interaction between researchers, users and the objects of study—which could be expected from a linear approach, is missing from much of the rhetoric associated with dissemination.

The nature of the research conducted on schools leads to research projects as the unit of knowledge acquisition, not schools or district-wide programs, as would be the case if schools or districts did the research. Questions posed at these different units of analysis are quite different. Leaving the integration across projects and units of analysis to inform the multiple needs of a given school or district to be conducted by the implementers does not work. Implementers do not usually have the time, inclination, knowledge needed to do more than pick and choose.

The resulting unorganized and fragmented nature of the education research knowledge base also makes it very hard for policymakers at every level to design and evaluate promising approaches to successful and sustainable reform in a reasonable large scale. We distinguish here engineering research (how to) from survey/evaluation research (what exists) and from feasibility research (what could be). "How to" research must be based on understanding "what exists" locally and on the comparative analysis of multiple "what could be" options.

Learning across different research projects is by definition a task that cannot be the responsibility of a single project. In general, and in areas where the problems of practice are not so complex and dependent upon local conditions, periodic reviews of the literature attempt the aggregation and comparison of the knowledge generated by different research projects. In studies of education reform, this process is too slow with respect to the need for action, and it is not very effective, given the central role of local conditions.

For the most part, the ways in which research has influenced or transformed practice are hidden from public view and have been subject to multiple interpretations heavily influenced by what local stakeholders know and how they think. There are no organizational structures that perform the learning engineering research and that establish protocols that move multiple research results into a limited set of lessons oriented, not to how practice "ought to be", but to how schools, districts and states can get there starting from where they happen to be.

This need—for a system (or ecology) of education research, not simply an aggregate of project results—presents both a problem and a major opportunity for researchers. How shall we understand the challenge of "building an effective local implementation of a successful educational system based on research"? Prior to achieving the allocation of a greater proportion of educational expenditures to research, as well as increasing the research preparation of practitioners, we need to understand what is meant-by us and by others-as "valid" education research that informs that actions of practitioners and policymakers.

Education Studies as an Ecology of Research Problems and Methodologies

To enable conducting the coherent implementation studies suggested earlier as a crucial strategy for educational improvement, the scholarly community must become more reflective and self-critical about current processes and goals of educational research. Analogous to research in engineering and the social sciences, educational studies involve developing knowledge about designed, human contexts less constant in their attributes than natural phenomena. The situations studied by educational researchers can be seen as complex systems with sophisticated feedback and non-linear causality, similar to biological or ecological systems. Such systems cannot be understood by considering pieces of the whole, and can therefore benefit from integrated system research strategies. Beyond independent scholarship, educational researchers also should play a role as intermediaries who enable experts in other disciplines, educational practitioners, learners, funders, and policy makers to understand each other's views of these complex-system perspectives.

As the field of education changes, the types of research requested and needed alter. In the 1980s, societal concern about educational outcomes led to a variety of descriptive studies designed to assess and understand problems in performance. At the start of the millennium, now that the causes underlying educational dysfunctions are better understood, practitioners and policy makers are asking researchers to focus on applied larger studies that improve practice in a sustainable, affordable, and scaleable manner. But this demand in itself, without a parallel recognition of what resources such long-term studies require, will not provide answers. Requests from the field for research results that inform practice have thus far resulted primarily in evaluative studies that provide limited evidence on whether current educational interventions are worth the cost and trouble involved in implementation. We can argue for the communal creation of ways of designing and reporting "use-driven" research and the conditions under which it is conducted, in ways that can help the community as a whole see the forest and not be blinded by the trees.

To shed light on the goal of this session, in what follows we take the point of view of the field of education, as contrasted with the point of view of a research project by itself.

One possible conjecture is that looking at research from the point of view of its eventual (aggregated) use in practice will help us understand replicability, adaptability, and validation in a large scale. A different analysis could be based on research that informs how we think epistemologically about practice; on what the Carnegie Foundation calls "scholarship of practice", and so on. Without clarity about what we mean by a certain type of research, we can have no clarity about the best methodology that addresses that goal.

Table I should be seen as a basis for conversations about the forward-looking part of a use-driven feedback loop between research and practice. Its only purpose is to start conversations and, by filling the columns and adding to the rows, attempt to differentiate between types of classroom studies, the research questions that could drive them and the appropriate research methodology. It is an attempt at an ecology of studies that bridge traditional research and practice. The network of studies we need should document the complete process, including failures, which are in fact only explorations critical to the field, if perhaps harmful to the project.

TABLE I: Different Styles of Classroom Research on Innovation and Practice and implications for partnerships

# A site that will conduct tests of the research, jointly with the research team. May have to share the learning with others (via visits, workshops, presentations, etc.)