Summary of the Panel on Issues Related to Obtaining Impact from NSF Studies of STEM Education

Purpose

Some research projects have had particular success in becoming well known to the American public but also in influencing public policy and the world of instructional practices. The question before the panel was, is there anything we can learn from these projects that would be useful to NSF and to other projects that would facilitate good communication between such studies and the public?

Three general principles were established early in the discussion that in many ways are counter-intuitive to most academics. The first flies in the face of a common belief that if you do good work the rest will follow. This might occur in those rare circumstances where what has been studied is controversial and occurs when not much else is attracting public attention, but in most cases good communication of the results of research studies occurs because of hard work aimed at making these results relevant to non-academic audiences. The second principle states simply that academics are not trained to do this type of work and to be successful, professionals such as public relations firms should be engaged in the process of obtaining impact. To academics this might sound "gimicky" and superficial. After all, they reason, why would we have to resort to such tactics to publicize good work? The problem is that good work alone does not assure the type of broad-based impact that is desired. Also, the professionals have the contacts who are critical, especially in the world of television. The last principle is related to the second; researchers themselves must become seriously engaged in the process if it is to be effective. This is not the responsibility of others to which the researcher delegates the task. If you want to influence policy, practice or the public, you - the researcher - must be a central figure in the process. This, of course, implies a major commitment of time.

Studies of Dissemination

[Valente, 1993 #6; Valente, 1995 #3; Valente, 1998 #19; Van de Ven, 1988 #18; Argyris, 1985 #13; Dearing, 1994 #5; Mundry, 2000 #8; Loucks-Horsley, 1996 #11]

Potential Levels of Influence

Several potential levels of impact were identified. These include:

• the national policy level, including the President and his advisors, House and Senate members and their staff
• national professional associations; for example, the National Academy of Science, American Chemical Society and NCTM
• the business community and its associations such as the Business Round Table
• the general public
• the state and policy level including governors and their advisors, state legislators, state superintendents of education and their staff and state boards of education
• the local policy level including district superintendents and local school boards
• teachers and principals

Although it might be desirable in some general way to have the policy and practical implications of a research study impact all of the above identified levels, it is probably not likely that all studies can achieve such a goal. For example, a requirement for the national video and print media to pick up a story usually requires some controversial or alarming result that will generate interest publicly.

What the group felt a more attainable and probably a more reasonable goal for STEM studies is as the results of the research become available and their policy and/or practical implications become identified, that at that point reasonable levels of possible influence be chosen. Those levels would then be targeted for a special communication effort (Rogers et al, 1999; Ellsworth, 2000). Put simply, not all studies need necessarily strive to reach all levels. Such an approach would increase the focus of the communication efforts and as a result increase the likelihood of success and lessen the frustration of not engaging other levels for which there was little likely effect right from the outset.

Some Suggested Principles

Once an appropriate level of influence is chosen what are some of the principles that would maximize the success of these efforts?

• What it takes to reach the different levels varies. The same approach will not necessarily work across levels. For example, a good PR firm with media connections is critical for reaching the public, especially through the medium of television. Such opportunities do not just generally occur; they are created through personal contacts by professionals (Valente, 1993; Hutchinson & Huberman, 1993).
• A related principle is that when speaking, generating written documents or just talking at each of these different levels, different approaches are required. This might well require special training.
• The traditional academic way of talking and writing is not the most appropriate approach at any of these levels. In fact, for this kind of effort at influencing policy and practice, most "academic instincts" are wrong and if followed will often lead to a closing of the doors.
• Don't over complicate your message with highly qualified constraints and caveats; keep it simple, but yet in a way that you feel honestly represents what you have found.
• Eliminate educational and technical jargon. Use simple sentences to define a term if it is critical, but also technical in nature.
• Metaphors can be a powerful way to bridge the important technical findings with common experience and common language. These "sound bites" if viewed in isolation can appear to be shallow especially to the academic, but if they set up a more thorough understanding of the ideas and are easily remembered and engage people, they can be a powerful device.
• Your time as the principle investigator is critical to the success of such an effort. You cannot do what most academics are inclined to do - finish the research and then hand the task of communication to someone else, perhaps someone in the University Public Relations Office. You might get a story in the newspapers, but you will not likely have any lasting impact on policy or practice or the public more generally.
• The last principle is that it takes money to do this effectively, especially if you want to reach the national level and most particularly, if you want to impact the public through newspapers, radio, and television. This needs to be built into your research budget or special supplementary funds sought to support such an effort.

Possible Role for NSF

Much of what is described in the preceding sections requires special effort and skills not necessarily possessed by academic researchers. One role NSF could assume is to provide special training seminars for principle investigators in conjunction with PI meetings or as separate seminars. These would need to be run by professionals such as those in a communications or public relations firm. These sessions could focus on speaking to the press, role acting TV or radio interviews and writing press releases - aspects of effective communication.

The second role for NSF would be to provide a special fund to support such efforts. The recommendation would be to have PI's write a short proposal for such supplementary funds after their results are known and the general thrust of the policy implications known. With this knowledge an application could be made specific to the level of impact desired. This would make clear the amount of funds needed and would also enable NSF to make judgements as to whether they judge the possibility of impact great enough to warrant special supplementary funding. In other words, communication concerning research and its utilization have long been recognized as distinct from the research endeavor itself (Rogers, 1995; Hutchinson & Huberman, 1993; Kaser, Bourexis, Loucks-Horsely, & Raizen, 1999).

References

Ellsworth, J. B. (2000). Surviving Change: A Survey of Educational Change Models. Syracuse University, Syracuse, NY: ERIC Clearinghouse on Information & Technology.

Hutchinson, J., & Huberman, M. (1993). Knowledge Dissemination and Use in Science and Mathematics Education: A Literature Review (NSF Publication No. 9375). Washington, DC: National Science Foundation.

Kaser, J. S., Bourexis, P. S., Loucks-Horsley, S., & Raizen, S. A. (1999). Enhancing Program Quality in Science and Mathematics. Thousand Oaks, CA: Corwin Press.

Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.

Rogers, E. M., Hall, B., Hashimoto, M., Steffensen, M., Speakman, K. L., & Timko, M. K. (1999). Technology Transfer from University-Based Research Centers: The University of New Mexico Experience. Journal of Higher Education, 70(6), 687-705.

Dissemination Study References:

Argyris, C. (1985). Strategy, Change and Defensive Routines. Boston: Pitman.

Dearing, J. W., Meyer, G., & Kazmierczak, J. (1994). Portraying the New: Communication between University Innovators and Potential Users. Science Communication, 16(1), 11-42.

Loucks-Horsley, S., & et al. (1996). Principles of Effective Professional Development for Mathematics and Science Education: A Synthesis of Standards. NISE Brief, 1(1).

Mundry, S., Britton, E., Raizen, S., & Loucks Horsley, S. (2000). Designing Successful Professional Meetings and Conferences in Education: Planning, Implementation, and Evaluation. Thousand Oaks, CA: Corwin Press.

Valente, T. W. (1993). Diffusion of Innovations and Policy Decision-Making. Journal of Communication, 43(1), 30-45.

Valente, T. W., Paredes, P., & Poppe, P. R. (1998). Matching the message to the process: The relative ordering of knowledge, attitudes, and practices in behavior change research. Human Communication Research, 24(3), 366-385.

Valente, T. W., & Rogers, E. M. (1995). The Origins and Development of the Diffusion of Innovations Paradigm as an Example of Scientific Growth. Science Communication, 16(3), 242-273.

Van de Ven, A. H., & Rogers, E. M. (1988). Innovations and Organizations: Critical Perspectives. Communication Research, 15(5), 632-651.