September 2019 science journal article summary

Greetings,

Scientists often have intended uses for their research in mind. Sometimes it works, sometimes it gets ignored, and other times it's used in unexpected ways (like liquid nitrogen being used to make caipirinha sorbet, above).

This month I finally focused on a single topic: how scientists may be able to improve the impact of their research. I'm also working on revising a paper with recommendations along these lines - let me know if you'd like to see the draft.

If you know someone who wants to sign up to receive these summaries, they can do so at http://bit.ly/sciencejon


RESEARCH IMPACT:
Cairney & Oliver 2018 summarizes 86 publications with recommendations on how scientists may be able to improve their impact (e.g. do high quality research, make it relevant and readable, understand the decision space, build relationships, etc.). They find that the advice is mostly consistent, albeit vague, but they reject that it is either practical or that it will be helpful. They argue that instead the policy theory literature can help more (e.g. highlighting the important of investing in relationships over the long term), and that scientists should also be aware that attempts to increase impact will typically not pay off. They also note that there can be reputational risks in attempting to do so (especially for women and people of color) and that there's inequality in which scientists are in a position to even make the attempt. At the same time, I think they paint a false dichotomy between research that clearly leads to its intended impact, and research that does not. In practice, these steps can likely increase the chance of impact (whether the original intended use or not), and research that ignores all of these recommendations is less likely to be discovered and used.

Pohl et al. 2017 is a cool but unusual science paper. The authors provide clear instructions in 10 steps for researchers to improve their impact (similar to the concepts in Enquist et al. 2017 but aimed at implementation). Table 1 has a great summary of the process - at a high level they recommend matching research questions to knowledge needed to inform action, thinking about who to involve (e.g. stakeholders) throughout the research process, and reflecting on lessons learned. The authors have walked a variety of researchers through these 10 steps in a single day. Steps 5-9 provide helpful tips on how to identify a body of stakeholders, and figure out how to break them down into who to co-produce knowledge with, who to consult with, and who to simply inform. It seems like a great framework to get scientists started, although it's a bit ironic to have scientists think on their own about how to better incorporate input and perspectives from stakeholders.

Jacobs et al. 2005 offers several recommendations for producing science relevant to decision making. They include understanding the decision making context & perspectives of end users of the information produced, building relationships, making the research available and understandable, and providing results that are relevant to potential decisions given constraints (deadlines, resources, scale of action, etc.). They also highlight: the challenges of ensuring equitable outcomes, the importance of 'science integrators & translators' (boundary spanners) to bridge the gap between scientists and others, potential measures of success for collaborations w/ stakeholders, and that all this takes lots of time and is hard to do.

Beier et al. 2017 makes 10 recommendations for actionable science to be co-produced by scientists, decision makers, and others. They include: decision makers should convey their need / problem to scientists (not ask for a product), scientists should understand the decision context before suggesting scientific products, have all partners & stakeholders meet in person, have a small technical advisory group and a steering committee for big complex projects, iteratively discuss assumptions / approaches / etc., decision makers should explain to scientists how they evaluate and manage risk and uncertainty, scientists should honestly convey implications of their research along w/ uncertainty and appropriate use, evaluate the coproduction process itself and share the findings, invest in boundary organizations dedicated to coproduction, and create incentives for academic scientists to engage in coproduction.

Bednarek et al. 2018 defines boundary-spanning (as connecting production and use of knowledge), why it matters, and how to do better at it. They argue that boundary spanner experts (potentially full-time) can improve research impact by serving as honest brokers and facilitating good research design and knowledge co-production. They emphasize that this is about ongoing relationships rather than a 1-way comms 'push.' Table 1 has a list of boundary-spanning orgs, and they give useful details of what boundary-spanners can do. They call for formal boundary-spanning positions, trainings that emphasize the skills needed, and having measures of successful boundary-spanning activities.

Dunn & Laing 2017 interviewed 72 Australian policy makers focused on water managements to ask what aspects of research were most likely to lead to influencing policy outcomes. They didn't prompt them on specific frameworks but summarized open-ended responses. They found the most important aspects were applicability (not only relevant, but solves the right problem w/ the right methods comprehensiveness, timing and accessibility), comprehensiveness (interdisciplinary, applicable to the whole life cycle of a policy process, and including the economic impact of policy), timing (agile enough to meet policy maker deadlines and work fast when opportunity windows open, and willingness to share results early), and accessibility (the audience should be readily able to access and understand the research, meaning it should be short and practical and make clear recommendations). They suggest ACTA as an acronym to capture these four aspects of useful research.

Enquist et al. 2017 is an overview of 'translational ecology' which they define as integrating ecological knowledge with decision making. Similar to calls for transdisciplinary research, the idea is for researchers to work with decision makers and stakeholders throughout the process and focus on real-world outcomes. They lay out 6 key principles (collaboration, engagement, commitment, communication, process, and decision-framing) and give examples of each in Panel 2. Panel 1 has a useful summary of relevant terms / jargon which can be confusing to folks new to this topic.

Wall et al. 2017 is another overview of translational ecology.  They focus heavily on the need for scientists to engage in building relationships and trust with decision makers and other stakeholders.

Ruhl et al. 2019 asked  which kinds of scientific papers are the most relevant to policy (clearly articulating a policy proposal, policy actors, and actions to implement it). They limited it to 220 papers published in Policy Forums in Science magazine in the last five years. The most interesting finding is that paper with the most policy relevance cited the most other papers but were cited the least often, indicating that papers aimed at decision makers may be of less interest to research scientists (Fig 3). See Fig 1 for how different fields rated in policy relevance (e.g. atmospheric & hydrospheric science had the highest rate of medium and high policy relevance, general interest articles had the lowest).

Salafsky et al. 2019 is a guide for conservation practitioners to define, generate, and use evidence. They offer a typology of different kinds of evidence (and different contexts where each may be most appropriate, see Tables 2 and S1), plus a decision tree to help choose how to use evidence in a given context (Figure 2). They close with a call to incorporate thoughtful use of evidence into conservation practice, learning from disciplines like medicine which have been doing so for longer.

REFERENCES:
Bednarek, A. T., Wyborn, C., Cvitanovic, C., Meyer, R., Colvin, R. M., Addison, P. F. E., … Leith, P. (2018). Boundary spanning at the science–policy interface: the practitioners’ perspectives. Sustainability Science, 13(4), 1175–1183. https://doi.org/10.1007/s11625-018-0550-9

Beier, P., Hansen, L. J., Helbrecht, L., & Behar, D. (2017). A How-to Guide for Coproduction of Actionable Science. Conservation Letters, 10(3), 288–296. https://doi.org/10.1111/conl.12300

Cairney, P., & Oliver, K. (2018). How Should Academics Engage in Policymaking to Achieve Impact? Political Studies Review. https://doi.org/10.1177/1478929918807714

Cameron, D. R., Marvin, D. C., Remucal, J. M., & Passero, M. C. (2017). Ecosystem management and land conservation can substantially contribute to California’s climate mitigation goals. Proceedings of the National Academy of Sciences, 201707811. https://doi.org/10.1073/pnas.1707811114

Dunn, G., & Laing, M. (2017). Policy-makers perspectives on credibility, relevance and legitimacy (CRELE). Environmental Science and Policy, 76(February), 146–152. https://doi.org/10.1016/j.envsci.2017.07.005

Enquist, C. A., Jackson, S. T., Garfin, G. M., Davis, F. W., Gerber, L. R., Littell, J. A., … Shaw, M. R. (2017). Foundations of translational ecology. Frontiers in Ecology and the Environment, 15(10), 541–550. https://doi.org/10.1002/fee.1733

Jacobs, K., Garfin, G., & Lenart, M. (2005). More than Just Talk: Connecting Science and Decisionmaking. Environment: Science and Policy for Sustainable Development, 47(9), 6–21. https://doi.org/10.3200/ENVT.47.9.6-21

Pohl, C., Krütli, P., & Stauffacher, M. (2017). Ten reflective steps for rendering research societally relevant. GAIA, 26(1), 43–51. https://doi.org/10.14512/gaia.26.1.10

Realmonte, G., Drouet, L., Gambhir, A., Glynn, J., Hawkes, A., Köberle, A. C., & Tavoni, M. (2019). An inter-model assessment of the role of direct air capture in deep mitigation pathways. Nature Communications, 10(1), 3277. https://doi.org/10.1038/s41467-019-10842-5

Roque, B. M., Salwen, J. K., Kinley, R., & Kebreab, E. (2019). Inclusion of Asparagopsis armata in lactating dairy cows’ diet reduces enteric methane emission by over 50 percent. Journal of Cleaner Production, 234, 132–138. https://doi.org/10.1016/j.jclepro.2019.06.193

Ruhl, J. B., Posner, S. M., & Ricketts, T. H. (2019). Engaging policy in science writing: Patterns and strategies. PLOS ONE, 14(8), e0220497. https://doi.org/10.1371/journal.pone.0220497

Salafsky, N., Boshoven, J., Burivalova, Z., Dubois, N. S., Gomez, A., Johnson, A., … Wordley, C. F. R. (2019). Defining and using evidence in conservation practice. Conservation Science and Practice, 1(5), e27. https://doi.org/10.1111/csp2.27

Wall, T. U., McNie, E., & Garfin, G. M. (2017). Use-inspired science: making science usable by and useful to decision makers. Frontiers in Ecology and the Environment, 15(10), 551–559. https://doi.org/10.1002/fee.1735


Sincerely,

Jon

p.s. If you'd like to keep track of what I write as well as what I read, I always link to both my informal blog posts and my formal publications (plus these summaries) at http://sciencejon.blogspot.com/