Peer Review and the Ex Ante Assessment of Societal Impacts Page: 10
The following text was automatically extracted from the image on this page using optical character recognition software:
preserved their autonomy by claiming a special ability to judge that has been institutionalized in the
disciplinary nature of peer review. If accountability demands that others have a voice in the decision-
making process, then peer review becomes interdisciplinary and transdisciplinary as well as disciplinary
in nature. Who, then, ought to count as a peer?
Scientists ought to play a central role in determining what research gets funded. But this will only
continue to be possible if scientists also embrace the fact that their research can be judged on its
potential societal impacts as well as its intrinsic intellectual merit. And unless scientists embrace their
own ability to judge impacts, their role in the decision-making process will increasingly be transferred to
The authors gratefully acknowledge the support for this research provided by the US National Science
Foundation under Grant No. 0830387. The authors also wish to express their appreciation to NSF and EC
officials who contributed to this research, while emphasizing that any opinions, conclusions, and
recommendations expressed here are those of the authors and do not necessarily reflect the views of
NSF or the EC. The authors also thank Brunel's Health Economics Research Group (HERG) for hosting the
international two-day workshop on 'State of the Art in Assessing Research Impact' and for including our
presentation in the workshop. Comments received during the workshop were very helpful in
formulating this paper. Special thanks are due Claire Donovan for inviting us to participate in the
workshop and to Martin Buxton for his insightful criticisms. Finally, the authors thank the anonymous
reviewers for their helpful comments.
1 The six agencies are: the US National Science Foundation, National Institutes of Health, and National Oceanic and
Atmospheric Administration; the European Commission 7th Framework Programme, and the Dutch Technology
Foundation; and the Canadian Natural Sciences and Engineering Research Council. More can be learned about
CAPR at <http://csid-capr.unt.edu/>.
2 We discuss this point in more detail, below, in comparing NSF and the EC. For a discussion of resistance to
Broader Impacts at NSF, see especially Holbrook (2005).
3 Concerning the survey, participants from the six funding agencies that are part of the CAPR project (NSF, NIH,
NOAA, NSERC, STW, and the EC) and the general public were invited via individual e-mailed invitations, listserv
postings, website and Facebook advertisements throughout 2010. 490 usable responses were collected, which
included 145 NSF participants and 77 EC participants. Publication of results is in progress (Holbrook and Hrotic,
4 For a discussion of NSF's Broader Impacts Criterion and its relation to education and outreach, see Holbrook,
2005; Frodeman and Holbrook, 2007; Holbrook and Frodeman, 2007; and the essays contained in the 2009 special
edition of Social Epistemology, (Holbrook, 2009).
s Though, the criteria are likely to change in 2012. See note 8, below.
6 Participants in the CAPR survey were asked pairs of questions regarding "intrinsic" and "instrumental" value of
research. Participants were asked to rate how strongly they agreed with statements on a 7-point Likert scale (1 =
"strongly agree," 7 = "strongly disagree"). The first pair of statements asked how comfortable they felt assessing
intrinsic and instrumental value. Results showed significantly more comfort reported for assessing intrinsic value:
"comfortable assessing INTRINSIC" (M=2.12, SD=1.34); ". .. INSTRUMENTAL" (M=2.78, SD=1.55); Z=-6.42, p<0.01.
However, the effect is quite small, and both statements were between "slightly agree" and "agree" on the scale
used. When asked a related pair of questions assessing if participants thought intrinsic and instrumental could be
Here’s what’s next.
This article can be searched. Note: Results may vary based on the legibility of text within the document.
Tools / Downloads
Get a copy of this page or view the extracted text.
Citing and Sharing
Basic information for referencing this web page. We also provide extended guidance on usage rights, references, copying or embedding.
Reference the current page of this Article.
Holbrook, J. Britt & Frodeman, Robert. Peer Review and the Ex Ante Assessment of Societal Impacts, article, 2011; [Guildford, United Kingdom]. (https://digital.library.unt.edu/ark:/67531/metadc38888/m1/10/: accessed May 23, 2019), University of North Texas Libraries, Digital Library, https://digital.library.unt.edu; crediting UNT College of Arts and Sciences.