What is it about?

When researchers apply for external funding to support a new project, funders contact subject experts (known as "peers") and ask them to evaluate and comment on the research proposal. In this paper, we report a thematic analysis of what subject experts said while they were conducting an independent review of a proposal in near real time.

Featured Image

Why is it important?

The involvement of independent scholars in the process of selecting applications for research funding is a very common practice. Yet, knowledge of how peer-reviewers approach this task and what they ponder on during this process is very scarce. In our study, we found that scholars engaged in thoughtful considerations during the peer-review process and identified five ethical dilemmas they faced while reviewing proposals: (1) whether to accept an invitation to review, (2) whether to rely only on the information presented in the application or to "google" applicants, (3) whether they should pay attention to institutional prestige or root for the underdog applicants, (4) whether they should comment on everything they are ask to comment on or only comment on what falls within the remits of their expertise, and (5) whether they should support riskier projects and overlook shortcomings or err on the side of caution. Better understanding how this "invisible work" gets done allows funders and applicants to better understand how proposals get evaluated. But, most importantly, it also provides valuable insights for improving the effectiveness, efficiency, and fairness of this crucial stage in the research funding process.

Perspectives

This article provides a fascinating account of what scholars actually do when they review their peers' ideas for future research. The dilemmas we've uncovered suggest that better understanding the norms and practices of peer-reviewing work is crucial for improving how we rely on them. Personally, I don't think peer-reviewers (all) deserve the bad press they often get for disagreeing with one another. As scholars, we may have different opinions about what is outstanding, original, or even rigorous, but this is not to say that our opinions are unfounded, on the contrary: most reviewers we observed were conscientious and diligent in how they carried out this citizenship duty. What we found, however, was that they differed in their views of what was the right (in the ethical sense) approach to approach their task, and what was required of them by the funders. Rather than bemoaning the lack of agreement among peer-reviewers, we could invest in better understanding what good practice is, and what the key factors reviewers and funders should pay attention to, in order to forecast what proposals are likely to have the biggest impact on advancing understanding. We could also recognise the ethical challenges involved in peer-reviewing and work to develop better guidelines to inform and train current and future generations of peer-reviewers.

Professor Gaelle Vallee-Tourangeau
Kingston University

Read the Original

This page is a summary of: Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences, Humanities and Social Sciences Communications, March 2022, Springer Science + Business Media,
DOI: 10.1057/s41599-022-01050-6.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page