We explore which characteristics of a reviewer affect disagreement with the other reviewers in research proposal evaluation, studying 52,488 proposals submitted to the Horizon 2020 Marie Skłodowska-Curie Actions (MSCA) and 1,939 proposals to the European Cooperation in Science and Technology Actions (COST). We find that experience in previous calls significantly reduces disagreement, and in MSCA - IF, an inverted U relationship between the number of proposals evaluated in a given call and disagreement, with a remarkable decrease of disagreement above thirteen proposals evaluated. The results suggest that reviewing experience improve reliability, curbing unwarranted disagreement by gradually fine-tuning reviewers’ evaluation.

Does reviewers' traits affect disagreement in proposal evaluation?

<br>



**Chair:** Marco Seeber, University of Agder


4. Peer review of grant proposals

technical paper

Following the success of the 1st PEERE International Conference on Peer Review, the PEERE COST Action follow-up community organised the second edition of the PEERE International Conference on Peer Review  and was celebrated virtually on 29 September – 1 October 2020.

The conference welcomed any contribution (e.g., original research, position papers, literature reviews) that used any method (e.g., quantitative, experimental or qualitative) to investigate peer review in a variety of scientific domains (e.g., scholarly journals, funding agencies, research assessment). Interdisciplinary research is especially welcome.

The program is published here: http://tiny.cc/peere2020 


PEERE

The second edition of the PEERE International Conference on Peer Review aims to provide a forum for scholars, practitioners and science stakeholders to share research, models, tools and experience on peer review in different fields, e.g., medicine, computer science, social sciences and humanities. It aims to stimulate the use of evidence-based research in the design and implementation of peer review in a variety of fields and encourage more systematic research.

Marco Seeber

1

SHORT BIO

Presentations

Does reviewers' traits affect disagreement in proposal evaluation?