Spain

PEERE

Researching peer review in biomedical journals: more collaborative cross-journal research is needed

Following the success of the 1st PEERE International Conference on Peer Review, the PEERE COST Action follow-up community organised the second edition of the PEERE International Conference on Peer Review  and was celebrated virtually on 29 September – 1 October 2020.

The conference welcomed any contribution (e.g., original research, position papers, literature reviews) that used any method (e.g., quantitative, experimental or qualitative) to investigate peer review in a variety of scientific domains (e.g., scholarly journals, funding agencies, research assessment). Interdisciplinary research is especially welcome.

The program is published here: http://tiny.cc/peere2020 


The second edition of the PEERE International Conference on Peer Review aims to provide a forum for scholars, practitioners and science stakeholders to share research, models, tools and experience on peer review in different fields, e.g., medicine, computer science, social sciences and humanities. It aims to stimulate the use of evidence-based research in the design and implementation of peer review in a variety of fields and encourage more systematic research.

technical paper

Problem
One of the mechanisms proposed to improve reporting of research results is the installation of reporting guidelines in scholarly journals. These guidelines have been issued in a variety of research contexts and with a diverse set of aims, including improving identification and validation of biomedical research materials. Considering these journal initiatives from a science policy perspective, they constitute rule-based or ‘regulatory’ policy, attempting to change researchers’ behaviour through regulation. Rules require policing, which is often expensive, and rules tend to fail if they lack community support. 
Approach
To analyse the effectiveness of journal reporting guidelines we assessed such guidelines in two contexts within biomedical science. We first analysed reporting guidelines in the context of misidentified cell lines, which constitutes a long-known and large-scale issue in biomedical research. Secondly, we assessed guidelines in the context of antibody validation.
Using bibliometric techniques, we set out to study how journal guidelines affect the number of journal articles reporting on misidentified cell lines and the provision of sufficient antibody validation information. For both cases, we compared journals with and without relevant reporting guidelines, as well as reporting before and after the introduction of guidelines.
Results
In the case of cell lines, we observed a decrease in the percentage of articles that use misidentified cell lines in journals with guidelines. However, this decrease already starts before the implementation of the guidelines and does accelerate when the guidelines are implemented. Similarly, we also find a decrease in the number of articles on misidentified cells in journals not introducing guidelines. In addition, the percentage of articles that use misidentified cell lines is higher in journals with guidelines than in those without.
Conclusions
Our results suggest that journal reporting guidelines are an only moderately effective mechanism to steer individual behaviour. In addition, the context and precise format in which they are implemented highly influences the likelihood of guidelines actually leading to improved reporting. In our studies, the implementation of guidelines seems to have resulted in symbolic or minimal compliance. The observed trend in reporting on misidentified cells, suggests that journal guidelines only formalise reporting behaviour that has already been adopted by the community.

Biomedical journal reporting guidelines: what they can and cannot do.

Peer Review Procedures & Trials

Peering Into Peer Review

Lightning Session 1

Lightning Session 2

Remodeling Peer Review In Light Of Preprints

Computational Studies Of Peer Review

Peer Review In The Post -Covid Era

Peer Review Of Grant Proposals

Policy Lessons From Social Simulation

https://d3smihljt9218e.cloudfront.net/transcript/1964/subtitle/b23dcdf55b50156eaeb08de3e815d430.vtt

English (automatic)

Downloads

Next from PEERE

Biomedical journal reporting guidelines: what they can and cannot do.

Similar lecture

Peer Review Metrics And Data Analysis

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from PEERE

Biomedical journal reporting guidelines: what they can and cannot do.

Similar lecture

Peer Review Metrics And Data Analysis

Downloads