United States

&lt;p&gt;Roy H. Perlis,&lt;sup&gt;1,2&lt;/sup&gt; Annette Flanagin,&lt;sup&gt;3&lt;/sup&gt; Jacob Kendall-Taylor,&lt;sup&gt;3&lt;/sup&gt; Michael Berkwits,&lt;sup&gt;4&lt;/sup&gt; Kirsten Bibbins-Domingo&lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective &lt;/h4&gt;
&lt;p&gt;The scientific community is divided about the appropriate role of artificial intelligence (AI) in manuscript preparation.&lt;sup&gt;1&lt;/sup&gt; In 2023, JAMA Network journals began requiring authors and reviewers to answer questions about use of AI to create or assist in the creation or editing of submitted manuscripts or with preparation of reviews.&lt;sup&gt;2&lt;/sup&gt; This study was conducted to assess author and reviewer declarations of such use.&lt;/p&gt;
&lt;h4&gt;Design &lt;/h4&gt;
&lt;p&gt;Study of manuscripts and peer reviews submitted to 13 JAMA Network journals from September 2023 to May 2025. Numbers and proportions of manuscripts with corresponding authors and peer reviewers declaring use of AI were examined, along with factors associated with use (eg, journal, country, article type). For authors’ use, reasons, type of AI, and editorial decisions were also assessed; for reviewers’ use, turnaround time and ratings of reviews were assessed. Where sample sizes were sufficient, logistic regression models were run, with use of AI as the dependent variable and article type, journal, and review outcome as independent variables. A 2-sided uncorrected &lt;i&gt;P&lt;/i&gt; value of &amp;lt;.05 was considered the threshold for statistical significance. This study was deemed exempt from ethical approval by the Mass General Brigham Institutional Review Board.&lt;/p&gt;
&lt;h4&gt;Results &lt;/h4&gt;
&lt;p&gt;Of 82,829 manuscripts submitted during the study period, 2257 (2.7%) declared AI use. Authors’ use of AI increased from 1.6% in September 2023 to 4.2% in May 2025 (&lt;i&gt;r&lt;/i&gt;&lt;sup&gt;2&lt;/sup&gt; = .34; &lt;i&gt;P&lt;/i&gt; &amp;lt; .01). &lt;b&gt;Table 25-1115&lt;/b&gt; contrasts manuscripts with or without disclosed AI use by author country, journal, manuscript type, study type, editorial decision, AI use categories, and common AI models used. Author AI use was highest in &lt;i&gt;JAMA Network Open&lt;/i&gt; (4.1%) and &lt;i&gt;JAMA&lt;/i&gt; (3.1%) and lowest in &lt;i&gt;JAMA Dermatology&lt;/i&gt; (1.6%) and was highest among authors from Taiwan (7.2%). Letters to the Editor (odds ratio [OR] 1.63, 95% CI 1.37-1.94) and Viewpoints (OR 1.44, 95% CI 1.21-1.72) were more likely to involve AI use than Original Investigations. The most common reported reasons for AI use by authors were for language and editing (50.0%), data analysis and statistics (11.8%), and content generation and drafting (8.2%). ChatGPT was the most common AI model used by authors (62.9%). Mean (SD) reviewer turnaround times were 11.8 (8.5) days for those who did not disclose AI use and 12.6 (7.6) days for reviewers who disclosed AI use. Reviewer use of AI was highest for &lt;i&gt;JAMA Network Open&lt;/i&gt; (0.9%) and &lt;i&gt;JAMA Psychiatry&lt;/i&gt; (0.8%) and among reviewers from the UK (2.7%).&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/549162fd8a7ceeca532967bc4026042c.png)

&lt;h4&gt;Conclusions &lt;/h4&gt;
&lt;p&gt;Among manuscripts submitted to 13 major medical journals, author declaration of AI use to create or edit content was low but more than doubled during the study period and varied by journal and article type. Very few reviewers disclosed use of AI, possibly because of guidance forbidding such use.&lt;sup&gt;3&lt;/sup&gt; Limitations include small numbers of reported AI use, which limited comparisons for some factors, and the likelihood of underreporting of AI use.&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Kwon D. Is it OK for AI to write science papers? &lt;i&gt;Nature&lt;/i&gt; survey shows researchers are split. &lt;i&gt;Nature&lt;/i&gt;. 2025;641:574-578. doi:10.1038/d41586-025-01463-8&lt;/p&gt;
&lt;p&gt;2. Flanagin A, Pirracchio R, Khera R, Berkwits M, Hswen Y, Bibbins-Domingo K. Reporting use of AI in research and scholarly publication—JAMA Network guidance. &lt;i&gt;JAMA.&lt;/i&gt; 2024;331(13):1096-1098. doi:10.1001/jama.2024.3471&lt;/p&gt;
&lt;p&gt;3. Flanagin A, Kendall-Taylor J, Bibbins-Domingo K. Guidance for authors, peer reviewers, and editors on use of AI, language models, and chatbots. &lt;i&gt;JAMA&lt;/i&gt;. 2023;330(8):702-703. doi:10.1001/jama.2023.12500&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;JAMA+ AI and &lt;i&gt;JAMA Network Open&lt;/i&gt;, Chicago, IL, rperlis@mgh. harvard.edu; &lt;sup&gt;2&lt;/sup&gt;Department of Psychiatry, Mass General Brigham, Boston, MA, US; &lt;sup&gt;3&lt;/sup&gt;&lt;em&gt;JAMA&lt;/em&gt; and the JAMA Network, Chicago, IL, US; &lt;sup&gt;4&lt;/sup&gt;Centers for Disease Control and Prevention, Atlanta, GA, US. 
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt; Roy H. Perlis receives payment for service as a scientific advisor to Circular Genomics, Genomind,and Alkermes. Annette Flanagin, Michael Berkwits, and Kirsten Bibbins-Domingo are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract. 
&lt;h4&gt;Additional Information&lt;/h4&gt;Llama 3.3 (Ollama) was used on June 13, 2025, to identify categories of author-declared use of artificial intelligence. Roy H. Perlis takes responsibility for the integrity of the content generated. 


The 10th Peer Review Congress

Factors Associated With Author and Reviewer Declared Use of AI in Medical Journals

artificial intelligence and large language models

editorial and peer review process

peer review

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Roy H. Perlis,<sup>1,2</sup> Annette Flanagin,<sup>3</sup> Jacob Kendall-Taylor,<sup>3</sup> Michael Berkwits,<sup>4</sup> Kirsten Bibbins-Domingo<sup>3</sup></p>
<h4>Objective </h4>
<p>The scientific community is divided about the appropriate role of artificial intelligence (AI) in manuscript preparation.<sup>1</sup> In 2023, JAMA Network journals began requiring authors and reviewers to answer questions about use of AI to create or assist in the creation or editing of submitted manuscripts or with preparation of reviews.<sup>2</sup> This study was conducted to assess author and reviewer declarations of such use.</p>
<h4>Design </h4>
<p>Study of manuscripts and peer reviews submitted to 13 JAMA Network journals from September 2023 to May 2025. Numbers and proportions of manuscripts with corresponding authors and peer reviewers declaring use of AI were examined, along with factors associated with use (eg, journal, country, article type). For authors’ use, reasons, type of AI, and editorial decisions were also assessed; for reviewers’ use, turnaround time and ratings of reviews were assessed. Where sample sizes were sufficient, logistic regression models were run, with use of AI as the dependent variable and article type, journal, and review outcome as independent variables. A 2-sided uncorrected <i>P</i> value of &lt;.05 was considered the threshold for statistical significance. This study was deemed exempt from ethical approval by the Mass General Brigham Institutional Review Board.</p>
<h4>Results </h4>
<p>Of 82,829 manuscripts submitted during the study period, 2257 (2.7%) declared AI use. Authors’ use of AI increased from 1.6% in September 2023 to 4.2% in May 2025 (<i>r</i><sup>2</sup> = .34; <i>P</i> &lt; .01). <b>Table 25-1115</b> contrasts manuscripts with or without disclosed AI use by author country, journal, manuscript type, study type, editorial decision, AI use categories, and common AI models used. Author AI use was highest in <i>JAMA Network Open</i> (4.1%) and <i>JAMA</i> (3.1%) and lowest in <i>JAMA Dermatology</i> (1.6%) and was highest among authors from Taiwan (7.2%). Letters to the Editor (odds ratio [OR] 1.63, 95% CI 1.37-1.94) and Viewpoints (OR 1.44, 95% CI 1.21-1.72) were more likely to involve AI use than Original Investigations. The most common reported reasons for AI use by authors were for language and editing (50.0%), data analysis and statistics (11.8%), and content generation and drafting (8.2%). ChatGPT was the most common AI model used by authors (62.9%). Mean (SD) reviewer turnaround times were 11.8 (8.5) days for those who did not disclose AI use and 12.6 (7.6) days for reviewers who disclosed AI use. Reviewer use of AI was highest for <i>JAMA Network Open</i> (0.9%) and <i>JAMA Psychiatry</i> (0.8%) and among reviewers from the UK (2.7%).</p>

![](https://assets.underline.io/markdown_image/1/image/549162fd8a7ceeca532967bc4026042c.png)

<h4>Conclusions </h4>
<p>Among manuscripts submitted to 13 major medical journals, author declaration of AI use to create or edit content was low but more than doubled during the study period and varied by journal and article type. Very few reviewers disclosed use of AI, possibly because of guidance forbidding such use.<sup>3</sup> Limitations include small numbers of reported AI use, which limited comparisons for some factors, and the likelihood of underreporting of AI use.</p>
<h4>References</h4>
<p>1. Kwon D. Is it OK for AI to write science papers? <i>Nature</i> survey shows researchers are split. <i>Nature</i>. 2025;641:574-578. doi:10.1038/d41586-025-01463-8</p>
<p>2. Flanagin A, Pirracchio R, Khera R, Berkwits M, Hswen Y, Bibbins-Domingo K. Reporting use of AI in research and scholarly publication—JAMA Network guidance. <i>JAMA.</i> 2024;331(13):1096-1098. doi:10.1001/jama.2024.3471</p>
<p>3. Flanagin A, Kendall-Taylor J, Bibbins-Domingo K. Guidance for authors, peer reviewers, and editors on use of AI, language models, and chatbots. <i>JAMA</i>. 2023;330(8):702-703. doi:10.1001/jama.2023.12500</p>
<sup>1</sup>JAMA+ AI and <i>JAMA Network Open</i>, Chicago, IL, rperlis@mgh. harvard.edu; <sup>2</sup>Department of Psychiatry, Mass General Brigham, Boston, MA, US; <sup>3</sup><em>JAMA</em> and the JAMA Network, Chicago, IL, US; <sup>4</sup>Centers for Disease Control and Prevention, Atlanta, GA, US. 
<h4>Conflict of Interest Disclosures</h4> Roy H. Perlis receives payment for service as a scientific advisor to Circular Genomics, Genomind,and Alkermes. Annette Flanagin, Michael Berkwits, and Kirsten Bibbins-Domingo are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract. 
<h4>Additional Information</h4>Llama 3.3 (Ollama) was used on June 13, 2025, to identify categories of author-declared use of artificial intelligence. Roy H. Perlis takes responsibility for the integrity of the content generated. 


technical paper

<p>Daniel S. Evanko,<sup>1</sup> Michael Di Natale<sup>1</sup></p>
<h4>Objective</h4>
<p> This study assessed the ability to reliably detect the use of generative artificial intelligence (genAI) by authors and reviewers in the field of cancer research, quantified this usage over time, and measured the impact of a change in policy on use of genAI by reviewers.</p>
<h4>Design</h4>
<p> In this cross-sectional study, original research manuscripts and reviewer comments submitted to 10 journals of the American Association for Cancer Research from 2021 through 2024 were analyzed for the presence of AI-generated text. The abstract, methods section, and reviewer comments were extracted from the submission system along with author and reviewer metadata. These sections were chosen because (1) the plain text was stored in the submission system, thus eliminating the need for extraction from files, and (2) we hypothesized that authors would most likely use genAI in the abstract and had observed that methods can be prone to false positives. Text was analyzed using an AI detection tool and classified by average AI likelihood score. Discrimination between full de novo generation of text, language translation, and editing was not possible.</p>
<h4>Results</h4>
<p> We analyzed 46,500 abstracts, 46,021 methods sections, and 29,544 reviewer comments. In 18,467 manuscripts submitted from 2021 through Q3 2022, only 7 abstracts—and no methods or reviewer comment text—were scored as highly likely to contain AI-generated text (AI likelihood score, 0.9 to 1). There were very few false positives (abstract: 0.22%, 40/18,467; methods: 0.07%, 13/18,303; reviewer comments: 0.03%, 4/12,908). Detection of AI-generated text in all 3 sources started increasing dramatically beginning in Q1 2023 after the public release of ChatGPT in November 2022 (<b>Figure 25-1180</b>). Detections in reviewer comments dropped by &gt;50% (from 46 to 34) in Q4 2023 after the journals implemented a policy prohibiting use of genAI for peer review before resuming a linear upward trend. In aggregate, the presence of AI-generated text in 2024 varied greatly between abstracts (23%, 2749/11,959), methods text (10%, 1209/11,875), and reviewer comments (4.8%, 348/7211). There was also substantial journal-dependent variation, from 12% to 15% of abstracts at 3 journals (183/1464, 29/214, 32/214) to 28% to 33% of abstracts at 2 journals (203/716, 438/1342). Authors affiliated with institutions in English-speaking countries were less than half as likely to use genAI compared with other authors. The presence of AI-generated text was associated with a substantially higher rejection rate before peer review regardless of institution location. There was a slight association of AI-generated text with lower-rated reviewer comments. </p>

![](https://assets.underline.io/markdown_image/1/image/0a82456056f57c54deb1f9fd0a3d35b4.png)

<h4>Conclusions</h4>
<p> AI-generated text can be detected in manuscripts and reviewer comments with virtually no false positives. GenAI usage is increasing rapidly, but journal policies to regulate usage can have a limited impact. Usage varies considerably between journals and the type of text and appears to be associated with different editorial outcomes.</p>
<sup>1</sup>American Association for Cancer Research, Philadelphia, PA, US, daniel.evanko@aacr.org. 
<h4>Conflict of Interest Disclosures</h4>
Daniel S. Evanko is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. 
<h4>Acknowledgment</h4> 
We thank John Ho for his assistance collecting the data and entering it into the AI detection tool to obtain the AI likelihood scores.


Quantifying and Assessing the Use of Generative AI by Authors and Reviewers in the Cancer Research Field

<p>Ana-Catarina Pinho-Gomes,<sup>1,2</sup> Carinna Hockham,<sup>1</sup> Mark Woodward<sup>1,3</sup></p>
<h4>Objective</h4>
<p> Women are underrepresented among authors of scientific papers. Although the number of retractions has been rising over the past few decades,<sup>1</sup> gender differences among authors of retracted papers remain poorly understood. Therefore, this study investigated gender differences in authorship of retracted articles available on RetractionWatch.</p>
<h4>Design</h4>
<p> RetractionWatch (<a href="http://retractiondatabase.org/">http://retractiondatabase.org/</a>) provided us a dataset comprising 64,658 articles published between January 1971 and June 2025. This is an update of a previous study.<sup>2</sup> No articles were excluded. We extracted data for the first and last author of each paper and inferred their gender based on their first names using Gender-API software. We accepted gender predictions when the accuracy was estimated to be at least 80%. We estimated the percentage of retractions that could be attributed to women and men first authors and last authors overall and by reason. We used binomial distributions to estimate percentages retracted by women authors, with 95% CIs, and tested whether these percentages were significantly different from 50%. We compared women’s representation as authors of retractions vs their representation as authors of articles overall (40% for first authors and 30% for last authors<sup>3</sup>).</p>
<h4>Results</h4>
<p> Among 64,658 articles retracted between 1971 and 2025 that included 64,659 first authors and last authors, women accounted for 26.4% (95% CI, 25.8%-26.9%) of first authors and 23.7% (95% CI, 23.2%-24.2%) of last authors. There were significant differences in gender of authors in retracted articles by reason, and women’s representation appeared to be lower than their representation as authors of scientific articles (<span class="CharOverride-4"><b>Figure 25-0862</b></span>). The lowest representation of women was found for ethics and legal issues (16.6% [95% CI, 14.2%-19.2%] for first authors and 19.4% [95% CI, 17.1%-21.8%] for last authors). For first authors, women’s representation was the highest for errors caused by editors and publishers (33.8% [95% CI, 31.4%-36.4%]). For last authors, the highest representation of women was for plagiarism (28.9% [95% CI, 27.4%-30.6%]). Most retractions (60.9%) had men as first and last authors.</p>

![](https://assets.underline.io/markdown_image/1/image/6e7f48f38cfc6dda1a863b051bc8705f.png)

<h4>Conclusions</h4>
<p> Women’s representation among authors of retracted articles seemed to be slightly lower than women’s representation as authors of scientific papers overall. Women’s underrepresentation as authors of retractions due to fraud and misconduct as well as ethical concerns suggests that gender equality might enhance research integrity and, hence, reduce the negative impact that retractions have on population health and trust in science.</p>
<h4>References</h4>
<p>1. Gaudino M, Robinson NB, Audisio K, et al. Trends and characteristics of retracted articles in the biomedical literature, 1971 to 2020. <i>JAMA Intern Med. </i>2021;181(8):1118-1121. doi:10.1001/jamainternmed.2021.1807</p>
<p>2. Pinho-Gomes A-C, Hockham C, Woodward M. Women’s representation as authors of retracted papers in the biomedical sciences. <i>PLoS ONE</i>. 2023;18(5):e0284403. doi:10.1371/journal.pone.0284403</p>
<p>3. Squazzoni F, Bravo G, Farjam M, et al. Peer review and gender bias: a study on 145 scholarly journals. <i>Science Advances</i>. 2021;7(2):eabd0299. doi:10.1126/sciadv.abd0299</p>
<h4>Affiliations</h4> <sup>1</sup>The George Institute for Global Health, Imperial College London, London, UK, a.pinho-gomes@imperial.ac.uk; <sup>2</sup>Institute for Global Health, University College London, London, UK; <sup>3</sup>The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia. <h4>Conflict of Interest Disclosures</h4> Mark Woodward has been a consultant for Amgen and Freeline. No other disclosures were reported. <h4>Funding/Support</h4> Ana-Catarina Pinho-Gomes is funded by an Academic Clinical Lectureship by the National Institute for Health and Care Research (NIHR). Mark Woodward is supported by the National Health and Medical Research Council of Australia grant APP1174120. <h4>Role of Funder/Sponsor</h4> The funders had no influence in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation. <h4>Disclaimer</h4> The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.



Comparison of Reasons for Retraction of Biomedical Articles by Women and Men Authors

<p>Tim Kersjes<sup>1</sup></p>
<h4>Objective </h4>
<p>The rise of paper mills presents a significant problem for academic publishers. It is suspected that paper mill actors not only pollute the literature by publishing low-quality or even fake articles; they also try and enhance the credibility of their product by citing their own paper mill articles. This reinforces their credibility and attractiveness for paying customers. Papers mills also attempt to subvert the peer review process by suggesting their own actors as peer reviewers. This study is an analysis of how one successful paper mill used journals that rely on author-suggested reviewers to increase their output and credibility through the creation and use of fake authors and reviewers. The objective was to analyze how these fake personas amplified this paper mill’s success.</p>
<h4>Design </h4>
<p>Following a research integrity investigation by publisher Springer Nature, 55 articles in 4 of their journals (<i>Boundary Value Problems</i>, <i>Journal of Inequalities and Applications</i>, <i>Fixed Point Theory and Applications</i>, and <i>Advances in Difference Equations</i>) were identified to be compromised by a single paper mill, probably operating out of China. These articles were published between September 2013 and August 2020 in journals that asked submitting authors to suggest reviewers. In total, 26 fake personas were identified by confirming with their listed affiliations that these authors were in fact not with these institutions. Network mapping identified the relations between the fake personas and how these aided their publications, as well as citation patterns between articles authored and reviewed by these fake personas.</p>
<h4>Results </h4>
<p>The analysis demonstrated that this paper mill successfully introduced 2 fake personas early on by managing to get these personas as co-authors on published articles. This allowed the paper mill to submit an increasing number of articles (starting with 2 submissions in 2013 to a maximum of 12 submissions in 2016) with new fake personas, with previously published fake personas as suggested reviewers. Because these fake personas infiltrated the peer review process, more fake authors ended up in the published literature, allowing the paper mill to increase its publication output. Of the 26 fake personas, 13 managed to end up as published authors. For these 55 articles, 13 fake personas that are published authors were invited to review for a total of 68 times. In contrast, 13 fake personas without a publication record were only invited 25 times in total. Additionally, 48 of the 55 paper mill articles published by these fake personas cited 2 or more articles by other fake personas in an apparent attempt to further build credibility.</p>
<h4>Conclusions </h4>
<p>Fake personas can serve as both authors and reviewers. Once the paper mill has established a fake persona as credible, it can be used to amplify the credibility of additional fake personas, allowing a paper mill to build a network of fake authors and reviewers to subvert the peer review process. Identity verification and citation analysis may prevent paper mills from gaining success in this way.</p>
<sup>1</sup>Springer Nature, Dordrecht, the Netherlands, tim.kersjes@ springernature.com. 
<h4>Conflict of Interest Disclosures</h4> 
Tim Kersjes is a full-time employee of Springer Nature and owns Springer Nature stock. He also serves on the trustee board of the Committee on Publication Ethics.


Paper Mill Use of Fake Personas to Manipulate the Peer Review Process

<p>Coromoto Power Febres,<sup>1</sup> Julia Gunn,<sup>1</sup> Laura Wilson<sup>1</sup></p>
<h4>Objective </h4>
<p>Publishers, industry bodies, and previous literature have identified authorship changes as a potential means of detecting systematic manipulation activity in submissions to academic journals.<sup>1</sup> There is limited empirical evidence on the features associated with systematic manipulation<sup>2</sup> and, consequently, on their accuracy as indicators. This study tested the hypothesis that attempts to change article authorship post submission indicate further concerns.</p>
<h4>Design </h4>
<p>Since March 2023, all requests across 1321 Taylor &amp; Francis journals to change 3 or more authors following initial manuscript submission have been investigated by the Publication Ethics and Integrity team. This threshold was set based on previous experience that 3 or more changes after review was frequently associated with concerning authorship behavior. The request and associated articles were assessed, including adherence to editorial policies and indicators of systematic manipulation.<sup>3</sup> This cross-sectional study analyzed all such assessments conducted in 2024, consisting of 496 articles; 44 articles were excluded due to incomplete assessment. Requests for fewer than 3 changes are assessed by the journal’s academic editors, for which data are unavailable. The outcomes measured were the authorship change decision recorded and whether other concerns were noted during assessment. Adherence to editorial policies (eg, ethics and consent declarations, conflict of interest disclosure) and indicators of systematic manipulation (eg, tortured phrases, peer review manipulation) were also recorded; these 2 categories were combined into concerns noted<i> </i>and classified as <i>yes</i> or <i>no</i>. Odds ratio (OR) analysis tested associations between denied authorship changes and noted concerns. STROBE guidelines informed the study.</p>
<h4>Results </h4>
<p>Of the 452 articles assessed, most requests for 3 or more authorship changes were denied; 367 (81.2%) were denied vs 85 (18.8%) approved. For the 170 articles for which the data were available, the median (range) number of authors was 4 (3-10). A total of 301 requests (66.6%) had concerns and 151 (33.41%) had no concerns. Of the denied requests for authors changes, 292 (79.6%) had concerns and 75 (20.44%) had no concerns. Of approved requests, 76 (89.41%) had no concerns and 9 (10.59%) had concerns. Denied requests were more likely to have concerns (OR, 32.9 [95% CI, 15.9-68.6]) compared with approved requests. The latest decisions per paper per category are shown in <span class="CharOverride-4"><b>Table 25-1075</b></span>.</p>

![](https://assets.underline.io/markdown_image/1/image/33937be29af5a55dad1ad8e8e63d1896.png)

<h4>Conclusions </h4>
<p>This study suggests that irregularities in authorship changes may indicate research integrity concerns. Limitations include the relatively small sample size and assessment variability across team members/individual assessors. Further analysis is required to establish whether authorship change requests correlate with other systematic manipulation indicators. Further research could also involve analysis by discipline and testing for an association between author demographics and other characteristics, such as co-authorship history and acceptance rate, and a comparison with requests to change 2 or fewer names.</p>
<h4>References</h4>
<p>1.<span class="CharOverride-4"> </span>Abalkina A. Publication and collaboration anomalies in academic papers originating from a paper mill: evidence from a Russia-based paper mill. <i>Learned Publishing. </i>2023;36:689-702. doi:10.1002/leap.1574</p>
<p>2. Byrne JA, Abalkina A, Akinduro-Aje O, et al. A call for research to address the threat of paper mills. <i>PLoS Biol</i>. 2024;22(11). doi:10.1371/journal.pbio.3002931</p>
<p>3. Parker L, Boughton S, Bero L, Byrne JA. Paper mill challenges: past, present, and future. <i>J Clin Epidemiol</i>. 2024;176. doi:10.1016/j.jclinepi.2024.111549</p>
<sup>1</sup>Taylor & Francis Group, Beeston, UK, coro.powerfebres@tandf. co.uk. 
<h4>Conflict of Interest Disclosures</h4> 
Coromoto Power Febres reported being a full-time employee of Taylor & Francis and being a member of the STM Research Integrity Committee and United 2 Act Education and Awareness working group. Julia Gunn reported being a full-time employee of Taylor & Francis. Laura Wilson reported being a full-time employee of Taylor & Francis and being a member of the STM Membership committee, United 2 Act Trust markers working group, and ALPSP Policy committee.


Authorship Changes as an Indicator of Research Integrity Concerns in Submissions to Academic Journals

<p>Nicholas J. DeVito,<sup>1</sup> Christine Cunningham,<sup>1</sup> Ben Goldacre<sup>1</sup></p>
<h4>Objective </h4>
<p>Continued citation of retracted articles is an ongoing issue.<sup>1</sup> RetractoBot examines whether informing authors that they cited a retracted paper reduces future citations to those articles.<sup>2</sup></p>
<h4>Design </h4>
<p>This was a parallel-group, 2-group, superiority trial. We identified and randomized retracted articles in PubMed published since 2000. The study was unblinded, but primary outcome ascertainment was objective, and randomization and allocation were fully and simultaneously computer automated. Authors with publicly available contact information were notified about citations of retracted articles in the intervention group; no notifications were sent about articles in the control group. The email included a voluntary survey question on knowledge of the retractions and invited additional voluntary feedback. Retraction, contact, and citation data were obtained from PubMed and Scopus. Follow-up started 3 months after the final intervention email was sent to allow in-press manuscripts that could not be affected by our intervention a chance to publish. The primary outcome was the difference in the rate of new citations of retracted articles between groups. We used a negative binomial model adjusted for years since retracted article publication, and our stratification variable, with outliers removed. Data analysis was performed in Stata version 16. We also report the results of a survey question on knowledge of retracted citations, and a sentiment analysis on email feedback. Additional methods details, including power calculations, are available in our prespecified protocol<sup>3</sup> and at retracted.net.</span></p>
<h4>Results </h4>
<p>We randomized 7958 retracted articles to the intervention group and 7963 to the control group (<b>Figure 25-0938</b>). Overall, 246,749 intervention emails were deliverable to authors between January 8 and February 7, 2024, with follow-up extending from May 8, 2024, to May 7, 2025. After removing outliers, there were 7939 citations of articles in the intervention group and 7943 in the control group. The effect of our intervention was not significant (mean citation rate, −0.007; 95% CI, −0.055 to 0.041) and held for all prespecified sensitivity analyses. We received 15,667 survey responses from notified authors with 80.6% (12,631) indicating they were unaware of all retracted citations we informed them about. We received 482 substantive pieces of email feedback, of which 57.7% (278) indicated a positive sentiment about the project (vs neutral, negative, mixed, or other).</p>

![](https://assets.underline.io/markdown_image/1/image/63dcfd9d05b6055115861cb2b4f70ce9.png)

<h4>Conclusions </h4>
<p>Our intervention did not significantly decrease citations of retracted articles at 1 year compared with the control group. We plan future secondary analyses to determine whether an effect emerges with increased follow-up. Of the authors who responded, most were unaware of the notified retractions and had a positive view of the intervention, indicating the potential for future interventions in this area. The continued uptake of retraction notifications in citation management software, using sources such as the RetractionWatch database, may already be having an effect in this space. </p>
<h4>References</h4>
<p>1. Kühberger A, Streit D, Scherndl T. Self-correction in science: the effect of retraction on the frequency of citations. <i>PLoS One</i>. 2022;17(12):e0277814.</p>
<p>2. Xu H, Ding Y, Zhang C, Tan BCY. Too official to be effective: an empirical examination of unofficial information channel and continued use of retracted articles. <i>Res Policy</i>. 2023;52(7):104815.</p>
<p>3. Cunningham C, DeVito N, Goldacre B. RetractoBot—revised stage 1 registered report protocol. Published online October 31, 2023. doi:10.6084/M9.FIGSHARE.24468391.V1</p>
<p><sup>1</sup>Bennett Institute for Applied Data Science, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK, nicholas.devito@phc.ox.ac.uk.</p>
<h4>Conflict of Interest Disclosures</h4> 
Nicholas J. DeVito reported receiving funding from the Naji Foundation, the Fetzer-Franklin Memorial Fund, the European Commission, and UK Research and Innovation (UKRI) outside the submitted work. Ben Goldacre reported receiving research funding from the Bennett Foundation, the Laura and John Arnold Foundation, the National Health Service (NHS) National Institute for Health Research (NIHR), the NIHR School of Primary Care Research, NHS England, the NIHR Oxford Biomedical Research Centre, the Mohn-Westlake Foundation, NIHR Applied Research Collaboration Oxford and Thames Valley, the Wellcome Trust, the Good Thinking Society, Health Data Research UK, the Health Foundation, the World Health Organization, UKRI Medical Research Council, Asthma UK, the British Lung Foundation, and the Longitudinal Health and Wellbeing strand of the National Core Studies program outside the submitted work; he has previously been a non-executive director at NHS Digital; he also receives personal income from speaking and writing for lay audiences on the misuse of science. 
<h4>Funding/Support</h4>The RetractoBot project was funded by a grant from the Laura and John Arnold Foundation and a grant and in-kind support from Elsevier. 
<h4>Role of Funder/Sponsor</h4> The funders had no role in the study design, data collection (outside of in-kind access to elements of the Scopus API from Elsevier), data analysis, interpretation, decision to submit, or any other facet of the planning, conduct, or dissemination of the presented work. 
<h4>Acknowledgment</h4> We would like to acknowledge the full RetractoBot study team including Karolina Wartolowska, Henry Drysdale, Benjamin Feakins, Helen J. Curtis, Francis Irving, Anna Powell-Smith, and Seb Bacon.


Notifying Authors That They Have Cited a Retracted Article and Future Citations of Retracted Articles: The RetractoBot Randomized Controlled Trial

<p>Leslie D. McIntosh,<sup>1</sup><span lang="ar-SA"> Hélène Draux</span>,<sup>1</sup> Elizabeth Smee,<sup>1</sup> Cynthia Hudson Vitale<sup>2</sup></p>
<h4>Objective </h4>
<p>To understand the broad impact of one questionable network on publishing, including the publishers, countries, and institutions involved and affected by the network’s activity.</p>
<h4>Design </h4>
<p>We examined the Pharmakon Neuroscience Research Network (PNN), a questionable entity listed in funding statements, acknowledgments, and author affiliations in articles published between 2019 and 2023. We selected the PNN because it was the first potential broad-scale case of scientific manipulation identified by one of the authors (L.D.M), and analyzing the PNN has helped the development of patterns to discover other unusual collaboration networks. All documents containing “Pharmakon Neuroscience” anywhere in the title, abstract, author affiliations, and full text were selected using Dimensions.<sup>1</sup> We excluded retraction and correction notices; articles for which a peer reviewer identified their organization as PNN; and articles with more than 25 authors. We inspected the metadata associated with the articles, notably the organizations related to the articles. We situated these practices within the Taxonomy of Scientific Manipulation,<sup>2</sup> systematically identifying (1) who was involved (authors, institutions, and publishers), (2) where<span class="CharOverride-9"> </span>mechanisms were used to manipulate science (journals, countries, and institutions), and (3) how the science was manipulated (methods for deceiving scholarly communication).</p>
<h4>Results </h4>
<p>Pharmakon Neuroscience was referenced in 140 published articles as of January 2025. After applying exclusion criteria, 123 articles remained. These articles included more than 6000 citations (citation ratio of 51 citations per article). The articles had 361 unique authors, including 29 without author identifiers (eg, ORCiD, Dimensions). The PNN involved 56 journals across 12 publishers, with Springer Nature (n = 29), Elsevier (n = 26), and Bentham Science Publishers (n = 25) having the most articles involved out of the total (n = 123). The involved authors were affiliated with 40 countries and 232 organizations. Of these organizations, 212 had research identifiers (eg, Research Organization Registry, Global Research Identifier Database), and 20 did not. Three of the organizations appeared to be personal homes, even though they were claimed as businesses or educational institutions. Additionally, one publication<sup>3</sup> highlighted the PNN as an affiliation among the “top 25 most predictive and impactful authors.”</p>
<h4>Conclusions </h4>
<p>This investigation leveraged publication databases and advanced forensic scientometric techniques to highlight critical research integrity and security vulnerabilities in a questionable research network. We use the term “questionable network” because this co-authorship network appears to have grown too rapidly to have developed naturally from scholarly affiliations over time. The impact of the PNN’s work might be legitimized through future studies and become seen as normative behavior. Although the most prolific PNN author and organization stem from one country, there is a global impact not confined to high- or low-income countries. While one individual appears central to the network, a single orchestrator does not imply sole responsibility. Instead, this case underscores the complexity of manipulating the scientific ecosystem, where questionable actors exploit the scholarly communication system through nonverifiable funders and organizations and artificially inflated citation metrics.</p>
<h4>References</h4>
<p>1. Hook DW, Porter SJ, Herzog C. Dimensions: building context for search and evaluation. <i>Front Res Metr Anal</i>. 2018;3:23. doi:10.3389/frma.2018.00023</p>
<p>2. McIntosh LD, White W, Vitale CH. Unveiling deception: establishing a taxonomic framework for disinformation within scientific discourse. <i>arXiv</i>. Preprint posted online November 19, 2023. doi:10.48550/arxiv.2311.11344</p>
<p>3. Sab MC, Ahmed KM. Tracing Bangladesh’s scientific footprints: a scientometric expedition into inflammation pharmacology. <i>Inform Res Comm</i>. 2024;1(1):9-21. doi:10.5530/irc.1.1.4</p>
<sup>1</sup>Digital Science, London, UK, l.mcintosh@digital-science.com; <sup>2</sup>Johns Hopkins University, Baltimore, MD, US. 
<h4>Conflict of Interest Disclosures</h4> Leslie D. McIntosh, Hélène Draux, and Elizabeth Smee are employees of Digital Science, a research technology company.


How a Questionable Research Network Manipulated Scholarly Publishing

<p>Michael O. Mensah,<sup>1</sup> Anand R. Habib,<sup>2</sup> Jacob Kendall-Taylor,<sup>3</sup> Mya Roberson,<sup>4</sup> Kanade Shinkai,<sup>4</sup> Annette Flanagin,<sup>3</sup> Preeti Malani<sup>3</sup></p>
<h4>Objective </h4>
<p>In March 2023, the JAMA Network invited peer reviewers to check a box when concerned about elements of equity, diversity, and inclusion (EDI) in manuscripts, and to explain such concerns in confidential comments to the editor. We examined peer reviewers’ use of the EDI concerns checkbox and categorized concerns described in confidential comments.</p>
<h4>Design </h4>
<p>In this cross-sectional, mixed-methods analysis, we initially examined all reviewed manuscripts for 13 JAMA Network journals between March 28, 2023, and December 31, 2024, and determined counts and percentages of reviews that utilized an EDI concerns checkbox. As a case study, we then used rigorous and accelerated data reduction (RADaR) to analyze deidentified, confidential reviewer comments for <i>JAMA Dermatology</i> manuscripts. RADaR develops all-inclusive data tables, then iteratively reduces those tables until only research question–relevant data remain.<sup>1</sup> Two former <i>JAMA</i> editorial fellows independently classified confidential comments as either relevant or irrelevant to EDI and then inductively coded comments. The coders agreed on which code(s) best described each comment and then distilled prominent categories among all comments. This study was deemed exempt by Yale School of Medicine’s institutional review board and follows STROBE and SRQR guidelines.</p>
<h4>Results </h4>
<p>Among 39,151 reviews across all JAMA Network journals, 5.3% (2075 of 39,151) utilized the EDI concerns checkbox, including 5.5% (1897 of 34,542) of research manuscript reviews. <i>JAMA Otolaryngology</i> reviews utilized the EDI checkbox least often (2.63% [29 of 1104] overall; 2.7% [26 of 948] of research manuscripts). <i>JAMA Pediatrics</i> reviews utilized the EDI checkbox most often (8.14% [146 of 1793]; 8.0% [126 of 1583] of research manuscripts). <i>JAMA Dermatology</i> reviews utilized the EDI checkbox at the median frequency among all JAMA Network journals (4.84% [91 of 1879]; 5.4% [78 of 1439] of research manuscripts) (<b>Table 25-1090</b>). Among <i>JAMA Dermatology </i>reviews utilizing the EDI checkbox, 34.1% (31 of 91) had confidential comments relevant to EDI, 50.5% (46 of 91) were irrelevant to EDI, and 15.4% (14 of 91) had no comment. Regarding EDI-relevant confidential comments, categories of responses included lack of clarity regarding race and ethnicity categories (48.4% [15]), statistical or methodological concerns (16.1% [5]), skin tone representation (9.7% [3]), ambiguity in language around demographic identifiers (9.7% [3]), and inappropriate use of gender ascribed by observers (6.5% [2]).</p>

![](https://assets.underline.io/markdown_image/1/image/d05d9cd83a84ebec0b4dfb72e01f8163.png)

<h4>Conclusions </h4>
<p>Overall,<span class="CharOverride-4"> </span>the EDI concerns checkbox was infrequently used across the JAMA Network journals. Among <i>JAMA Dermatology</i> reviews using the EDI concerns checkbox, less than half had confidential comments relevant to EDI, perhaps because instructions to reviewers did not define EDI concerns. Prominent categories among EDI-relevant comments included concerns about proper usage of race and ethnicity categories, absence of photos depicting dermatologic conditions in darker skin tones, and use of inclusive language regarding sex and gender, consistent with equity issues previously noted by the American Academy of Dermatology.<sup>2</sup> Future work could explore ways of systematically capturing EDI concerns from reviewers and their eventual influence on manuscripts accepted for publication.</p>
<h4>References</h4>
<p>1. Watkins DC. Rapid and rigorous qualitative data analysis: the “RADaR” technique for applied research. <i>Int J Qualitative Methods</i>. 2017;16(1):1609406917712131. doi:10.1177/1609406917712131</p>
<p>2. American Academy of Dermatology Association. Diversity and the academy. Accessed June 8, 2025. <a href="https://www.aad.org/member/career/diversity">https://www.aad.org/member/career/diversity</a></p>
<sup>1</sup>Department of Psychiatry, Yale School of Medicine, New Haven, CT, US, michael.mensah@yale.edu; <sup>2</sup>National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, US; <sup>3</sup>JAMA and the JAMA Network, Chicago, IL, US; <sup>4</sup>JAMA Dermatology, Chicago, IL, US. 
<h4>Conflict of Interest Disclosures </h4>
Michael O. Mensah is co-editor of the Race and Mental Health Equity column in Psychiatric Services, unrelated to this work. Mya Roberson receives consulting fees from the National Committee for Quality Assurance outside the scope of the submitted work. Annette Flanagin is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.


An Analysis of Equity, Diversity, and Inclusion Concerns From JAMA Network Peer Reviewers

<p>Anne Lasinsky,<sup>1</sup> James Wrightson,<sup>2</sup> Matthew Hogel,<sup>3</sup> Alannah Brown,<sup>3</sup> Adrian Mota,<sup>3</sup> Karim M. Khan,<sup>1,2,4</sup> Clare L. Ardern<sup>5,6</sup></p>
<h4>Objective</h4>
<p> A small number of Canadian research funders have implemented interventions to address known biases in grant peer review. Equalization, which aims to match the proportion of funded grant applications to the proportion of submitted applications from specific underrepresented groups, is one such intervention. In 2016, the Canadian Institutes of Health Research (CIHR) implemented equalization for early career researcher (ECR) principal applicants in its Project Grant Competition. In 2021, equalization expanded to female principal applicants and French-language applicants. The objective of this study was to describe the outcome of equalization in 2022.</p>
<h4>Design</h4>
<p> This was a retrospective analysis of the number of funded grants in the spring 2022 CIHR Project Grant Competition. The equalization intervention was applied to applications submitted by ECR applicants, female principal applicants, and applications in French. To apply the intervention, first, scores for all applications submitted to the Project Grant Competition, which were reviewed by 59 committees, were converted to a percentage rank to account for scoring differences across committees. CIHR funded applications in rank order from the top percent rank, as far down as the competition budget allowed, and intervened to ensure that the next-ranked applications from each underrepresented applicant group were funded. Equalization matched the proportion of applications funded to the proportion of applications submitted by each group. No grants were defunded. Any grants that were equalized were added to the pool of funded grants—they did not displace another applicant’s score-win. This study descriptively analyzed routinely collected data from CIHR. The main outcome was the number of grant applications funded for each underrepresented applicant group, with and without equalization. The secondary outcome was the proportion of funded applications for each underrepresented applicant group, with and without equalization.</p>
<h4>Results</h4>
<p> There were 2095 applications submitted to the spring 2022 Project Grant Competition. Before equalization, 370 applications (17.7%) were funded. After equalization, 405 applications (19.3%) were funded with a total of CAD$325 million. ECR principal applicants submitted 580 applications (27.7%), female principal applicants submitted 774 applications (36.9%), and 25 applications (1.2%) were submitted in French). After equalization, 21 additional applications from ECR principal applicants and 22 applications from female principal applicants were funded (<b>Table 25-1092</b>). The funding success rates increased from 16.9% to 20.7% for ECR principal applicants and 17.5% to 20.3% for female principal applicants. One additional French-language application was funded with equalization; the success rate for French-language applicants increased from 17.4% to 21.7%.</p>

![](https://assets.underline.io/markdown_image/1/image/7ed3f1babc4b418643dbb1605b2d00b4.png)

<h4>Conclusions</h4>
<p> In the spring 2022 CIHR Project Grant Competition, equalization increased the number of health research grants awarded and the funding success rate for ECR and female principal applicants, and for applications submitted in French.</p>
<h4>Affiliations</h4> <sup>1</sup>School of Kinesiology, The University of British Columbia, Vancouver, Canada; <sup>2</sup>Department of Family Practice, The University of British Columbia, Vancouver, Canada; <sup>3</sup>Canadian Institutes of Health Research, Ottawa, Canada; <sup>4</sup>Canadian Institutes of Health Research-Institute of Musculoskeletal Health and Arthritis, Vancouver, Canada; <sup>5</sup>Department of Physical Therapy, The University of British Columbia, Vancouver, Canada, clare.ardern@ ubc.ca; <sup>6</sup>Sport and Exercise Medicine Research Centre, La Trobe University, Melbourne, Australia. <h4>Conflict of Interest Disclosures</h4> Matthew Hogel is Deputy Director, Funding Analytics at the Canadian Institutes of Health Research (CIHR). Alannah Brown is Senior Advisor to the Associate Vice-President at CIHR. Adrian Mota is Acting Vice President, Research—Programs at CIHR. Karim M. Khan is Scientific Director for CIHR’s Institute of Musculoskeletal Health and Arthritis (20172025). No other conflicts were reported. <h4>Funding/Support</h4> This work was supported by a CIHR Research Operating Grant (Scientific Directors) held by Karim M. Khan. CIHR’s Funding Analytics coordinated data management and analysis as part of its mandate to foster and deliver high quality peer review for health research in Canada. <h4>Role of the Funder/Sponsor</h4> CIHR did not participate in preparing the abstract, nor in the decision to submit the abstract for presentation.



Assessment of an Intervention to Equalize the Proportion of Funded Grant Applications for Underrepresented Groups at the Canadian Institutes of Health Research

<p>Noémie Aubert Bonn,<sup>1,2</sup> Lukas Hughes-Noehrer<sup>1</sup></p>
<h4>Objective </h4>
<p>The impact that research environments have on the quality of research has gained heightened visibility in the past few years. The Research Excellence Framework (REF), a UK-wide funding exercise that assesses higher education institutions (HEIs) on the excellence of their research, includes research environment as part of the elements it assesses. Recently, the weight of research environment was increased to one-fourth of the total score, bringing it closer to the other elements of research outputs and impacts. However, measuring positive research environments is challenging and clear indicators are still hard to delineate.</p>
<h4>Design </h4>
<p>To identify indicators that may be associated with positive research environments, we conducted a qualitative analysis of environment statements submitted to REF 2021.<sup>1</sup> We sampled 34 four-star statements drawn from different units of assessment (ie, academic disciplines) available openly on the REF website. Four-star scores are given to statements that describe “an environment that is conducive to producing research of world-leading quality and enabling outstanding impact, in terms of its vitality and sustainability.” Using these statements, 1 researcher conducted a mix of inductive (first 17 statements in NVivo 12) and deductive (last 17 statements) thematic analysis to extract the potential indicators of positive research environments. These indicators were organized in core themes, discussed with the research team, and compared with existing recommendations, policies, and literature on healthy research environments to build an exhaustive overview of possible indicators.</p>
<h4>Results </h4>
<p>The analysis  identified a vast array of elements, which were used to showcase positive research environments. We organized the elements extracted in indicators providing details on people, such as diversity metrics, staff and student characteristics, and measures to improve equity, diversity, and inclusion; indicators demonstrating positive research cultures, such as responsible leadership, collaborative culture, research integrity, and open research; and indicators demonstrating positive research environments, such as capacity-building measures, staff and financial support, employment security and progression, and responsible assessment practices (<b>Table 25-0889</b>). The breadth of indicators identified showcased differences and similarities in how different institutions and units of assessment qualified their research environments.</p>

![](https://assets.underline.io/markdown_image/1/image/8f8b88702af17ad7b2039178c62af22e.png)

<h4>Conclusions </h4>
<p>Our analysis provided insights on the breadth of elements that can come into play in creating a positive research environment. Despite being drawn from the UK landscape, our findings could be relevant to a broad diversity of research settings. For example, our findings could help support more comprehensive research assessment or they could help inspire concrete actions to improve research environments in HEIs.</p>
<h4>Reference</h4>
<p>1. Research Excellence Framework. Results and submissions. May 12, 2022. Accessed June 24, 2025. <a href="https://results2021.ref.ac.uk/">https://results2021.ref.ac.uk/</a></p>
<sup>1</sup>The University of Manchester, Department of Computer Science, Faculty of Science and Engineering, Manchester, UK, noemie. aubertbonn@manchester.ac.uk; <sup>2</sup>Hasselt University, Department of Healthcare & Ethics, Faculty of Medicine, Hasselt, Belgium. 
<h4>Conflict of Interest Disclosures</h4> Noémie Aubert Bonn was employed as a senior policy advisor with Research England from 2022 to 2023. 
<h4>Funding/Support</h4> The conceptualization, design, and part of the analysis was carried out and supported during Noémie Aubert Bonn’s employment at Research England. The analysis software subscription was provided by Hasselt University, Belgium. <h4>Role of the Funder/Sponsor</h4> The conclusions of the study are independent from Research England. 
<h4>Acknowledgment</h4> We wish to thank all the members of the People, Culture and Environment team at Research England who provided feedback and discussion throughout this project, particularly Catriona Firth, Claire Fraser, Steven Hill, Myles Furr, Marie-Helene Nienaltowski, Duncan Shermer, and Nicole Dixon. We also wish to thank everyone within UK Research and Innovation and The University of Manchester who provided feedback and insights on this project.


Extracting Research Environment Indicators From the UK Research Excellence Framework 2021 Statements

<p>Ahmad Sofi-Mahmudi,<sup>1</sup> Hesam Salmabadi<sup>2</sup></p>
<h4>Objective</h4>
<p> To evaluate the association between countries’ democratic status and scientific article retraction rates while exploring potential factors influencing this association.</p>
<h4>Design</h4>
<p> This retrospective cohort study analyzed publication and retraction data from 2006 to 2023. Data sources included the Retraction Watch Database for retraction data (n = 64,264 retractions), the Economist Intelligence Unit’s Democracy Index for democratic status, and SCImago for country-specific publication outputs (n = 59,385,751 publications). Our primary outcome was retraction rate (retractions per 10,000 publications) rather than absolute retraction counts, to account for varying publication volumes across countries. Using bayesian hierarchical negative binomial regression models with multiple imputations for missing data, we examined the association between the democracy index and article retractions while controlling for key confounders, including gross domestic product (GDP) per capita, research investment as percentage of GDP, English language proficiency scores, corruption control, government effectiveness, regulatory quality, rule of law, press freedom, and international collaboration rates, across 167 countries.</p>
<h4>Results</h4>
<p> The analysis of 167 countries demonstrated a median democracy index score of 5.71 (range, 1.05-9.81) and a median scientific publication retraction rate of 5.3 per 10,000 publications (range, 0-214.8). Higher democracy scores were associated with lower retraction rates (adjusted coefficient, −0.46; 95% credible interval [CrI], −0.59 to −0.32) after controlling for confounders (<b>Figure 25-1106</b>). Among institutional factors, rule of law showed a positive association (0.40; 95% CrI, 0.17-0.63), whereas corruption control demonstrated a negative association (−0.54; 95% CrI, −0.72 to −0.36) with retraction rates. The association between democracy and retractions remained stable across periods (interaction coefficient, −0.03; 95% CrI, −0.46 to 0.39), although press freedom effects varied significantly between the 2006 to 2012 and 2013 to 2023 periods (interaction coefficient, −1.40; 95% CrI, −1.77 to −1.03). GDP per capita (0.09; 95% CrI, 0.01-0.17) and government effectiveness (0.21; 95% CrI, 0.01-0.41) were positively associated with retraction rates, suggesting better detection and correction mechanisms in wealthier, more effectively governed countries.</p>

![](https://assets.underline.io/markdown_image/1/image/7d8b6c98d8de5538c342378e51b17ad2.png)

<h4>Conclusions</h4>
<p> Democratic status shows a negative association with scientific article retraction rates, although this association is complex and mediated by institutional factors. The findings suggest that while democratic societies may have lower rates of problematic research requiring retraction, they also demonstrate stronger institutional capacity for detecting and addressing research misconduct.</p>
<h4>Affiliations</h4> <sup>1</sup>Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada, a.sofimahmudi@ gmail.com; <sup>2</sup>Department of Environmental Sciences, University of Quebec in Trois-Rivières, Trois-Rivières, Quebec, Canada. <h4>Conflict of Interest Disclosures</h4> Ahmad Sofi-Mahmudi is an employee of Cytel Canada Health Inc. <h4>Additional Information</h4> We used Claude, 3.5 Sonnet (claude-35-sonnet-20241022) through its website for refining the analysis codes. Also, it was used as a grammar and flow checker of the abstract.


Downloads

Next from The 10th Peer Review Congress

Quantifying and Assessing the Use of Generative AI by Authors and Reviewers in the Cancer Research Field

Similar lecture

Authors Who Publish in a Journal and Likelihood to Serve as Reviewers

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from The 10th Peer Review Congress

Quantifying and Assessing the Use of Generative AI by Authors and Reviewers in the Cancer Research Field

Similar lecture

Authors Who Publish in a Journal and Likelihood to Serve as Reviewers

Downloads