United States

&lt;p&gt;Michael C. Ferraro,&lt;sup&gt;1,2&lt;/sup&gt; Aidan G. Cashin,&lt;sup&gt;1,2&lt;/sup&gt; Amanda C. de C. Williams,&lt;sup&gt;3&lt;/sup&gt; Emma Fisher,&lt;sup&gt;4&lt;/sup&gt; Gavin Stewart,&lt;sup&gt;5&lt;/sup&gt; Christopher Eccleston,&lt;sup&gt;4,6,7&lt;/sup&gt; Neil E. O’Connell&lt;sup&gt;8&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective &lt;/h4&gt;
&lt;p&gt;Continued citation of retracted literature is a threat to scientific integrity. Most authors who cite retractions are unaware the article has been withdrawn.&lt;sup&gt;1&lt;/sup&gt; While the International Committee of Medical Journal Editors (ICMJE) requires authors to check that no references in a manuscript cite retracted articles,&lt;sup&gt;2&lt;/sup&gt; few journals include this requirement in author submission guidelines.&lt;sup&gt;3&lt;/sup&gt; This study evaluated the effect of an audit and feedback intervention on retraction checking instructions in the top 20 pain and top 20 anesthesiology journal author submission guidelines.&lt;/p&gt;
&lt;h4&gt;Design &lt;/h4&gt;
&lt;p&gt;This pre-post audit and feedback intervention evaluated author submission guidelines of the top 20 pain and top 20 anesthesiology journals (according to Journal Citation Reports 2022 Impact Factor) for (1) recommendations to check manuscript references for retractions and (2) guidance on how to identify retractions. Each item was judged as yes or no (1 or 0, respectively) and given a total score out of 2. The feedback intervention was targeted at the journal editors in chief via a single email contact. Based on the journal’s score, a tailored letter was constructed (for pain and anesthesiology journals separately) including the journal score, comparison with other journals evaluated, and guidance (including a template) to implement ICMJE recommendations within author guidelines. A follow-up audit was performed at 6 months. This study was approved by University of New South Wales ethics and registered on Open Science Framework on April 2, 2024.&lt;/p&gt;
&lt;h4&gt;Results &lt;/h4&gt;
&lt;p&gt;The baseline audit was performed in April 2024. The mean (SD) impact factor was 3.6 (1.5) for pain journals and 4.6 (2.6) for anesthesiology journals. Twelve of 20 pain journals and 9 of 20 anesthesiology journals were ICMJE signatories. At baseline, 0 of 20 pain journals and 3 of 20 anesthesiology journals included recommendations to check references for retractions and 0 of 20 pain journals and 1 of 20 anesthesiology journals provided guidance on how to identify retractions. At 6-month follow-up in October 2024, 3 of 20 pain journals and 5 of 20 anesthesiology journals included recommendations to check references for retractions and 3 of 20 pain journals and 3 of 20 anesthesiology journals provided guidance on how to identify retractions (&lt;b&gt;Figure 25-0881&lt;/b&gt;). Overall, this represented an absolute score change of 25%. Correspondence with editors of 2 pain journals and incorporation of the guidance template in 1 pain journal confirmed that these changes resulted from the feedback intervention.&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/c3e027bdf09d715677071bd08372e17c.png)

&lt;h4&gt;Conclusions &lt;/h4&gt;
&lt;p&gt;The number of pain and anesthesiology journals with author instructions to check references for retractions increased following a tailored audit and feedback intervention. However, overall uptake of the feedback was low, suggesting that more sustained editorial engagement strategies may be necessary to facilitate implementation of recommendations. Future studies should evaluate whether implementation of retraction screening recommendations into author submission guidelines reduces citation of retracted literature.&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. De Cassai A, Geraldini F, De Pinto S, et al. Inappropriate citation of retracted articles in anesthesiology and intensive care medicine publications. &lt;i&gt;Anesthesiology&lt;/i&gt;. 2022;137(3):341-350. doi:10.1097/ALN.0000000000004302&lt;/p&gt;
&lt;p&gt;2. International Committee of Medical Journal Editors. Preparing a manuscript for submission to a medical journal. Accessed January 15, 2025. &lt;a href=&quot;https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;3. Boudry C, Howard K, Mouriaux F. Poor visibility of retracted articles: a problem that should no longer be ignored. &lt;i&gt;BMJ&lt;/i&gt;. 2023;381:e072929. doi:10.1136/bmj-2022-072929&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Centre for Pain IMPACT, Neuroscience Research Australia, Sydney, New South Wales, Australia, m.ferraro@neura.edu.au; &lt;sup&gt;2&lt;/sup&gt;School of Health Sciences, Faculty of Medicine and Health, University of New South Wales, Sydney, New South Wales, Australia; &lt;sup&gt;3&lt;/sup&gt;Research Department of Clinical, Educational and Health Psychology, University College London, London, UK; &lt;sup&gt;4&lt;/sup&gt;Centre for Pain Research, University of Bath, Bath, UK; &lt;sup&gt;5&lt;/sup&gt;School of Natural and Environmental Sciences, University of Newcastle upon Tyne, Newcastle, UK; &lt;sup&gt;6&lt;/sup&gt;Department of Health and Clinical Psychology, the University of Ghent, Ghent, Belgium; &lt;sup&gt;7&lt;/sup&gt;Department of Psychology, The University of Helsinki, Helsinki, Finland; &lt;sup&gt;8&lt;/sup&gt;Department of Health Sciences, Centre for Wellbeing Across the Lifecourse, Brunel University of London, Uxbridge, UK.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures &lt;/h4&gt;
&lt;p&gt;Aidan G. Cashin, Amanda C. de C. Williams, Emma Fisher, and Christopher Eccleston are editorial board members for 2 of the pain journals (&lt;i&gt;PAIN, Journal of Pain&lt;/i&gt;) included in this audit. No other disclosures were reported.&lt;/p&gt;
&lt;h4&gt;Funding/Support &lt;/h4&gt;
&lt;p&gt;No specific funding was received for this project. Michael C. Ferraro was supported by an Australian Government Research Training Program PhD scholarship and a Neuroscience Research Australia top-up scholarship.&lt;/p&gt;
&lt;h4&gt;Role of the Funder/Sponsor &lt;/h4&gt;
&lt;p&gt;The sponsor, Neuroscience Research Australia, had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and decision to submit the manuscript.&lt;/p&gt;
&lt;h4&gt;Additional Information&lt;/h4&gt;
&lt;p&gt; This study was registered on Open Science Framework at &lt;a href=&quot;https://osf.io/gbkjy/&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://osf.io/gbkjy/&lt;/span&gt;&lt;/a&gt;.&lt;/p&gt;


The 10th Peer Review Congress

An audit and feedback intervention to reduce inappropriate citation of retracted literature in the pain and anaesthesiology fields

retractions

misconduct

research integrity

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Michael C. Ferraro,<sup>1,2</sup> Aidan G. Cashin,<sup>1,2</sup> Amanda C. de C. Williams,<sup>3</sup> Emma Fisher,<sup>4</sup> Gavin Stewart,<sup>5</sup> Christopher Eccleston,<sup>4,6,7</sup> Neil E. O’Connell<sup>8</sup></p>
<h4>Objective </h4>
<p>Continued citation of retracted literature is a threat to scientific integrity. Most authors who cite retractions are unaware the article has been withdrawn.<sup>1</sup> While the International Committee of Medical Journal Editors (ICMJE) requires authors to check that no references in a manuscript cite retracted articles,<sup>2</sup> few journals include this requirement in author submission guidelines.<sup>3</sup> This study evaluated the effect of an audit and feedback intervention on retraction checking instructions in the top 20 pain and top 20 anesthesiology journal author submission guidelines.</p>
<h4>Design </h4>
<p>This pre-post audit and feedback intervention evaluated author submission guidelines of the top 20 pain and top 20 anesthesiology journals (according to Journal Citation Reports 2022 Impact Factor) for (1) recommendations to check manuscript references for retractions and (2) guidance on how to identify retractions. Each item was judged as yes or no (1 or 0, respectively) and given a total score out of 2. The feedback intervention was targeted at the journal editors in chief via a single email contact. Based on the journal’s score, a tailored letter was constructed (for pain and anesthesiology journals separately) including the journal score, comparison with other journals evaluated, and guidance (including a template) to implement ICMJE recommendations within author guidelines. A follow-up audit was performed at 6 months. This study was approved by University of New South Wales ethics and registered on Open Science Framework on April 2, 2024.</p>
<h4>Results </h4>
<p>The baseline audit was performed in April 2024. The mean (SD) impact factor was 3.6 (1.5) for pain journals and 4.6 (2.6) for anesthesiology journals. Twelve of 20 pain journals and 9 of 20 anesthesiology journals were ICMJE signatories. At baseline, 0 of 20 pain journals and 3 of 20 anesthesiology journals included recommendations to check references for retractions and 0 of 20 pain journals and 1 of 20 anesthesiology journals provided guidance on how to identify retractions. At 6-month follow-up in October 2024, 3 of 20 pain journals and 5 of 20 anesthesiology journals included recommendations to check references for retractions and 3 of 20 pain journals and 3 of 20 anesthesiology journals provided guidance on how to identify retractions (<b>Figure 25-0881</b>). Overall, this represented an absolute score change of 25%. Correspondence with editors of 2 pain journals and incorporation of the guidance template in 1 pain journal confirmed that these changes resulted from the feedback intervention.</p>

![](https://assets.underline.io/markdown_image/1/image/c3e027bdf09d715677071bd08372e17c.png)

<h4>Conclusions </h4>
<p>The number of pain and anesthesiology journals with author instructions to check references for retractions increased following a tailored audit and feedback intervention. However, overall uptake of the feedback was low, suggesting that more sustained editorial engagement strategies may be necessary to facilitate implementation of recommendations. Future studies should evaluate whether implementation of retraction screening recommendations into author submission guidelines reduces citation of retracted literature.</p>
<h4>References</h4>
<p>1. De Cassai A, Geraldini F, De Pinto S, et al. Inappropriate citation of retracted articles in anesthesiology and intensive care medicine publications. <i>Anesthesiology</i>. 2022;137(3):341-350. doi:10.1097/ALN.0000000000004302</p>
<p>2. International Committee of Medical Journal Editors. Preparing a manuscript for submission to a medical journal. Accessed January 15, 2025. <a href="https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html"><span class="Hyperlink CharOverride-6">https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html</span></a></p>
<p>3. Boudry C, Howard K, Mouriaux F. Poor visibility of retracted articles: a problem that should no longer be ignored. <i>BMJ</i>. 2023;381:e072929. doi:10.1136/bmj-2022-072929</p>
<sup>1</sup>Centre for Pain IMPACT, Neuroscience Research Australia, Sydney, New South Wales, Australia, m.ferraro@neura.edu.au; <sup>2</sup>School of Health Sciences, Faculty of Medicine and Health, University of New South Wales, Sydney, New South Wales, Australia; <sup>3</sup>Research Department of Clinical, Educational and Health Psychology, University College London, London, UK; <sup>4</sup>Centre for Pain Research, University of Bath, Bath, UK; <sup>5</sup>School of Natural and Environmental Sciences, University of Newcastle upon Tyne, Newcastle, UK; <sup>6</sup>Department of Health and Clinical Psychology, the University of Ghent, Ghent, Belgium; <sup>7</sup>Department of Psychology, The University of Helsinki, Helsinki, Finland; <sup>8</sup>Department of Health Sciences, Centre for Wellbeing Across the Lifecourse, Brunel University of London, Uxbridge, UK.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Aidan G. Cashin, Amanda C. de C. Williams, Emma Fisher, and Christopher Eccleston are editorial board members for 2 of the pain journals (<i>PAIN, Journal of Pain</i>) included in this audit. No other disclosures were reported.</p>
<h4>Funding/Support </h4>
<p>No specific funding was received for this project. Michael C. Ferraro was supported by an Australian Government Research Training Program PhD scholarship and a Neuroscience Research Australia top-up scholarship.</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The sponsor, Neuroscience Research Australia, had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and decision to submit the manuscript.</p>
<h4>Additional Information</h4>
<p> This study was registered on Open Science Framework at <a href="https://osf.io/gbkjy/"><span class="Hyperlink CharOverride-6">https://osf.io/gbkjy/</span></a>.</p>


poster

<p>Jing Cui,<sup>1</sup> Nan Yang,<sup>1</sup> Kexin Ji,<sup>1</sup> Dingran Yin,<sup>1</sup> Chen Shen,<sup>2</sup> Zhaolan Liu,<sup>2</sup> Han Tan,<sup>3</sup> Yaxin Sun,<sup>4</sup> Zhaoqi Huo,<sup>5</sup> Shuo Liu,<sup>5</sup> Huiyu Wang,<sup>5</sup> Xintong Zhang,<sup>6</sup> Jing Guo,<sup>1</sup> Yufei Wang,<sup>7</sup> Xiaoqi Ren,<sup>5</sup> Vincent C. H. Chung,<sup>8</sup> Jianping Liu<sup>2</sup></p>
<h4>Objective</h4>
<p> Publication retractions, especially in the field of medicine, have long been a matter of significant concern.<sup>1</sup> The trends and characteristics of retractions of publications on Traditional Chinese medicine (TCM) by authors from mainland China remain unclear. This study is dedicated to exploring the prevalence and underlying reasons of retractions in TCM publication.</p>
<h4>Design </h4>
<p>This cross-sectional study identified and analyzed publications that were marked as retractions or withdrawal. It encompassed retractions of TCM literature published by mainland Chinese authors in international peer-reviewed journals from January 2000 to November 2024. Seven medical databases were systematically searched, including 3 English-language databases (PubMed, Embase, and Cochrane Library) and 4 Chinese-language databases (China National Knowledge Infrastructure, Wanfang Data, VIP Database, SinoMed), as well as the Retraction Watch Database. Duplicate data were excluded. Article screening and data extraction were independently conducted by 2 authors. Discrepancies were resolved through discussion. The retractions of TCM publications by mainland Chinese authors were thoroughly analyzed. The numbers and characters of retractions across different study types were compared, and retraction reasons were comprehensively summarized according to Committee on Publication Ethics (COPE) guidelines. Main outcomes included the prevalence, reasons, and relevant factors of retraction (year, authors’ region and affiliation, study types, study subjects, and reasons).</p>
<h4>Results </h4>
<p>A total of 24,479 retracted articles were initially identified, and 679 retracted publications on TCM met inclusion criteria. Retractions encompassed laboratory studies (381 [56.1%]), clinical studies (189 [27.8%]), and secondary studies (109 [16.1%]). Notably, the number of retractions surged to 354 articles (52.1%) in 2023 (<b>Figure 25-0930</b>). Most first authors were affiliated with Grade A tertiary hospitals (the highest accreditation level in China’s health care system; 415 [61.1%]). Active-ingredient extracts of TCM (289 [42.6%]) and neoplasms (145 [22.6%]) emerged as the most prevalent study subjects. A total of 77.6% of retractions were attributed to multiple reasons, so that the sum of the percentage of reasons may exceed 100%. A total of 12 retracted articles had no specific reasons found. Author misconduct drove 576 retractions (86.4%), predominantly involving peer review manipulation (374 [56.1%]), falsification or fabrication of data (262 [39.3%]), and paper mills (244 [36.6%]).</p>

![](https://assets.underline.io/markdown_image/1/image/1dd3a7c51afdb9fddf1603eab8ac2367.png)

<h4>Conclusions </h4>
<p>Currently, retractions of TCM publications have become increasingly conspicuous, particularly those from higher-level academic institutions and hospitals. Systemic author misconduct, notably compromised peer review processes and industrialized fraud, may constitute the primary driver. However, some publishers regularly remove retracted literature from the platform, which results in omissions in the data. To promote scientific credibility in TCM publications, we recommend (1) enhanced institutional accountability mechanisms, (2) standardized retraction policies aligned with COPE guidelines, (3) mandatory research integrity training, and (4) transparent postretraction investigations.<sup>2</sup></p>
<h4>References</h4>
<p>1. Else H. Biomedical paper retractions have quadrupled in 20 years—why? <i>Nature</i>. 2024;630(8016):280-281. doi:10.1038/d41586-024-01609-0</p>
<p>2. Bakker CJ, Reardon EE, Brown SJ, et al. Identification of retracted publications and completeness of retraction notices in public health. <i>J Clin Epidemiol</i>. 2024;173:111427. doi:10.1016/j.jclinepi.2024.111427</p>
<sup>1</sup>Qi-Huang Chinese Medicine School, Beijing University of Chinese Medicine, Beijing, China; <sup>2</sup>Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China, liujp@bucm.edu.cn; <sup>3</sup>School of Acupuncture-Moxibustion and Tuina, Beijing University of Chinese Medicine, Beijing, China; <sup>4</sup>School of Management, Beijing University of Chinese Medicine, Beijing, China; <sup>5</sup>School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China; <sup>6</sup>School of Humanities, Beijing University of Chinese Medicine, Beijing, China; <sup>7</sup>College of Continuing Education, Hebei University of Chinese Medicine, Shijiazhuang, China; <sup>8</sup>Jockey Club School of Public Health and Primary Care, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> High-level Traditional Chinese medicine key subjects construction project of National Administration of Traditional Chinese Medicine—Evidence-Based Traditional Chinese Medicine (award zyyzdxk-2023249).</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The sponsor contributed to the decision to submit the abstract for presentation.</p>
<h4>Additional Information</h4>
<p> Jing Cui and Nan Yang contributed equally to this work, and Kexin Ji and Dingran Yin contributed equally to this work. Vincent C. H. Chung is a co–corresponding author (vchung@cuhk.edu.hk).</p>


Prevalence of and Reasons for Retractions on Traditional Chinese Medicine Research Publications by Authors from Mainland China in International Peer-review Journals

<p>Yagmur Ozturk,<sup>1</sup> Frédérique Bordignon,<sup>2,3</sup> Cyril Labbé,<sup>1</sup> François Portet<sup>1</sup></p>
<h4>Objective</h4>
<p> To address the challenge of tracking the use of unreliable sources in scientific literature, we analyzed citations to retracted articles (CRAs) using retraction reasons (RRs) and citation context (CC) analysis. Our aim was to inform the development of automated tools for postpublication peer review (PPPR) by identifying instances in which a CRA is not only an unreliable source, but may also introduce unreliability into the citing work itself, depending on the way it is cited. We examined the degree to which the content of the citation context aligns with the RR, indicating potential risk for the integrity of the citing article.</p>
<h4>Design</h4>
<p> RRs were obtained from the Retraction Watch database<sup>1</sup> for a set of 22,558 publications that cite retracted sources accessed from the Problematic Paper Screener.<sup>2</sup> We selected 29 of 108 retraction reasons that directly relate to scientific content of the article (eg, unreliable or fabricated data or results). To ensure access to CCs, we limited our dataset to open access citing articles and excluded those that had themselves been retracted. This yielded 4979 citing articles and 7424 unique CRA instances; 88% of these were preretraction citations. We then limited the rest of the analysis to a subset of 3049 CCs from the PubMed Central Open Access corpus.<sup>3</sup> We used regular expressions to search for terms referencing concepts related to the retraction reasons (eg, “data,” “results,” “findings”) and manually validated their use in the CCs.</p>
<h4>Results</h4>
<p> Of 3049 CCs, we identified 57 that explicitly mentioned the exact issue mentioned in the RR. Among these, 12 emphasized similarity or consistency with the retracted findings, which are instances we interpret as potential indicators of unreliability in the citing paper, meriting PPPR. These 57 CCs span 18 research fields and involve 60 distinct RRs (some overlapping with the selected 29). The cases are distributed across a range of publishers and fields, offering a basis to be used in the development of automatic systems.</p>

![](https://assets.underline.io/markdown_image/1/image/c740d6d31d9977a9f35654827d4b160b.png)

<h4>Conclusions</h4>
<p> While retracted publications are unreliable sources, we argue that how the retracted work is used and why it was retracted matter more than the fact that a retracted reference is found in a publication. If the retracted source is used to support the citing work, this calls for a secondary assessment when it is detected. By combining RRs with CC analysis, our approach informs a practical method to flag such cases for PPPR and supports the development of tools for large-scale CRA assessment.</p>
<h4>References</h4>
<p>1. The Center for Scientific Integrity. Retraction Watch Database.<a href="https://retractiondatabase.org/"><span class="Hyperlink CharOverride-6"> </span></a><a href="https://retractiondatabase.org/"></a></p>
<p>2. Cabanac G, Labbé C, Magazinov A. The ‘Problematic Paper Screener’ automatically selects suspect publications for post-publication (re)assessment. <i>arXiv</i>. Preprint posted online October 7, 2022. doi:10.48550/arXiv.2210.04895</p>
<p>3. National Library of Medicine. PMC Open Access Subset. Accessed January 30, 2025. <a href="https://pmc.ncbi.nlm.nih.gov/tools/openftlist/"><span class="Hyperlink CharOverride-6">https://pmc.ncbi.nlm.nih.gov/tools/openftlist/</span></a></p>
<sup>1</sup>Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, Grenoble, France, yagmur.ozturk@univ-grenoble-alpes.fr; <sup>2</sup>Ecole Nationale des Ponts et Chaussées, Institut Polytechnique de Paris, Marne-la-Vallée, France; <sup>3</sup>LISIS, INRAE, Univ Gustave Eiffel, CNRS, Marne-la-Vallée, France.</p>
<h4><a id="_idTextAnchor124"></a>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> We acknowledge the NanoBubbles project that has received Synergy grant funding from the European Research Council within the European Union’s Horizon 2020 program (grant agreement number 951393) (<a href="https://cordis.europa.eu/project/id/951393"><span class="Hyperlink CharOverride-6">https://cordis.europa.eu/project/id/951393</span></a>).</p>


Citation Context Analysis of Retracted Articles: Leveraging Retraction Reasons to Track Unreliability in Citing Literature

<p>Caitlin J. Bakker,<sup>1,2</sup> Erin E. Reardon,<sup>3</sup> Nicole Theis-Mahon,<sup>4</sup> Sara Schroter,<sup>5,6</sup> Lex Bouter,<sup>7,8</sup> Maurice P. Zeegers<sup>2</sup></p>
<h4>Objective</h4>
<p> Systematic reviews gather, appraise, and synthesize studies to inform research, practice, and policy. However, the inclusion of retracted articles, which often contain flaws and falsified or fabricated data, undermines the credibility of systematic reviews. Identifying retracted articles is challenging, as they are inconsistently flagged.<sup>1,2</sup> Our study validated and compared approaches to identify retracted articles and their retraction notices.</p>
<h4>Design</h4>
<p> Our study, guided by an advisory panel of information specialists and researchers, evaluated approaches for identifying retracted publications from 8 health sciences databases (Cochrane Library, Embase.com, Ovid Embase, Ovid Medline, Ovid PsycINFO, PubMed, Scopus, and Web of Science). Using a reference set of 43,544 retracted publications and 27,755 associated retraction notices from Retraction Watch, we identified items found in each database. From August 10 to 14, 2024, we determined how many of the available items could be retrieved using each approach per database. Two search strategies, database indexing, and 2 citation managers were evaluated. The complete methodology, including search strategies, is available in our protocol.<sup>3</sup> Recall (sensitivity) was calculated to evaluate identification effectiveness.</p>
<h4>Results</h4>
<p> Recall of retracted publications and notices varied across databases and retrieval approaches. Across databases, search strategy 2 consistently achieved the highest recall, with values ranging from 74.6% to 96.9%. Search strategy 1 also performed strongly, particularly in PubMed and Web of Science (both &gt;93%). In contrast, indexing-based retrieval showed variable performance, with high recall in PubMed (94.6%) and Ovid Medline (94.5%) but much lower in Embase.com (40.6%) and PsycINFO (34%). Citation manager tools (EndNote and Zotero) yielded lower recall, with values rarely exceeding 64%. Recall was lowest for Ovid Embase and PsycINFO regardless of method, while PubMed and Web of Science showed the highest recall.</p>

![](https://assets.underline.io/markdown_image/1/image/b3957ae8a92f24f08736a44da92e7d56.png)

<h4>Conclusions</h4>
<p> There was substantial variability in the ability of databases and retrieval approaches to identify retracted publications and notices. No single approach captured all items, underscoring the need for multiple approaches in an iterative identification process. </p>
<h4>References</h4>
<p>1. Bakker CJ, Reardon EE, Brown SJ, et al. Identification of retracted publications and completeness of retraction notices in public health. <i>J Clin Epidemiol</i>. 2024;173:111427. doi:10.1016/j.jclinepi.2024.111427</p>
<p>2. Boudry C, Howard K, Mouriaux F. Poor visibility of retracted articles: a problem that should no longer be ignored. <i>BMJ</i>. 2023;381:e072929. doi:10.1136/bmj-2022-072929</p>
<p>3. Bakker C, Reardon EE, Theis-Mahon NR, Schroter S, Bouter L, Zeegers M. Validation and comparison of methods to identify retracted publications during information retrieval. Open Science Framework. Cited June 7, 2025. <a href="https://osf.io/rwzym/"><span class="Hyperlink CharOverride-6">https://osf.io/rwzym/</span></a></p>
<sup>1</sup>University of Regina, Regina, SK, Canada, caitlin.bakker@uregina.ca; <sup>2</sup>Maastricht University, Maastricht, the Netherlands; <sup>3</sup>Emory University, Atlanta, GA, US; <sup>4</sup>University of Minnesota, Minneapolis, US; <sup>5</sup>BMJ, London, UK; <sup>6</sup>London School of Hygiene and Tropical Medicine, London, UK; <sup>7</sup>Amsterdam University Medical Center, Amsterdam, the Netherlands; <sup>8</sup>Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Caitlin J. Bakker is cochair of the National Information Standards Organization Communication of Retractions, Removals and Expressions of Concern Standing Committee. No other disclosures were reported.</p>
<h4>Funding/Support </h4>
<p>This research is part of an ongoing PhD collaboration between BMJ and the team Meta-Research at Maastricht University (UM) on the responsible conduct of publishing scientific research. BMJ is published by BMJ Group, a wholly owned subsidiary of the British Medical Association. UM is a public legal entity in the Netherlands. This study is part of Caitlin J. Bakker’s self-funded BMJ/UM PhD. No exchange of funds has taken place for this research project.</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The authors are wholly responsible for the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the abstract; and decision to submit the abstract for presentation.</p>
<h4>Additional Information</h4>
<p> All authors express their own opinions and not necessarily that of their employers.</p>


Evaluating Approaches for Identifying Retracted Articles and Retraction Notices in Systematic Review Searching

<p>Guillaume Cabanac,<sup>1,2</sup> Kathryn Weber-Boer<sup>3</sup></p>
<h4>Objective</h4>
<p> We analyzed postretraction citations of retracted publications in biomedical and clinical sciences (BCS). Previous studies have investigated the citation of retracted clinical trials<sup>1</sup> and the citation of retracted publications in policy documents,<sup>2</sup> but the share of references cited after their retraction in BCS literature is unknown. We identified the articles citing postretracted references and present the results aggregated at the journal and publisher levels.</p>
<h4>Design</h4>
<p> We used Dimensions data accessed in BigQuery to delineate the corpus, including publications of type ‘article’, classified as “review article” or “research article”, with the field of research category BCS (32, in the ANZSRC 2020 classification), published between 2005 and 2024. We used the snapshot of the Dimensions dataset from 1 June 2025. These 12,240,884 articles were published in 44,901 journals from 11,961 publishers, and contained 401,909,282 references of 28,058,673 unique publications. We extracted the year of publication, journal titles, publisher names, and list of references cited. We retained those references which were removed, retracted, or withdrawn (henceforth ‘retracted’), based on data derived from Crossref, PubMed, Retraction Watch, and retraction notices issued by publishers. We excluded publications which cited retractions which had a retraction year earlier than or equal to the year of publication, to consider only “post-retraction references.”</p>
<h4>Results</h4>
<p> We observed a growing number of publications in the BSC literature that cite retracted publications. In 2005, there were 674 publications referencing 638 retracted publications 688 times. This grew to 13,927 publications referencing 14,214 retracted publications 15,339 times in 2024: a 2066% increase in publications and retracted references and a 2228% increase in retracted publications cited. This compared to an increase over the same period of 267% for publications in BSC (from 368,338 to 983,837) and 390% in citations (8,495,152 to 33,143,231). This is an increase in the proportion of publications with retracted references from to 0.18% to 1.69% of all BSC publications. The number of retracted publications referenced doubled since 2019. The connection between journal and publisher reflected the ownership relationship as of approximately January 2025. The journals in which post-retraction referencing of retracted literature occurred are many (7225 distinct journals, 17%), but over half of the publications (57,292 of 110,492) are currently owned by four major publishers: Elsevier (23,725; 21%), Springer Nature (17,727; 16%), Wiley (9488; 9%), and Frontiers (6352; 6%).<b>(Table 25-1151)</b></p>
<a href="/wp-content/uploads/2025_conference/tables/prc25-1151t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1151t1.png"> </a>
<h4>Conclusions</h4>
<p> The publishers of articles responsible for the most citations of postretracted references are the largest publishers of scientific literature by any measure. We could have ranked or displayed the results by publisher or journal as a percentage of publications overall; however, in this case, quantity matters more than share. This practice is preventable, with system-wide corrective efforts by the publishing industry, such as prepublication checks for retracted references, as a complement to the postpublication checks of retracted references that are also needed.<sup>3</sup></p>
<h4>References</h4>
<p>1. Kataoka Y, Banno M, Tsujimoto Y, et al. Retracted randomized controlled trials were cited and not corrected in systematic reviews and clinical practice guidelines. <i>J Clin Epidemiol</i>. 2022;150:90-97. doi:10.1016/j.jclinepi.2022.06.015</p>
<p>2. Malkov D, Yaqub O, Siepel J. The spread of retracted research into policy literature. <i>Quant Sci Stud</i>. 2023;4(1):68-90. doi:10.1162/qss_a_00243</p>
<p>3. Cabanac G. Chain retraction: how to stop bad science propagating through the literature. <i>Nature</i>. 2024;632(8027):977-979. doi:10.1038/d41586-024-02747-1</p>
<sup>1</sup>Université de Toulouse, IRIT (UMR 5505 CNRS), Toulouse, France; <sup>2</sup>Institut Universitaire de France, Paris, France; <sup>3</sup>Digital Science, London, UK, k.weberboer@digital-science.com.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Kathryn Weber-Boer is employed by Digital Science, which provided access to the Dimensions datasets that this work analyzed. Guillaume Cabanac received funding from the Institut Universitaire de France.</p>
<h4>Additional Information </h4>
<p>Guillaume Cabanac and Kathryn Weber-Boer are co–corresponding authors.</p>


Post-retraction References in Biomedical and Clinical Sciences

<p>Lonni Besançon,<sup>1</sup> Guillaume Cabanac,<sup>2,3</sup> Kathryn Weber-Boer<sup>4</sup></p>
<h4>Objective </h4>
<p>We identified retracted references in clinical guidelines published between 1991 and 2025 that were cited before and after retraction to study the scale of this contamination. Clinical guidelines have a concrete impact on the practice of medicine and, while active, are meant to reflect the state of the art. Previous work<sup>1</sup> has explored the citation of retracted trials in systematic reviews and clinical guidelines, showing both pre- and postretraction citations. We explored a population of retracted publications cited in clinical guidelines.</p>
<h4>Design </h4>
<p>We drew on 2 new datasets: clinical guidelines from Altmetric (n = 20,553) and retractions identified by Dimensions (n = 63,047). The clinical guidelines dataset<sup>2</sup> (articles indexed in Dimensions classified as clinical guidelines and documents published online by learned societies and associations) should consist of the latest version of each guideline, although as guidelines are updated, the guidelines that we studied may become obsolete. This dataset included links to guidelines and a list of publications referenced in each guideline. The retractions dataset was derived from CrossRef, PubMed, and Retraction Watch, and by matching publisher notices to publications; it included an identifier for the retracted publication and relevant linked notices with their type and date. We identified publications found both in the list of guideline references and in the list of retracted publications and selected those cited after their retraction date (398 unique guideline-retracted publication pairs). We manually examined a randomly selected 10% of the accessible English-language publications to assess the accuracy of the dataset (42 guideline-reference pairs). Our evaluation of this sample of our dataset assessed both whether the retraction identification was accurate and whether the citation of these publications in the guidelines was critical (or mentioned the retraction).</p>
<h4>Results </h4>
<p>We considered publications cited in the year of retraction to be preretraction citations. We found 487 guidelines containing 596 references to 319 publications that had been retracted after citation. Of these, 179 guidelines (37%) referenced 134 retracted publications 204 times within 5 years after retraction (<b>Figure 25-1004</b>). There were 107 guidelines (22%) containing 126 references to 73 publications that had been retracted more than 5 years before citation. All retraction datasets, including ours, contained inaccuracies. We could not find retraction notices for 4 references marked as retracted. Only 2 retracted references were cited critically (presented with some doubt as to the strength of their methods or conclusions), and none acknowledged the retracted status of the reference. Most references were used either as background literature or to support claims and recommendations made in the guidelines.</p>

![](https://assets.underline.io/markdown_image/1/image/ecbf789bf3acbffc6717850847a3a40b.png)

<h4>Conclusions </h4>
<p>Postretraction citations are preventable. The longer after the retraction appears, the more concerning the citation. In guidelines in particular, the persistence of retracted references is concerning. Our results highlight the need to systematically screen both new and existing clinical guidelines for contamination.</p>
<h4>References</h4>
<p>1. Kataoka Y, Banno M, Tsujimoto Y, et al. Retracted randomized controlled trials were cited and not corrected in systematic reviews and clinical practice guidelines. <i>J Clin Epidemiol</i>. 2022;150:90-97. doi:10.1016/j.jclinepi.2022.06.015</p>
<p>2. Altmetric. Clinical guidelines. Accessed June 19, 2025. <a href="https://www.altmetric.com/about-us/our-data/clinical-guidelines/">https://www.altmetric.com/about-us/our-data/clinical-guidelines/</a></p>
<sup>1</sup>Linköping University, Linköping, Sweden; <sup>2</sup>Université de Toulouse, Institut de Recherche en Informatique de Toulouse (UMR 5505), Centre National de la Recherche Scientifique, Toulouse, France; <sup>3</sup>Institut Universitaire de France, Paris, France; <sup>4</sup>Digital Science, London, UK, k.weberboer@digital-science.com.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Kathryn Weber-Boer is employed by Digital Science, which provided access to the Altmetric and Dimensions datasets that this work analyzed. Lonni Besançon has received a catalyst grant from Digital Science. No other disclosures were reported.</p>


Retracted Publications Referenced in Clinical Guidelines

<p>Ivan D. Florez,<sup>1,2</sup> Alberto Henriquez,<sup>3,4</sup> Andrés F. Estupinan-Bohorquez<sup>3,5</sup></p>
<h4>Objective </h4>
<p>We described the influence of retracted systematic reviews and meta-analyses (SRMAs) on clinical practice guidelines (CPGs) and the characteristics of retractions in CPGs.</p>
<h4>Design </h4>
<p>This cross-sectional study was conducted in 2 stages based on searches focused on the Retraction Watch (RW) database and MEDLINE from inception to November 30, 2024. In the first stage, we included SRMAs. We described the reasons for retractions, and we recategorized them based on our assessment. We identified the CPGs that cited the SRMA in the Google Scholar database. In the second stage, we included the retracted CPGs, described the reasons for retractions, and recategorized them into ethical or nonethical reasons based on our assessment. Nonethical reasons were categorized as editorial or administrative or outdated guidelines, while ethical reasons were reported according to RW categories. We used descriptive statistics to summarize the findings.</p>
<h4>Results </h4>
<p>In the first stage, we included 377 SRMAs, of which 211 (56.0%) were retracted due to peer review or publication manipulation (eg, detected “fake” reviewers); 30 (8.0%), due to duplicate or redundant publication; and 136 (36.1%), due to intellectual or authorship disputes, plagiarism, outdated publication, retraction of included studies, methodologic or data errors, and conflicts of interests. For 49 (13.0%), specific reasons were not provided. Of the retracted SRMAs, 41 (10.9%) were cited in CPGs; 19 (46.3%) of these SRMAs were retracted due to research integrity issues and 12 (29.3%), due to data errors or being outdated. For 10 (24.4%), specific reasons were not provided. Most retractions were due to manipulation of the publication or peer review process. The median time between publication and retraction of the SRMA used in CPGs was 12.0 (IQR, 3.5-25.0) months, and the median number of SRMA citations was 40 (IQR, 22-191). In the second stage, we included 36 CPGs of the 138 potential CPGs identified. Nine CPGs (25.0%) were retracted because of ethical reasons and 22 (61.1%) for nonethical reasons; the rest had no available information. The most common ethical reasons were plagiarism, authorship or intellectual property disputes, lack of disclosure of conflicts of interest, and discrepancies between the content and the cited evidence. Among the 22 CPGs retracted for nonethical reasons, 11 were due to dual publication or incorrect citations and 9 were due to outdated recommendations. The median publication to retraction time was 10 (range, 3-96) months. All of these CPGs were cited after their retraction date, and in all cases, the citations were used to support the background of the research studies.</p>
<h4>Conclusions </h4>
<p>Retracted SRMAs have been informing CPGs, which provide recommendations in practice and policy. The retraction of CPGs has been neglected. The RW database should be revised according to the specificities of CPGs. The most concerning reasons are ethical. Retracted CPGs continue to be cited after their retraction, mainly to inform the background sections of articles.</p>
<sup>1</sup>Department of Pediatrics, University of Antioquia, Medellín, Colombia, ivan.florez@udea.edu.co; <sup>2</sup>School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada; <sup>3</sup>Universidad del Norte, Barranquilla, Colombia; <sup>4</sup>Universidad Metropolitana, Barranquilla, Colombia; <sup>5</sup>EPICLINICA SAS, Barranquilla, Colombia.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Retraction of Systematic Reviews and Clinical Practice Guidelines

<p>Caden Seraphine,<sup>1</sup> Abigail Chambers,<sup>1</sup> Susan Galandiuk<sup>2</sup></p>
<h4>Objective</h4>
<p> In recent years, <i>Diseases of the Colon &amp; Rectum</i> (<i>DCR</i>), a single society-owned specialty journal, sought to grow its online presence. The journal launched a YouTube channel in 2018, featuring videos of monthly English-language journal clubs (JCs) discussing topical publications beginning in 2020, followed by article discussions in Spanish (2022-2023), Japanese (2024), and Korean (2024). The journal maintained its Facebook and X (formerly Twitter) presence and started LinkedIn and Instagram accounts in 2023. Articles were given a nonpaywalled status for 1 month, and upcoming discussions with article hyperlinks were announced 4 times per month via social media. Articles were accessible by hyperlinks, and article citations were displayed in JC discussant slides. We sought to explore how these activities impacted measures of journal interaction and global reach.<sup>1,2</sup></p>
<h4>Design</h4>
<p> We performed a 7-year (2018-2024) retrospective cohort study of JC video-based educational content provided by <i>DCR</i> with respect to journal website and social media access following STROBE guidelines.<sup>3</sup> We examined efforts toward global engagement, including addition of journal-specific non–English-language (2 Spanish, 1 Korean, and 1 Japanese) discussions. Analyses of 4 popular social media platforms’ native analytics reporting and website engagement data examined viewer demographics (age, language, viewing country, and traffic source) and journal web-based interaction (number of views, number of unique visitors, and country of access) by month and year, trends in individual JC videos, and overall page engagement. We compared the median number of citations of JC articles (cumulative Web of Science citations) and their Altmetric scores (indicating article attention received) with mean citations of <i>DCR</i> articles per year.</p>
<h4>Results</h4>
<p> Only 1 JC featured videos of surgical procedures; all others discussed original articles. The growth in YouTube engagement since the journal’s YouTube channel launched in 2018 grew exponentially with the addition of monthly JCs in 2020. LinkedIn and Instagram were implemented in 2023, and cumulative traffic to the journal’s website from social media platforms tripled within 1 year, from 15,414 clicks in 2023 to 56,609 in 2024 (Facebook, X, Instagram, LinkedIn, and YouTube). Throughout the period, individuals aged 25 to 34 years led YouTube viewership. International engagement increased with <i>DCR</i>’s implementation of non-English JC discussions in 2022. Publications discussed during JCs consistently had higher Web of Science citations compared with the mean citations in the journal that year (<span class="CharOverride-4"><b>Table 25-0959</b></span>).</p>

![](https://assets.underline.io/markdown_image/1/image/dd454d0ce367b165c77b22150974e865.png)

<h4>Conclusions</h4>
<p> Younger audiences engage beyond traditional platforms of scholarly publishing in ways that hold inherent educational value. Journal Impact Factors exclude this reach and overlook the impact of web-based peer-reviewed educational content. Inclusion of social media and video-based learning tools, as well as efforts toward global reach, contribute to journal impact.</p>
<h4>References</h4>
<p>1. Narayan RR, Fleming AM, Gunder M, et al. Reflections from the Annals of Surgical Oncology social media committee: the impact of promoting surgical science online. Ann Surg Oncol. 2025;32(2):656-664. doi:10.1245/s10434-024-16420-4</p>
<p>2. Özkent Y. Social media usage to share information in communication journals: an analysis of social media activity and article citations. PLoS One. 2022;17(2):e0263725. doi:10.1371/journal.pone.0263725</p>
<p>3. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008</p>
<sup>1</sup>Price Institute of Surgical Research, University of Louisville, Louisville, KY, US; <sup>2</sup>Division of Colorectal Surgery, Department of Surgery, University of Louisville, Louisville, KY, US, s0gala01@louisville.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Susan Galandiuk receives a stipend for services as editor for the journal Diseases of the Colon &amp; Rectum from the American Society of Colon and Rectal Surgeons. No other disclosures were reported.</p>
<h4>Acknowledgments</h4>
<p> Margaret Abby (Managing Editor, Diseases of the Colon &amp; Rectum; Research Grants Coordinator, Department of Surgery, University of Louisville, Louisville, KY), helped with obtaining data on articles discussed during journal clubs and their Web of Science citations.</p>


Video and Social Media Performance at a Surgical Journal with Video Journal Clubs

<p>Jeremy Y. Ng,<sup>1,2,3</sup> Daivat Bhavsar,<sup>4</sup> Laura Duffy,<sup>4</sup> Hamin Jo,<sup>4</sup> Cynthia Lokker,<sup>4</sup> R. Brian Haynes,<sup>4,5</sup> Alfonso Iorio,<sup>4,5</sup> Ana Marušić<sup>6</sup></p>
<h4>Objective </h4>
<p>This study examined the policies implemented by academic publishers regarding authors’ use of generative artificial intelligence (GenAI) tools, focusing on their regulation, disclosure requirements, and role in ensuring the integrity of scientific publications. By analyzing the prevalence and content of these policies, this study aimed to provide insight into the current landscape and inform future policy development in the rapidly evolving field of artificial intelligence (AI)–assisted research and publication.</p>
<h4>Design </h4>
<p>A cross-sectional audit was conducted on the publicly available policies of 163 academic publishers listed as members of the International Association of Scientific, Technical, and Medical Publishers. Policies were collected and analyzed between September 1 and December 31, 2023. Publishers without publicly accessible policies specific to GenAI use by authors were excluded. Data extraction and analysis were conducted independently in duplicate, with a third reviewer resolving discrepancies. The key policy components analyzed included authorship accreditation, disclosure requirements, and permissions for tasks such as research methods, content generation, image creation, and proofreading. Descriptive statistics were used to summarize the findings. Our protocol was registered.<sup>1</sup></p>
<h4>Results </h4>
<p>Of 163 academic publishers, 56 (34.4%) had publicly available policies guiding GenAI use by authors. None permitted authorship accreditation for AI tools, citing accountability concerns and alignment with ethical guidelines. Nearly all publishers with policies (49 of 56 [87.5%]) mandated disclosure of GenAI use, primarily in the Methods or Acknowledgments section. However, disclosure practices varied, with some publishers providing standardized templates while others left requirements vague. Four publishers completely prohibited GenAI use in manuscript preparation, while others allowed their use for specific tasks. Most (33 of 56 [58.9%]) publishers permitted GenAI for drafting nonmethodological sections (eg, Introductions), while 18 (32.1%) permitted their use in research methods, such as data analysis and organization. Few publishers addressed GenAI use in image generation (14 of 163 [8.6%]) or proofreading (15 of 163 [9.2%]). Only 1 publisher (0.6%) allowed citation of AI as primary sources, while 19 (11.6%) explicitly prohibited such citations. Our study has been published.<sup>2</sup></p>
<h4>Conclusions </h4>
<p>This audit highlights the inconsistent development of GenAI policies among academic publishers, with large variability in scope and clarity. While the prohibition of AI authorship and the emphasis on mandatory disclosure are consistent themes, inconsistencies in regulating specific tasks suggest a need for standardized and comprehensive policies. As AI technology and its applications in research evolve, publishers must adapt to safeguard scientific integrity. Given that the AI landscape is fast moving, future research includes updating this audit and comparing and contrasting current policies with those found in this study. Future research should also assess how policies are implemented and enforced by examining samples of published articles, as well as explore how these policies affect editors and reviewers, taking into account potential risks, such as privacy breaches and bias.</p>
<h4>References</h4>
<p>1. Bhavsar D, Lokker C, Haynes RB, Iorio A, Marusic A, Ng JY. Academic publisher artificial intelligence chatbot policies for authors: a cross-sectional audit. OSF Registries. Accessed July 11, 2025. doi:10.17605/OSF.IO/937ES</p>
<p><span lang="fr-FR">2. Bhavsar D, Duffy L, Jo H, et al. Policies on artificial intelligence chatbots among academic publishers: a cross-sectional audit. </span><i>Res Integr Peer Rev</i><span lang="fr-FR">. 2025;10(1):1. </span>doi:10.1186/s41073-025-00158-y</p>
<sup>1</sup>Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany, jeremyyng.phd@gmail.com; <sup>2</sup>Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany; <sup>3</sup>Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada; <sup>4</sup>Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada; <sup>5</sup>Department of Medicine, McMaster University, Hamilton, Ontario, Canada; <sup>6</sup>Department of Research in Biomedicine and Health and Center for Evidence-Based Medicine, University of Split School of Medicine, Split, Croatia.</p>
<h4>Conflicts of Interest Disclosures </h4>
<p>The authors declare no conflicts of interest. Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>


[V] Policies on Artificial Intelligence Among Academic Publishers

<p>Eva Sarachaga,<sup>1</sup> Ines Steffens<sup>1</sup></p>
<h4>Objective </h4>
<p>In September 2023, <i>Eurosurveillance</i> adopted an artificial intelligence (AI) policy encouraging authors and reviewers to use AI tools responsibly. The policy states that authors should disclose tools such as large language models (LLMs), chatbots, and image-generating algorithms used in the production and writing of the manuscript. It further outlines that reviewers should declare whether they have used a chatbot or LLM tool in the generation of their review or in their correspondence to authors or editors.<sup>1</sup> Fifteen months after launching the policy, we investigated how many authors and reviewers had declared using AI tools and how. Additionally, we examined whether possible AI use was flagged by the plagiarism detection system, iThenticate, which checks whether incoming submissions overlap with published material.</p>
<h4>Design </h4>
<p>Data were extracted from the article submission system, Editorial Manager. In a cross-sectional design, we included all articles and reviews in the system since the policy was published in September 2023 until November 2024. Results were analyzed by domain, AI tool, and tool version. To investigate whether results from iThenticate differed between articles where AI use was declared versus where it was not declared, we compared scores of a convenience sample of 43 articles in each group and checked manually thereafter.</p>
<h4>Results </h4>
<p>Only 7% of authors (70 of 961) and 2% of reviewers (11 of 707) declared using AI tools. Language was the most frequent domain in both groups: 61 of 70 authors and 10 of 11 reviewers. Authors declared other domains, such as coding, data analysis, prediction identification, correspondence, and figure creation. ChatGPT was the most used AI tool in both groups: 54 of 70 authors and 6 of 11 reviewers. The second most used AI tools were DeepL for authors (2 of 70) and Paperpal for reviewers (2 of 11). Our results were in line with results from other studies.<sup>2,3</sup> AI use was not declared according to the policy: only 20% of authors (13 of 70) and 20% of reviewers (2 of 11) declared the tool version as required. The plagiarism score was lower in articles where AI use was declared (35 of 43) versus where it was not declared (39 of 43).</p>
<h4>Conclusions </h4>
<p>In line with findings by others, only a limited proportion of authors and reviewers declared using AI tools following implementation of an AI policy. The proportion was similar for authors and reviewers, with language being the most frequent domain and ChatGPT the most used tool in both groups. Authors and reviewers need to be reminded to comply better with the AI policy and to declare tool version.</p>
<h4>References</h4>
<p>1. E<span class="text-italic">urosurveillance. </span>Editorial policy: Responsible use of artificial intelligence (AI) tools. Accessed July 10, 2025. <a id="_idTextAnchor012"></a><a href="https://www.eurosurveillance.org/editorial-policy"><span class="Hyperlink CharOverride-6">https://www.eurosurveillance.org/editorial-policy#AI%20policy</span></a></p>
<p>2. Salvagno M, De Cassai A, Zorzi S, et al. The state of artificial intelligence in medical research: a survey of corresponding authors from top medical journals. <i>PLOS One</i>. 2024;19(8):e0309208. doi:10.1371/journal.pone.0309208</p>
<p>3. Else H. Should researchers use AI to write papers? Group aims for community-driven standards. <em>Science</em>. 2024;384(6693). doi:10.1126/science.z9gp5zo</p>
<sup>1</sup>European Centre for Disease Prevention and Control (ECDC), Stockholm, Sweden, evasarachaga01@gmail.com.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>
<h4>Acknowledgment</h4>
<p> We would like to thank the Eurosurveillance editorial team (Alina Buzdugan, Anirban Dey, Kathrin Hagmaier, Megan Osler, Elina Tast-Lahti, and Karen Wilson), with special thanks to Kathrin Hagmaier, at the ECDC who provided their feedback on this work.</p>


[V] Use of Generative Artificial Intelligence Tools by Authors and Reviewers of of the Journal Eurosurveillance

<p>Pawin Taechoyotin,<sup>1</sup> Daniel E. Acuna<sup>1</sup></p>
<h4>Objective </h4>
<p>Nowadays, there is an excessive load on human reviewers to the point where the quality of peer reviews is often compromised. To alleviate this excessive load, we explored the possibility of a multiagent AI peer review panel that we have developed that considers text, figures, and citations to produce peer reviews. This study is based on the advancement of large language models<sup>1,2</sup> and the study of AI agents for peer review,<sup>3</sup> which so far only utilize the textual content within the manuscript.</p>
<h4>Design</h4>
<p> The AI panel consisted of a leader agent, an experiment agent, an impact agent, and a clarity agent. Each agent utilized zero-shot prompting to the LLM model Claude Sonnet 3.5. Prompt engineering was used to craft the system and task prompt. The system prompts guided the agents to assume their specific role, and the task prompts provided clear instructions on the tasks to perform. Novelty was assessed by comparing the manuscript with similar articles from an external database of published literature. The manuscript, novelty assessment, and figures were sent to each agent for analysis, and the leader agent compiled the information from all agents, requested clarifications if needed, and produced the final review. Our AI multiagent panel was evaluated and compared with human reviews as well as simple prompt-based review<sup>2</sup> and findings of 2 previous studies.<sup>1,3</sup> Evaluation was done by graduate students via an arena where pairs of systems were selected based on their Elo score. The Elo score is a measure of the relative capability of each system based on their win/loss in the arena against other systems and was calculated using the Bradley-Terry model. The system names were hidden from the participants to reduce bias. The participants evaluated the reviews based on technical quality, constructiveness, clarity, and overall assessment before making their choice.</p>
<h4>Results</h4>
<p> Eleven graduate students in the computer science department participated in this study, which resulted in 140 matches between pairs of systems. The systems with similar Elo scores were most likely to be paired together. Mean Elo scores of the reviews produced from our system were higher than those of human review scores (<b>Table 25-0971</b>). Further analysis showed that reviews produced by our system included more detailed assessment of the limitations and possible improvements of the manuscript than reviews produced by humans. However, our system tended to overly praise work, which humans rarely do.</p>

![](https://assets.underline.io/markdown_image/1/image/e9b3e219d84889a96133d3b20f6fe73d.png)

<h4>Conclusions</h4>
<p> Our AI peer review system demonstrated potential in aiding the peer review process of publications as an addition to human reviews. It might act as a quick response to authors while human reviewers are reviewing the manuscript. Although there might be bias within the review, the author can choose to verify and utilize comments from this quick feedback as needed. Harm or the potential for negative effects were not studied. We acknowledge that the number of participants in this study was small. Thus, a larger study might have different results from our findings.</p>
<h4>References</h4>
<p>1. Liang W, Zhang Y, Cao H, et al. Can large language models provide useful feedback on research papers—a large-scale empirical analysis. <i>NEJM AI</i>. 2024;1(8):AIoa2400196. doi:10.1056/AIoa2400196</p>
<p>2. Wang H, Fu T, Du Y, et al. Scientific discovery in the age of artificial intelligence. <i>Nature</i>. 2023;620(7972):47-60. doi:10.1038/s41586-023-06221-2</p>
<p>3. D’Arcy M, Hope T, Birnbaum L, Downey D. Marg: multiagent review generation for scientific papers. <i>arXiv</i>. Preprint posted online January 8, 2024. doi:10.48550/arXiv.2401.04259</p>
<sup>1</sup>Science of Science and Computational Discovery Lab, Department of Computer Science, University of Colorado Boulder, Boulder, CO, US, daniel.acuna@colorado.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Daniel E. Acuna reported being the founder of ReviewerZero AI LLC (<a href="https://www.reviewerzero.ai"><span class="Hyperlink CharOverride-6">https://www.reviewerzero.ai</span></a>), a company focused on research integrity solutions, which may be relevant to the topic of this publication.</p>


Next from The 10th Peer Review Congress

Prevalence of and Reasons for Retractions on Traditional Chinese Medicine Research Publications by Authors from Mainland China in International Peer-review Journals

Similar lecture

[V] Editorial Landscape of Journals in Kenya, Ethiopia, Nigeria, and Mozambique