United States

&lt;p&gt;Stephan D. Fihn,&lt;sup&gt;1,3&lt;/sup&gt; Roy H. Perlis,&lt;sup&gt;2,3&lt;/sup&gt; Jacob Kendall-Taylor,&lt;sup&gt;3&lt;/sup&gt; Annette Flanagin&lt;sup&gt;3&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective&lt;/h4&gt;
&lt;p&gt; The major challenges for editors of journals include attracting submissions of high-quality manuscripts and recruiting reviewers.&lt;sup&gt;1&lt;/sup&gt; In light of a dearth of relevant research, we sought to ascertain the association between publishing manuscripts as an author and participation in peer review for the same journal.&lt;/p&gt;
&lt;h4&gt;Design&lt;/h4&gt;
&lt;p&gt; We conducted a cross-sectional study using data extracted about authors and reviewers for &lt;i&gt;JAMA Network Open&lt;/i&gt;, a high-volume open access general medical journal, for the years 2019 to 2023, including number of articles submitted and published as first or corresponding author during this period, numbers of requests to review and response (submitted, declined, no response), and quality of review submitted as determined by reviewing editor on a 5-point Likert scale (excellent = 1 to poor = 5). Data were analyzed using R 4.4.2.&lt;/p&gt;
&lt;h4&gt;Results&lt;/h4&gt;
&lt;p&gt; Among 38,683 invited reviewers, 30,082 (77.8%) submitted no manuscripts; 4014 (10.6%) submitted ≥1 article as first or corresponding author but had no published articles; and 4497 (11.6%) submitted ≥1 article as first or corresponding author and had ≥1 publication. &lt;b&gt;Table 25-1097&lt;/b&gt; summarizes review requests to these individuals and compares their rates of accepting reviews and ratings among those who returned reviews. The mean (SD) number of invitations was greatest among published reviewers (5.1 [9.5]), followed by those with submissions but no publications (3.4 [6.1]), and then individuals with no submissions (2.7 [5.9]) (&lt;i&gt;P&lt;/i&gt; &amp;lt; .007 by analysis of variance). Reviewers with publications were significantly more likely to accept invitations and significantly less likely to ignore invitations compared with the other 2 groups (&lt;i&gt;P&lt;/i&gt; &amp;lt; .01 by analysis of variance and post hoc pairwise test). The absolute difference in proportion of accepted invitations was 26.2% (95% CI, 24.8%-27.6%; &lt;i&gt;P&lt;/i&gt; &amp;lt; .001) between published and never-submitted reviewers and 11.6% (95% CI, 9.7%-13.5%; &lt;i&gt;P&lt;/i&gt; &amp;lt; .001) between published and nonpublished submitters. Among those who returned reviews, quality (mean rating) was significantly higher among reviewers with publications (absolute difference from nonsubmitters: 0.25; 95% CI, −0.29 to −0.20; &lt;i&gt;P&lt;/i&gt; &amp;lt; .001). Differences between submitters without publications (mean difference: 0.03) and never-submitters (mean difference: 2.5%) were minimal for both rating metrics (&lt;i&gt;P&lt;/i&gt; &amp;gt; .50).&lt;/p&gt;
&lt;a href=&quot;/wp-content/uploads/2025_conference/tables/prc25-1097t1.png&quot;&gt; &lt;img loading=&quot;lazy&quot; src=&quot;/wp-content/uploads/2025_conference/tables/prc25-1097t1.png&quot;&gt; &lt;/a&gt;
&lt;h4&gt;Conclusions &lt;/h4&gt;
&lt;p&gt;Authors who published in &lt;i&gt;JAMA Network Open &lt;/i&gt;were significantly more likely to accept an invitation to review even though they received nearly twice as many requests as those who did not publish. Moreover, the quality of their reviews was significantly greater. Individuals who submitted manuscripts but did not publish were intermediate between these 2 groups in terms of accepting invitations to reviews and quality of reviews. These results suggest that investigators may be willing to commit more of their academic effort to journals that publish their work. Limitations of this analysis include the inability to determine the relative timing of publication versus review; restriction of the analysis to a single high-volume journal; exclusion of co-authors; and limitation to a 4-year time frame.&lt;/p&gt;
&lt;h4&gt;Reference&lt;/h4&gt;
&lt;p&gt;1. Zupanc GKH. “It is becoming increasingly difficult to find reviewers”—myths and facts about peer review. J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2024;210(1):1-5. doi:10.1007/s00359-023-01642-w&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Department of Medicine, University Washington, Seattle, WA, US; steve.fihn@jamanetwork.org; &lt;sup&gt;2&lt;/sup&gt;Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, US; &lt;sup&gt;3&lt;/sup&gt;JAMA Network, Chicago, IL, US.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; Stephan D. Fihn (SDF) and Roy H. Perlis (RHP) are paid as consultants for work as editors of JAMA Network Open (SDF and RHP) and JAMA+ AI (RHP). Jacob Kendall-Taylor and Annette Flanagin are on the editorial staff of the JAMA Network. Annette Flanagin is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. None of the authors have any other conflicts of interest to disclose.&lt;/p&gt;
&lt;h4&gt;Funding/Support &lt;/h4&gt;
&lt;p&gt;The authors received no direct funding for this work apart from that described above. &lt;/p&gt;


The 10th Peer Review Congress

Authors Who Publish in a Journal and Likelihood to Serve as Reviewers

peer review process and models

authorship and contributorship

peer review

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Stephan D. Fihn,<sup>1,3</sup> Roy H. Perlis,<sup>2,3</sup> Jacob Kendall-Taylor,<sup>3</sup> Annette Flanagin<sup>3</sup></p>
<h4>Objective</h4>
<p> The major challenges for editors of journals include attracting submissions of high-quality manuscripts and recruiting reviewers.<sup>1</sup> In light of a dearth of relevant research, we sought to ascertain the association between publishing manuscripts as an author and participation in peer review for the same journal.</p>
<h4>Design</h4>
<p> We conducted a cross-sectional study using data extracted about authors and reviewers for <i>JAMA Network Open</i>, a high-volume open access general medical journal, for the years 2019 to 2023, including number of articles submitted and published as first or corresponding author during this period, numbers of requests to review and response (submitted, declined, no response), and quality of review submitted as determined by reviewing editor on a 5-point Likert scale (excellent = 1 to poor = 5). Data were analyzed using R 4.4.2.</p>
<h4>Results</h4>
<p> Among 38,683 invited reviewers, 30,082 (77.8%) submitted no manuscripts; 4014 (10.6%) submitted ≥1 article as first or corresponding author but had no published articles; and 4497 (11.6%) submitted ≥1 article as first or corresponding author and had ≥1 publication. <b>Table 25-1097</b> summarizes review requests to these individuals and compares their rates of accepting reviews and ratings among those who returned reviews. The mean (SD) number of invitations was greatest among published reviewers (5.1 [9.5]), followed by those with submissions but no publications (3.4 [6.1]), and then individuals with no submissions (2.7 [5.9]) (<i>P</i> &lt; .007 by analysis of variance). Reviewers with publications were significantly more likely to accept invitations and significantly less likely to ignore invitations compared with the other 2 groups (<i>P</i> &lt; .01 by analysis of variance and post hoc pairwise test). The absolute difference in proportion of accepted invitations was 26.2% (95% CI, 24.8%-27.6%; <i>P</i> &lt; .001) between published and never-submitted reviewers and 11.6% (95% CI, 9.7%-13.5%; <i>P</i> &lt; .001) between published and nonpublished submitters. Among those who returned reviews, quality (mean rating) was significantly higher among reviewers with publications (absolute difference from nonsubmitters: 0.25; 95% CI, −0.29 to −0.20; <i>P</i> &lt; .001). Differences between submitters without publications (mean difference: 0.03) and never-submitters (mean difference: 2.5%) were minimal for both rating metrics (<i>P</i> &gt; .50).</p>
<a href="/wp-content/uploads/2025_conference/tables/prc25-1097t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1097t1.png"> </a>
<h4>Conclusions </h4>
<p>Authors who published in <i>JAMA Network Open </i>were significantly more likely to accept an invitation to review even though they received nearly twice as many requests as those who did not publish. Moreover, the quality of their reviews was significantly greater. Individuals who submitted manuscripts but did not publish were intermediate between these 2 groups in terms of accepting invitations to reviews and quality of reviews. These results suggest that investigators may be willing to commit more of their academic effort to journals that publish their work. Limitations of this analysis include the inability to determine the relative timing of publication versus review; restriction of the analysis to a single high-volume journal; exclusion of co-authors; and limitation to a 4-year time frame.</p>
<h4>Reference</h4>
<p>1. Zupanc GKH. “It is becoming increasingly difficult to find reviewers”—myths and facts about peer review. J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2024;210(1):1-5. doi:10.1007/s00359-023-01642-w</p>
<sup>1</sup>Department of Medicine, University Washington, Seattle, WA, US; steve.fihn@jamanetwork.org; <sup>2</sup>Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, US; <sup>3</sup>JAMA Network, Chicago, IL, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Stephan D. Fihn (SDF) and Roy H. Perlis (RHP) are paid as consultants for work as editors of JAMA Network Open (SDF and RHP) and JAMA+ AI (RHP). Jacob Kendall-Taylor and Annette Flanagin are on the editorial staff of the JAMA Network. Annette Flanagin is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. None of the authors have any other conflicts of interest to disclose.</p>
<h4>Funding/Support </h4>
<p>The authors received no direct funding for this work apart from that described above. </p>


poster

<p>Sidney Xiang,<sup>1</sup> Daniel M. Romero,<sup>1,2,3</sup> Misha Teplitskiy<sup>1</sup></p>
<h4>Objective</h4>
<p> Prior literature on interdisciplinary research evaluation does not include prepublication manuscripts or account for differential evaluation over multiple dimensions of interdisciplinarity. Based on management science literature, we introduce 2 novel dimensions: topic interdisciplinarity, which may incur evaluation penalties by cutting across disciplinary standards and threatening symbolic boundaries, and knowledge-base interdisciplinarity, which may incur benefits by combining larger pools of nonredundant information. Evaluations may also depend on how these dimensions align with each other and the audience. Our study tested these hypotheses using a large-scale dataset of journal submissions.</p>
<h4>Design</h4>
<p> We performed a case-control study using administrative data from the Institute of Physics Publishing, a major STEM publisher. The data included 128,950 manuscripts (accepted and rejected) submitted to 62 physical science journals from 2018 to 2022. We calculated topic interdisciplinarity with concept tags of the manuscript’s associated OpenAlex record<sup>1</sup> and knowledge-base interdisciplinarity with concept tags of the manuscript’s references. For each type, we quantified interdisciplinarity using the integration index,<sup>2</sup> ranging from 0 to 1. Logistic regression was used to model the relationship between both types of interdisciplinarity and 2 evaluation outcomes: final decision and review positivity, controlling for journal, manuscript type, year, number of authors, number of authors’ prior citations, number of authors’ prior publications, and number of references. We retained observations with both interdisciplinarities and all covariates.</p>
<h4>Results</h4>
<p> The marginal effect of a 1-SD increase in knowledge-base interdisciplinarity was a 0.9 (0.7) percentage point higher acceptance (positive review) probability, while a 1-SD increase in topic interdisciplinarity corresponded to a 1.2 (0.4) percentage point lower acceptance (positive review) probability. Interactions revealed that high knowledge-base interdisciplinarity attenuated the negative relationship between topic interdisciplinarity and acceptance. We did not see a symmetric effect when high knowledge-base interdisciplinarity was accompanied by high topic interdisciplinarity. When disaggregating model results by journal interdisciplinarity according to the publisher’s 2022 product catalogue,<sup>3</sup> journals categorized by the publisher as interdisciplinary or multisubject had positive associations between both types of interdisciplinarity and evaluation outcomes, whereas monodisciplinary journals had negative associations. Our results were robust to the inclusion and transformation of manuscript and team covariates.</p>

![](https://assets.underline.io/markdown_image/1/image/3f8d7212732299f660aee5d48fd7f1cb.png)

<h4>Conclusions</h4>
<p> Our findings indicate that different dimensions of interdisciplinarity have different relationships to evaluation and that, for successful publication, authors must attend to alignment between types of interdisciplinarity within the manuscript and between the manuscript and journal audience. Limitations include the scope of journals in our dataset (STEM only) and the reliance on matching submissions to OpenAlex records.</p>
<h4>References</h4>
<p>1. Priem J, Piwowar H, Orr R. OpenAlex: a fully-open index of scholarly works, authors, venues, institutions, and concepts. arXiv. Preprint posted online May 4, 2022. doi:10.48550/ARXIV.2205.01833</p>
<p>2. Porter AL, Cohen AS, David Roessner J, Perreault M. Measuring researcher interdisciplinarity. Scientometrics. 2007;72(1):117-147. doi:10.1007/s11192-007-1700-5</p>
<p>3. IOP Publishing. Product catalogue 2022. <a href="https://ioppublishing.org/wp-content/uploads/2022/03/IOPP-Catalogue-2022.pdf"><span class="Hyperlink CharOverride-6">https://ioppublishing.org/wp-content/uploads/2022/03/IOPP-Catalogue-2022.pdf</span></a></p>
<sup>1</sup>University of Michigan School of Information, Ann Arbor, MI, US, tepl@umich.edu; <sup>2</sup>University of Michigan Department of Electrical Engineering and Computer Science, Ann Arbor, MI, US; <sup>3</sup>University of Michigan Center for the Study of Complex Systems, Ann Arbor, MI, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Topic and Knowledge-base Interdisciplinarity in Manuscripts Submitted to Physical Science Journals vs Editorial Decision and Reviewer Positivity

<p>Neil Millar,<sup>1</sup> Brian Budgell<sup>2</sup></p>
<h4>Objective</h4>
<p> Promotional language (hereafter, <i>hype</i>) in scientific writing has increased significantly over the past 40 years.<sup>1</sup> Analyses of grant proposals have found the incidence of hype to be positively associated with funding success<sup>2</sup>; however, evidence of causality is still lacking. Furthermore, an underpowered pilot study suggested that hype may bias clinicians’ perceptions and evaluations of evidence.<sup>3</sup> This study tested the hypothesis that students’ appraisal of biomedical research abstracts is influenced by hype.</p>
<h4>Design</h4>
<p> A double-blind randomized controlled trial was conducted to assess readers’ evaluation of hyped vs nonhyped abstracts. Four structured abstracts were selected from leading spinal care journals, controlling for length and absence of hype. Hyped abstracts were modified to include 6 common hype terms, such as “carefully designed” to highlight methodological rigor, “this is the first study to” to emphasize novelty, “convincing evidence” to amplify results, and “conducted by an experienced radiologist” to emphasize researcher competence. Inclusion criteria were that participants were second-year or higher chiropractic students recruited from institutions in the UK, US, and Canada. A blinded researcher randomly selected folders for students to evaluate. Participants rated abstracts on a 10-point scale, using 4 criteria: implementation likelihood, rigor, novelty, and researcher competence. A linear mixed-effects model was used to assess whether evaluations were influenced by hype, accounting for variability across participants and abstracts.</p>
<h4>Results</h4>
<p> This ongoing study began on April 2, 2024. As of January 2025, 36 students at 3 institutions had completed the rating task. Mixed-effects modeling, including interactions between condition and criterion, showed no significant differences between hyped and original abstracts across all evaluation criteria (estimate, 0.067; <i>P</i> &gt; .05) (<b>Figure 25-0927</b>). While novelty received consistently lower ratings (estimate, −1.479; <i>P</i> &lt; .001), the influence of hype did not vary significantly by criterion, as interaction terms were nonsignificant (eg, condition × novelty interaction [estimate, 0.517; <i>P</i> &gt; .05]), indicating minimal measurable effect of promotional language.</p>

![](https://assets.underline.io/markdown_image/1/image/ed072eece70a4477cfa47ac6438bf2df.png)

<h4>Conclusions </h4>
<p>Within this experimental paradigm, hype did not appear to bias readers’ evaluation of the scientific merit of the literature. These findings are discussed in relation to evidence-based medicine and bias on the part of researchers, reviewers, editors, and other stakeholders. Limitations of the study include the small sample size reported to date and restriction of the experimental texts to abstracts.</p>
<h4>References</h4>
<p>1. Millar N, Batalo B, Budgell B. Trends in the use of promotional language (hype) in National Institutes of Health funding opportunity announcements, 1992-2020. <i>JAMA Netw Open</i>. 2022;5(11):e2243221. doi:10.1001/jamanetworkopen.2022.43221</p>
<p>2. Qiu HS, Peng H, Fosse HB, Woodruff TK, Uzzi B. Use of promotional language in grant applications and grant success. <i>JAMA Netw Open</i>. 2024;7(12):e2448696. doi:10.1001/jamanetworkopen.2024.48696</p>
<p>3. Millar N, Budgell B. Impact of hype on clinicians’ evaluation of trials—a pilot study. <i>J Can Chiropr Assoc</i>. 2023;67(1):38-49.</p>
<sup>1</sup>Faculty of Engineering, Information and Systems, University of Tsukuba, Tsukuba, Ibaraki, Japan; <sup>2</sup>Canadian Memorial Chiropractic College, Toronto, Ontario, Canada, bbudgell@cmcc.ca.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>
<h4>Funding/Support </h4>
<p>This study was supported by grant 21K02919 from the Japan Society for the Promotion of Science.</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>


The Influence of Promotional Language on Evaluations of Biomedical Literature: A Randomized Controlled Trial

<p>Vincent Yuan,<sup>1</sup> Aidan Christopher Tan<sup>2</sup> </p>
<h4>Objective </h4>
<p>Our study aimed to determine the prevalence, types, and facilitators of “spin” and other selective reporting practices in recently published medical artificial intelligence (AI) research.</p>
<h4>Design </h4>
<p>This was a cross-sectional meta-research study of interventional and observational studies related to medical AI. Medical AI was defined as computational models (eg, machine learning, deep learning, natural language processing) that amplify or augment health care–related tasks (eg, diagnosis, prognosis, risk stratification).<sup>1</sup> Studies were identified by searching MEDLINE, Embase, and the Cochrane Library. Studies were included if they were observational or interventional studies related to medical AI published in 2023, English-language, and full text. Spin was defined as the distortion of findings to present the performance of a medical AI tool as more beneficial than it appears. Examples of spin include misrepresentation or omission of significant or nonsignificant results and selective focus on abstract conclusions. Other study characteristics investigated included the journal, academic qualifications of first authors, funding, and competing interests. The features of spin were derived from coding manuals of Boutron et al<sup>2</sup> and Andaur Navarro et al.<sup>3</sup> Spin was categorized as either misleading interpretation, referring to an overemphasis of results, or misleading transportability, referring to an exaggeration of a model’s clinical applicability. Instances of spin were abstracted by manually reviewing the Results and Discussion sections of abstracts and body and extracting the data into a prepiloted data collection form. This form was modified from SPIN-Prediction Models to exclude or include certain indicators, with the addition of custom items based on previous spin studies. Data were collected by 1 reviewer, and a second reviewer independently recollected the data in instances of uncertainty.</p>
<h4>Results </h4>
<p>A total of 423 studies were included (<b>Figure 25-0950</b>). Slightly more than one-third (35% [n = 149]) of studies described AI as superior to the comparator. Nine percent of studies (n = 38) contained leading words in the title, and more than half (56% [n = 236]) selectively focused abstract conclusions on significant results. In almost half (48% [n = 203]) of study abstracts, qualifiers (eg, <i>might</i>, <i>very</i>) were present to subtly suggest stronger findings than likely warranted. The majority of studies (71% [n = 300]) contained strong statements (eg, <i>clearly shows</i>) in the body to emphasise the AI model’s performance, accuracy, or effectiveness. Twenty-four percent (n = 101) of studies made recommendations for clinical practice in the body without external validation in the same study.</p>

![](https://assets.underline.io/markdown_image/1/image/8d8ebe9fade171778923d51191e80204.png)

<h4>Conclusions </h4>
<p>Our findings suggest that selective reporting practices, including spin, are prevalent in medical AI research. These practices can misinform readers of the technical performance and real-world applicability of AI tools. This is especially crucial for titles and abstracts because these sections often influence first impressions and inform clinical decision-making.</p>
<h4>References</h4>
<p>1. Bajwa J, Munir U, Nori A, Williams B. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthc J. 2021;8(2):e188-e194. doi:10.7861/fhj.2021-0095 </p>
<p>2. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058-2064. doi:10.1001/jama.2010.651 </p>
<p>3. Andaur Navarro CL, Damen JAA, Ghannad M, et al. SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models. J Clin Epidemiol. 2024;170:111364. doi:10.1016/j.jclinepi.2024.111364 </p>
<sup>1</sup>School of Clinical Medicine, Faculty of Medicine and Health, University of New South Wales, Sydney, NSW, Australia,<i> </i>vincent.yuan1@student.unsw.edu.au; <sup>2</sup>Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Unraveling the 'Spin' and Selective Reporting in Medical AI Research: A Cross-Sectional Meta-Research Study

<p>Joe D. Menke,<sup>1</sup> Mengfei Lan,<sup>1</sup> Halil Kilicoglu<sup>1</sup></p>
<h4>Objective </h4>
<p>The importance of self-acknowledged limitations (SALs) has long been recognized.<sup>1</sup> Prior work detecting<sup>2</sup> and categorizing<sup>3</sup> limitation statements using natural language processing (NLP) has made large-scale assessments of limitation reporting possible. By comparing SALs in research articles with limitations identified by systematic reviewers, this study aimed to evaluate the quality and coverage of SALs in relation to risk of bias.</p>
<h4>Design</h4>
<p> We constructed a dataset pairing SALs with limitations discussed by reviewers through risk-of-bias assessments. Articles from the Cochrane Database of Systematic Reviews available in the open-access subset of PubMed Central were downloaded in September 2024. For papers cited in these reviews, full texts were downloaded when available, and existing NLP models<sup>3</sup> were used to extract and classify sentences discussing limitations (December 2024). Identifying information and risk-of-bias assessments for cited articles were extracted from reviews’ Characteristics of Included Studies tables when a Risk of Bias section was present; reviews without this section were excluded. For living reviews, duplicate assessments were removed to retain only 1 assessment per cited article across versions of a living review. Risk-of-bias statements, originally in tabular form (eg, “Allocation concealment [Selection bias] High risk CBA; no allocation concealment done.”), were rephrased into natural language using a large language model (Llama3.3-70B). Low-risk components were excluded, retaining only high- and unclear-risk items. A random subset was manually reviewed to check for systematic issues, which did not reveal concerning patterns or hallucinations. Paraphrased information from reviews and SALs from cited papers were classified using NLP models that recognized limitations sentences and types.<sup>3</sup> We analyzed and compared the distribution of categorized limitations mentioned in both groups in an effort to assess the quality of SALs.</p>
<h4>Results</h4>
<p> From 145 reviews, we extracted and classified limitations statements for 803 cited articles (5.54 articles per review). From risk-of-bias tables, 3172 limitations were extracted (3.95 per cited article). From 750 cited articles, 136,290 sentences were processed and classified (170 per article), yielding 5616 SAL sentences (7.49 per article). No SALs were found in 53 articles. Categorized data are reported in <b>Table 25-1017</b>. Outcome measures were the most common SAL (69.12% of articles), while reviewers noted missing data as the most common high-risk limitation (33.75% of articles) and blinding as the most common unclear risk (48.19% of articles). On average, 56.67% of reviewer-identified limitations were not self-acknowledged, including 38.10% of high-risk and 53.01% of unclear-risk limitations.</p>

![](https://assets.underline.io/markdown_image/1/image/94ce029a42605d4892ff758a6a004f27.png)

<h4>Conclusions</h4>
<p> We present categorized limitations data and compared SALs with reviewer-derived limitations, showing that more than one-third of high-risk limitations are not self-acknowledged. Our findings may be used to better understand the quality of existing self-acknowledged limitations as well as highlight categories that are potentially underreported by authors.</p>
<h4>References</h4>
<p>1. Ioannidis JP. Limitations are not properly acknowledged in the scientific literature. J Clin Epidemiol. 2007;60(4):324-329. doi:10.1016/j.jclinepi.2006.09.011</p>
<p>2. Kilicoglu H, Rosemblat G, Malički M, Ter Riet G. Automatic recognition of self-acknowledged limitations in clinical research literature. J Am Med Inform Assoc. 2018;25(7):855-861. doi:10.1093/jamia/ocy038</p>
<p>3. Lan M, Cheng M, Hoang L, Ter Riet G, Kilicoglu H. Automatic categorization of self-acknowledged limitations in randomized controlled trial publications. J Biomed Inform. 2024;152:104628. doi:10.1016/j.jbi.2024.104628</p>
<sup>1</sup>School of Information Sciences, University of Illinois Urbana-Champaign, Champaign, IL, US, jmenke2@illinois.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> This work was supported by the National Library of Medicine of the National Institutes of Health (NIH; R01LM014079).</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funder had no role in the study design or in the collection, analysis, or interpretation of data; writing of the report; or decision to submit the abstract for publication.</p>
<h4>Additional Information</h4>
<p> The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.</p>


A Comparison of Self-Acknowledged Limitations With Risk-of-Bias Assessments From Systematic Reviews

<p>Kathryn A. Phillips,<sup>1,2</sup> Danea M. Horn<sup>1,2</sup></p>
<h4>Objective</h4>
<p> The move toward open science has focused attention on journal review and publication times,<sup>1</sup> particularly for health care research, where rapid dissemination is important for decision-making. We examined how academic journals that publish on health policy report and measure their review and publication times (RPTs) and assessed patterns across journals. Compared with earlier research that focused on article-level metrics for specialty journals,<sup>2</sup> we focused on journal-level metrics across a broad range of journals, enhancing generalizability.</p>
<h4>Design</h4>
<p> We conducted a cross-sectional analysis of 57 academic journals that publish on health policy topics following STROBE guidelines. Institutional review board review was not required. We included journals with a Google h5-index greater than 23 and a Clarivate impact factor of 2 or higher as of November 18, 2024. Final inclusion was based on review by 3 journal editors. We examined 6 metrics (<b>Table</b> <b>24-0820</b>) and used descriptive and statistical analyses to assess variation in the definition and reporting of RPTs. We compared RPTs across journal characteristics using the Wilcoxon rank sum test (with significance established at <i>P</i> &lt; .05).</p>

![](https://assets.underline.io/markdown_image/1/image/06280c0b892539b55c17e4d5a6db2179.png)

<h4>Results</h4>
<p> Of the 57 journals analyzed, 9 (15.8%) did not publicly report any RPTs. Conversely, only 21 journals (36.8%) reported 3 or more metrics. There was wide variation in each reported RPT, with the full peer-reviewed publication process taking 5.6 to 11.9 months. Open access journals had faster median submission to final decision (peer-reviewed) times compared with hybrid and subscription journals (184.5 vs 228.0 days; <i>P</i> = .03). Highly selective journals (journals with high impact factors and low acceptance rates) had a faster median time to first decision than less selective journals (3.0 vs 13.0 days; <i>P</i> &lt; .001). Within each RPT metric, there was considerable variation in reporting frequency and definitions. Journals often did not clarify whether the time to first decision included peer review and presented ranges or used qualifiers such as <i>typically</i> or <i>approximately</i> for post–peer review decision times. Few journals reported metrics by article type or provided metric time frames. Lastly, <i>mean</i> and <i>median</i> were used inconsistently.</p>
<h4>Conclusions </h4>
<p>There was substantial variation across journals in whether and how they report RPTs and selective reporting and considerable variation in how timing metrics were defined. Publication speed is influenced by multiple factors, including review processes and publication frequency. The lack of standardized reporting practices limits transparency in academic publishing and suggests a need for journals to more consistently report RPTs and for RPTs to be included in standardized databases and tracking systems. This would better inform author submission decisions and inform future research and testing of RPT interventions using data from editorial systems. Improving reporting and transparency is especially important for research on urgent health policy topics and for early-career academics.<sup>3</sup></p>
<h4>References</h4>
<p>1. Phillips KA. Open access publication at a crossroads—implications for researchers and beyond. JAMA Health Forum. 2024;5(10):e242914. doi:<a href="https://doi.org/10.1001/jamahealthforum.2024.2914"><span class="Hyperlink CharOverride-6">10.1001/jamahealthforum.2024.2914</span></a></p>
<p>2. Petrou C. Guest post—publishing fast and slow: a review of publishing speed in the last decade. The Scholarly Kitchen. November 8, 2022. Accessed November 8, 2024. <a href="https://scholarlykitchen.sspnet.org/2022/11/08/guest-post-publishing-fast-and-slow-a-review-of-publishing-speed-in-the-last-decade/"><span class="Hyperlink CharOverride-6">https://scholarlykitchen.sspnet.org/2022/11/08/guest-post-publishing-fast-and-slow-a-review-of-publishing-speed-in-the-last-decade/</span></a></p>
<p>3. Andersen MZ, Fonnes S, Rosenberg J. Time from submission to publication varied widely for biomedical journals: a systematic review. Curr Med Res Opin. 2021;37(6):985-993. doi:<a href="https://doi.org/10.1080/03007995.2021.1905622"><span class="Hyperlink CharOverride-6">10.1080/03007995.2021.1905622</span></a></p>
<sup>1</sup>Center for Translational and Policy Research on Precision Medicine, Department of Clinical Pharmacy, University of California, San Francisco, CA, US, kathryn.phillips@ucsf.edu; <sup>2</sup>Institute for Health Policy Studies, University of California, San Francisco, CA, US.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Kathryn A. Phillips is the founding editor in chief of Health Affairs Scholar: Emerging and Global Health Policy. No other conflicts were reported.</p>
<h4>Funding/Support </h4>
<p>This abstract was partially supported by Health Affairs and Project HOPE.</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funder had no role in the preparation, review, or approval of the abstract.</p>


Review and Publication Times Across Journals Publishing on Health Policy

<p>Farbod Zahedi Tajrishi,<sup>1</sup> Sina Rashedi,<sup>2</sup> Ashkan Hashemi,<sup>3</sup> Isaac Dreyfus,<sup>4</sup> Nicholas Varunok,<sup>5</sup> John Burton,<sup>6</sup> Seng Chan You,<sup>7</sup> Björn Redfors,<sup>3,8</sup> Gregory Piazza,<sup>2</sup> Joshua D. Wallach,<sup>9</sup> Lesley Curtis,<sup>10</sup> Sanjay Kaul,<sup>11</sup> David J. Cohen,<sup>8,12</sup> Roxana Mehran,<sup>13</sup> Flavia Geraldes,<sup>14</sup> Joseph S. Ross,<sup>15</sup> Jane Leopold,<sup>2</sup> Harlan M. Krumholz,<sup>15</sup> Gregg W. Stone,<sup>13</sup> Behnood Bikdeli<sup>2,15</sup></p>
<h4>Objective </h4>
<p>Altmetric scores capture online engagement with research, including social media mentions and news coverage, whereas citation counts represent academic impact. It remains uncertain whether the source of funding for randomized clinical trials (RCTs) correlates with the online and academic impact of RCTs overall and within the subsets that meet or do not meet their prespecified primary outcomes. We hypothesized that privately funded trials, particularly those with positive results, would be cited more often and have higher Altmetric scores than publicly funded trials.</p>
<h4>Design </h4>
<p>We searched PubMed for cardiovascular RCTs published in <i>JAMA</i>, <i>The Lancet</i>,<i> </i>and<i> New England Journal of Medicine </i>between 2014 and 2019. This timeframe was selected to ensure sufficient follow-up for citation accrual and stability of Altmetric activity while capturing contemporary cardiovascular research. Trials were classified by funding source: public (ie, governmental), private (including for-profit industry and nonprofit organizations), or hybrid (with both public and private funding). RCT results were considered positive if they met their primary outcome or at least 1 of their coprimary outcomes and negative if they did not. Altmetric scores were obtained from Altmetric.com, and citation counts from Google Scholar (last accessed February 8, 2025). Both metrics were adjusted for publication year.</p>
<h4>Results </h4>
<p>A total of 435 cardiovascular RCTs were analyzed, of which 130 (29.9%) were publicly funded and 197 (45.3%) were privately funded (<b>Figure 25-1147</b>). A total of 175 negative trials (40%) were identified. There was no significant difference across trials for Altmetric score based on the funding source (median [IQR] scores: public, 218.1 [112.3-465.6]; private, 215.1 [112.9-506.6]; and hybrid, 244.5 [127.3-555.8]; <i>P</i> = .49). Publicly funded trials were cited less frequently than privately funded or hybrid-funded trials (median [IQR] citations: public, 410.0 [226.5-724.0]; private, 599.0 [287.0-1022.0]; and hybrid, 607.0 [301.8-936.0]; <i>P</i> = .003). When exploring the findings based on the primary results of the RCTs, median (IQR) citations for positive vs negative publicly funded trials (402.5 [216.3-827.0] vs 419.5 [240.3-644.0] citations) and hybrid trials (614.5 [311.8-1173.3] vs 597.0 [300.5-808.3] citations) were substantively similar, whereas median (IQR) citations for privately funded trials (705.0 [298.5-1395.5] vs 383.0 [262.3-694.8] citations) were higher if they met the primary outcome (<i>P</i> for interaction &lt; .001).</p>

![](https://assets.underline.io/markdown_image/1/image/01c2c8ab244adfa241066ffe3d0e10cc.png)

<h4>Conclusions </h4>
<p>Funding source was associated with subsequent citations, but not the Altmetric score. Publicly funded trials received significantly fewer citations than privately funded trials. Publicly and hybrid-funded trials showed similar citation patterns regardless of outcome, whereas privately funded trials were cited more when the primary outcome was met. These findings highlight the role of funding source in shaping the impact of cardiovascular RCTs.</p>
<sup>1</sup>Tulane University School of Medicine, New Orleans, LA, US; <sup>2</sup>Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US, bbikdeli@bwh.harvard.edu; <sup>3</sup>Weill Cornell Medicine, New York, NY, US; <sup>4</sup>David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA, US; <sup>5</sup>Vanderbilt University Medical Center, Nashville, TN, US; <sup>6</sup>Keck School of Medicine, University of Southern California (USC), Los Angeles, CA, US; <sup>7</sup>Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, South Korea; <sup>8</sup>Cardiovascular Research Foundation, New York, NY, US; <sup>9</sup>Rollins School of Public Health, Emory University, Atlanta, GA, US; <sup>10</sup>Duke University and Duke Clinical Research Institute (DCRI), Durham, NC, US; <sup>11</sup>Cedars-Sinai Medical Center, Los Angeles, CA, US; <sup>12</sup>St Francis Hospital &amp; Heart Center, Roslyn, NY, US; <sup>13</sup>Icahn School of Medicine at Mount Sinai, New York, NY, US; <sup>14</sup>The Lancet Group, London, UK; <sup>15</sup>Center for Outcomes Research and Evaluation (CORE), Yale School of Medicine, New Haven, CT, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Funding Sources and the Online and Academic Impact of Cardiovascular Trials Published in Highest-Impact Journals

<p>Justin N. Nguyen,<sup>1</sup> Robert T. Rubin<sup>1,2</sup></p>
<h4>Objective</h4>
<p> Several issues about scientific publishing, including integrity of peer review, honorary/ghost authorships, commercial influences, and inadequate disclosure of conflicts of interest, have been receiving increasing scrutiny.<sup>1</sup> The question arises as to how often journal editors in chief (EICs) publish research and review articles in their journals during their tenure compared with before and after their editorships.<sup>2,3</sup> If psychiatry EICs are publishing significantly more often during their tenure, might they be taking advantage of their position in their own journals? Our null hypothesis was that psychiatry EICs do not favor their own journals for self-publication.</p>
<h4>Design</h4>
<p> As an exploratory study, EICs’ publishing practices were examined for 7 general and specialty psychiatry journals (<i>Amer J Psychiatry</i>, <i>Acta Neuropsychiatrica</i>, <i>Biol Psychiatry</i>, <i>Can J Psychiatry</i>, <i>JAMA Psychiatry</i>, <i>J Clin Psychopharmacology</i>, and <i>Neuropsychopharmacology</i>) and 7 general medicine journals (<i>Am J Med</i>, <i>Ann Int Med</i>, <i>Br Med J</i>, <i>Can Med Assoc J</i>, <i>JAMA</i>, <i>Lancet</i>, and <i>N Engl J Med</i>). Data collection occurred between October 2021 and August 2024. For each journal, 1 to 6 psychiatry and 2 to 5 medicine EICs were identified, depending on lengths of tenure. From PubMed, publication lists were compiled for each EIC tenure period, 5 years prior, and 5 years following. Articles related to editors’ responsibilities were omitted; only research and review articles (articles also appropriate for other journals) were included. Two-way analysis of variance with repeated measures was used to assess differences by specialty and time period (the repeated measure).</p>
<h4>Results</h4>
<p> This study included 450 articles published in psychiatry journals and 96 published in general medicine journals. The mean and median articles per year for each time period are presented in <b>Table 25-0853</b>. Psychiatry editors published several times more research and review articles in their own journals than did medicine editors. In addition, psychiatry editors published 128% more frequently in their own journals while in office vs before compared with 24% more frequently for medicine editors. Analysis of variance indicated that the differences in both journal type (F = 10.33; df = 1,33) and time periods (F = 6.26; df = 2,66) were significant (<i>P</i> &lt; .003 for both). Their interaction (F = 3.94; df = 2, 66) also was significant (<i>P</i> &lt; .02). A total of 4 of the 25 psychiatry EICs accounted for most of this difference, with one EIC having published 9 times more frequently in their journal vs before and after their tenure.</p>

![](https://assets.underline.io/markdown_image/1/image/bbc98d3f7fd23c0b0e3cb2a042c01244.png)

<h4>Conclusions</h4>
<p> Psychiatry EICs published significantly more often when leading their journals, whereas medicine EICs did not. This finding has several important limitations. Selection of general medicine journals was subjective, and the medicine journals had higher Impact Factors, longer tenure of editors, and likely greater editorship responsibilities than the psychiatry journals. Articles with EICs as first, senior, and co-authors were included, and there may have been different author motivations for submission. We did not assess if the articles with EICs as authors included disclaimers about the role, or lack of role, of the EICs in editorial review and decision for these articles. Nevertheless, the increase in self-publishing occurred in a small number of psychiatry EICs. This should be further explored through study of a larger number of specialties, more closely matched journals, and more EICs.</p>
<h4>References</h4>
<p>1. Ioannidis JPA, Berkwits M, Flanagin A, Bloom T. Peer review and scientific publication at a crossroads. BMJ. 2023;382:1992. doi:10.1136/bmj.p1992</p>
<p>2. Helgesson G, Radun I, Radun J, Nilsonne G. Editors publishing in their own journals: a systematic review of prevalence and a discussion of normative aspects. Learned Publishing. 2022;35:229-240.</p>
<p>3. Liu F, Holme P, Chiesa M, AlShebli B, Rahwan T. Gender inequality and self-publication are common among academic editors. Nat Hum Behav. 2023;7:353-364. doi:10.1038/s41562-022-01498-1</p>
<sup>1</sup>Community Memorial Hospital Psychiatry Residency, Ventura, CA, rtrubin@yahoo.com; <sup>2</sup>Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine at UCLA, Los Angeles, CA.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Psychiatry Editors-in-Chief Publishing Practices in Their Own Journals

<p>Miro Vukovic,<sup>1</sup> Ana Marušić<sup>1</sup></p>
<h4>Objective</h4>
<p> Conflict of interest (COI) disclosures are critical for the transparency of health technology assessment (HTA) reports, which inform health care policy. Previous studies have shown that COI disclosures in biomedical research and clinical guidelines are often incomplete or inconsistent, raising concerns about transparency and potential bias.<sup>1-3</sup> However, limited empirical evidence exists on how COIs are disclosed in HTA reports despite their critical role in health care decision-making. Our study aimed to address this gap by systematically examining COI disclosure practices across HTA publications.</p>
<h4>Design</h4>
<p> We conducted a cross-sectional study of all English-language HTA reports (mini HTA, rapid reviews, and full HTA) from the International HTA Database (<a href="https://database.inahta.org/"><span class="Hyperlink CharOverride-6">https://database.inahta.org/</span></a>). Reports without accessible full text or not meeting HTA criteria were excluded. Data were extracted from reports published up to December 2023. We documented the presence of COI disclosure sections, their content, and use of standardized forms (eg, International Committee of Medical Journal Editors [ICMJE]) and reviewed adherence to the ICMJE COI reporting guidelines. Two independent reviewers conducted data extraction and coding, with disagreements resolved through discussion. Descriptive statistics were applied to report frequencies and proportions. Based on the study type, the STROBE guideline was followed for reporting.</p>
<h4>Results</h4>
<p> Of 1218 included HTA reports, 598 (49.1%) were full HTAs, 174 (14.3%) were rapid reviews, and 78 (6.4%) were mini HTAs; 368 reports (30.2%) were excluded due to inaccessibility or duplication. Reports originated from 16 countries, with 330 (38.8%) from England. Overall, COI disclosure sections were present in 679 of 850 reports (79.9%). The lowest reporting was among mini HTAs (49 of 78 [62.8%]) compared with full HTAs (489 or 598 [81.8%]) and rapid reviews (141 of 174 [81.0%]). Only 98 reports (11.5% of included reports) used a standardized disclosure form, with the ICMJE form cited in 90. In another 8 reports, the EUnetHTA Declaration of Interest and Confidentiality Undertaking of Interest statement was used.</p>
<h4>Conclusions</h4>
<p> While most HTA reports included COI disclosures, practices varied by report type and country. Standardized disclosure tools were rarely used. Further analysis is needed to support the development of harmonized and transparent COI policies across HTA-producing organizations.</p>
<h4>References</h4>
<p>1. Norris SL, Holmer HK, Ogden LA, Selph SS, Fu R. Conflict of interest disclosures for clinical practice guidelines in the National Guideline Clearinghouse. PLoS One. 2012;7(11):e47343. doi:10.1371/journal.pone.0047343</p>
<p>2. Norris SL, Burda BU, Holmer HK, Ogden LA. Conflicts of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153. doi:10.1371/journal.pone.0025153</p>
<p>3. Mandrioli D, Kearns CE, Bero LA. Relationship between research outcomes and risk of bias, study sponsorship, and author financial conflicts of interest in reviews of the effects of artificially sweetened beverages on weight outcomes: a systematic review of reviews. PLoS One. 2016;11(9):e0162198. doi:10.1371/journal.pone.0162198</p>
<sup>1</sup>Centre for Evidence-Based Medicine, University of Split School of Medicine, Split, Croatia, miro.vukovic@mefst.hr.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.</p>


Prevalence and Nature of Conflict of Interest Disclosures in Published Health Technology Assessment Reports

<p>Andreas Lundh,<sup>1,2,3</sup> Isabelle Boutron,<sup>4,5,6</sup> Lesley A. Stewart,<sup>7</sup> Asbjørn Hróbjartsson<sup>1,2</sup></p>
<h4>Objective </h4>
<p>Industry funding and authors’ financial conflicts of interest are common in clinical trials and associated with favorable trial conclusions and seemingly greater treatment effects. However, there has been little guidance on how to take funding and conflicts of interest into account in evidence synthesis. To address this, we developed the Tool for Addressing Conflicts of Interest in Trials (TACIT).</p>
<h4>Design</h4>
<p> We established an international working group of 20 experts in trial and systematic review methods, conflicts of interest, and biomedical publishing. Four substudies informed tool development. First, a systematic review found that current study appraisal tools address funding and conflicts of interest superficially.<sup>1</sup> Second, an interview study of experienced trial researchers found that funding and conflicts of interest may unduly influence trials through various mechanisms but influence may be minimized if properly managed.<sup>2</sup> Third, a cross-sectional study of trials included in Cochrane reviews found that searching beyond trial publications using a structured information retrieval approach may improve identification of trial funding and researchers’ conflicts of interest.<sup>3</sup> Fourth, an interview and questionnaire study of expected users piloting a preliminary version of TACIT found that users generally had a positive experience but suggested more guidance on ease of use and how to deal with insufficient information on conflicts of interest. The tool has been iteratively revised mainly based on feedback from the working group and expected users. A final consensus meeting was held in May 2025, where the result of substudies, themes for discussion, and a near-final version of the tool were presented. Based on the consensus meeting, the tool will be revised and made public together with a Microsoft Excel–based application after summer 2025.</p>
<h4>Results</h4>
<p> Through a series of guidance questions, the tool facilitates a judgment of notable concern about conflicts of interest and the sufficiency of information that the judgment of concern was based on (<b>Table 25-0917</b>). The judgment involves 3 levels: trial funders, the trial’s primary academic researchers, and overall trial level. The tool provides an overview of funding and conflicts of interest information for trials included in the systematic review, and it categorizes trials into trials with or without notable concern about conflicts of interest. Results of the assessments may be incorporated into review syntheses using subgroup or sensitivity analysis.</p>

![](https://assets.underline.io/markdown_image/1/image/df2f2880de816dfdbf399fc687a72b5b.png)

<h4>Conclusions </h4>
<p>TACIT provides a structured, coherent, and transparent assessment of conflicts of interest in trials included in systematic reviews. The tool may be used by systematic reviewers, guideline panels, and other users undertaking evidence syntheses.</p>
<h4>References</h4>
<p>1. Lundh A, Rasmussen K, Østengaard L, Boutron I, Stewart LA, Hróbjartsson A. Systematic review finds that appraisal tools for medical research studies address conflicts of interest superficially. J Clin Epidemiol. 2020;120:104-115. doi:10.1016/j.jclinepi.2019.12.005</p>
<p>2. Østengaard L, Lundh A, Tjørnhøj-Thomsen T, et al. Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study. BMJ. 2020;371:m3764. doi:10.1136/bmj.m3764</p>
<p>3. Faltinsen E, Todorovac A, Boutron I, Stewart LA, Hróbjartsson A, Lundh A. A structured approach to information retrieval improved identification of funding and researchers’ conflicts of interest in trials included in Cochrane reviews. J Clin Epidemiol. 2023;161:104-115. doi:10.1016/j.jclinepi.2023.06.020</p>
<sup>1</sup>Cochrane Denmark &amp; Centre for Evidence-Based Medicine Odense (CEBMO), Department of Clinical Research, University of Southern Denmark, Odense, Denmark, alundh@health.sdu.dk; <sup>2</sup>Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark; <sup>3</sup>Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital—Bispebjerg and Frederiksberg, Copenhagen, Denmark; <sup>4</sup>Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAe, Centre for Research in Epidemiology and Statistics (CRESS), Paris, France; <sup>5</sup>Centre d’Épidémiologie Clinique, Hôpital Hôtel Dieu, Université Paris Cité, Paris, France; <sup>6</sup>Cochrane France, Paris, France; <sup>7</sup>Centre for Reviews and Dissemination, University of York, York, UK.</p>
<h4>Conflict of Interest Disclosures:</h4>
<p> Isabelle Boutron is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.</p>
<h4>Additional Information:</h4>
<p> The authors are members of the Tool for Addressing Conflicts of Interest in Trials (TACIT) Steering Group. We thank members of the TACIT working group and coauthors of TACIT substudies for their contribution to the development of TACIT.</p>


Development of a Tool for Addressing Conflicts of Interest in Trials (TACIT) for Use in Systematic Reviews

<p>Marc-Andre Gagnon,<sup>1</sup> Blue Miaoran Dong<sup>2</sup></p>
<h4>Objective </h4>
<p>This study investigates BP’s influence on peer-reviewed research in the wake of the 2010 Deepwater Horizon oil spill, with particular attention to how corporate oversight shaped the production and publication of scientific knowledge about the disaster and its aftermath. Despite the scale and impact of the incident, no academic work to date has systematically analyzed BP’s internal documents—made public during litigation—or assessed their role in steering scholarly discourse through subtle or direct influence on academic journals.</p>
<h4>Design </h4>
<p>Our study, grounded in 9 internal BP documents made public by The Downs Law Group (<a href="https://downslawgroup.com/bp-papers/">https://downslawgroup.com/bp-papers/</a>), systematically examines BP’s publication strategies. These documents reveal the extensive oversight of BP’s legal and editorial teams in shaping the content of academic articles related to the spill. By analyzing publication plans, journal selection processes, thematic focuses, and the assertions made in these papers, we provide crucial insights into the challenges and manipulations inherent in peer review, scientific publication, and research dissemination.</p>
<h4>Results </h4>
<p>The analysis highlights BP’s direct role in the production of 37 peer-reviewed articles, as identified in one of the 9 leaked internal documents, “Scientific Literature Publication Tracker.”<sup>1</sup> The body of literature under review was published between 2011 and 2024. Collaborating with institutions such as Florida International University, Oregon State University, SINTEF research company, and Commonwealth Scientific and Industrial Research Organisation, BP shaped research agendas, defined testing protocols, and influenced interpretations of results. A large portion of the articles (33) focused on “seeps,” emphasizing natural seepage and exploration risks, while other themes included surf waters (18 articles), dispersants (12 articles), oil weathering (11 articles), and Corexit (6 articles). These studies often advanced conclusions that aligned with BP’s strategic interests, such as promoting the ecological and economic benefits of chemical dispersants. Notably, less frequent topics like phototoxicity (3 articles) or biological impacts received comparatively limited attention. BP’s editorial practices frequently framed findings within narrowly defined scopes to minimize ecological and legal accountability, steering the scientific narrative away from issues that could increase its liability. Among 37 articles, 17 did not disclose BP’s involvement in the writing process, while 21 articles explicitly acknowledged BP’s role. Although the 21 articles mentioned BP’s funding and their receipt of a portion of the $500 million allocated through the Gulf of Mexico Research Initiative, they did not disclose any editorial review conducted by BP (<b>Table 25-0937</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/d667da5e620cbd08219a2803f707cdb7.png)

<h4>Conclusions </h4>
<p>The findings reveal vulnerabilities in the peer-review process, where corporate influence can compromise the objectivity and integrity of scientific research. BP’s involvement demonstrates how editorial control and selective dissemination of findings can obscure accountability and shift public discourse. Enhanced transparency and robust safeguards are essential to protect the integrity of research and ensure it prioritizes public interest over corporate objectives.</p>
<h4>Reference</h4>
<p>1. BP. BP Scientific Literature Publication Tracker. Published online November 29, 2022. Accessed May 19, 2025. <a href="https://downslawgroup.com/bp-papers/"><span class="Hyperlink CharOverride-6">https://downslawgroup.com/bp-papers/</span></a></p>
<sup>1</sup>The School of Public Policy, Carleton University, Ottawa, ON, Canada, ma.gagnon@carleton.ca; <sup>2</sup>School of Journalism and Communication, Carleton University, Ottawa, ON, Canada.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support </h4>
<p>This research is financially supported by an Insight Grant awarded by the Social Sciences and Humanities Research Council of Canada (SSHRC) under the reference number 435-2021-0715, along with funding from the SSHRC Initiative for Digital Citizen Research under the reference number 1403-2021-0715.</p>
<h4>Role of the Funder/Sponsor</h4>
<p> SSHRC did not play any role in the work described in the abstract.</p>


Next from The 10th Peer Review Congress

Topic and Knowledge-base Interdisciplinarity in Manuscripts Submitted to Physical Science Journals vs Editorial Decision and Reviewer Positivity

Similar lecture

Factors Associated With Author and Reviewer Declared Use of AI in Medical Journals