United States

&lt;p&gt;Nina Vitlov,&lt;sup&gt;1&lt;/sup&gt; Nensi Bralić,&lt;sup&gt;1&lt;/sup&gt; Tina Poklepović Peričić,&lt;sup&gt;2&lt;/sup&gt; Daniel Garcia-Costa,&lt;sup&gt;3&lt;/sup&gt; Emilia López-Iñesta,&lt;sup&gt;4&lt;/sup&gt; Elena Álvarez-García,&lt;sup&gt;3&lt;/sup&gt; Francisco Grimaldo,&lt;sup&gt;3&lt;/sup&gt; Ana Marušić&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective&lt;/h4&gt;
&lt;p&gt; To explore how GRADE and Cochrane experts, clinicians, and health care consumers interact with Summary of Findings (SoF) tables from systematic reviews of evidence when answering questions about table content.&lt;/p&gt;
&lt;h4&gt;Design&lt;/h4&gt;
&lt;p&gt; A quasi-experimental study was conducted from February to June 2025, using the Read&amp;amp;Learn platform, an online tool designed for monitoring the interaction with a word problem or a table.&lt;sup&gt;1,2&lt;/sup&gt; The sample size was calculated at 25 participants per group to detect a mean (SD) of 4 (5) points’ difference in the number of correct answers (range, 0-16), with 80% power and 5% α level. The study intervention was blurring of SoF table cells, which the participants were required to navigate and open in order to answer questions about 4 different SoF tables, alternating between the question screen and table cells. All necessary data for answering the questions were contained within the target cells, requiring no additional calculations. We measured the number of correct answers to the questions about SoF tables, total number of table cells visited, number of target table cells visited, number of nontarget table cells visited, total time spent on the quiz from beginning to end, time spent on initial reading, total time spent reading table cells, time spent on reading target table cells, and time spent on reading nontarget table cells. The study was approved by the Ethics Committee of the University of Split School of Medicine, and all participants gave informed consent.&lt;/p&gt;
&lt;h4&gt;Results&lt;/h4&gt;
&lt;p&gt; We collected the data from 40 experts, 40 clinicians, and 40 health care consumers (&lt;b&gt;Table 25-0884&lt;/b&gt;). There were no significant differences between experts, clinicians, and health care consumers in terms of quiz performance, with all groups achieving a similar number of correct answers. Additionally, total time spent on the quiz, initial reading, and quiz questions did not differ between the groups. Significant differences were observed in their interaction with the SoF tables. Experts visited fewer numbers of table cells in total and fewer nontarget segments and spent the least amount of time reading the nontarget segments, compared with clinicians and health care consumers. They also spent the least amount of time reading the question statement.&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/341c5dad8e21fa52c9cd92a2a6c0f0c1.png)

&lt;h4&gt;Conclusions&lt;/h4&gt;
&lt;p&gt; Experts in evidence synthesis, Cochrane systematic review authors, and GRADE experts showed different interaction patterns with the SoF table, possibly reflecting their greater familiarity with the content. Future studies, including qualitative approaches, should further explore how different users approach and use the information presented in SoF tables.&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Matas J, Tokalić R, García-Costa D, et al. Tool to assess recognition and understanding of elements in Summary of Findings table for health evidence synthesis: a cross-sectional study. &lt;i&gt;Sci Rep&lt;/i&gt;. 2023;13:18044. doi:10.1038/s41598-023-45359-x&lt;/p&gt;
&lt;p&gt;2. Sanz MT, López-Iñesta E, García-Costa D, Grimaldo F. Measuring arithmetic word problem complexity through reading comprehension and learning analytics. &lt;i&gt;Mathematics&lt;/i&gt;. 2020;8(9):1556. doi:10.3390/math8091556&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia, nina.vitlov@mefst.hr; &lt;sup&gt;2&lt;/sup&gt;Department of Prosthodontics, School of Dental Medicine, University of Split School of Medicine, Split, Croatia; &lt;sup&gt;3&lt;/sup&gt;Department of Computer Science, Universitat de València, València, Spain; &lt;sup&gt;4&lt;/sup&gt;Department of Didactics of Mathematics, Universitat de València, València, Spain.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures &lt;/h4&gt;
&lt;p&gt;Nina Vitlov is funded by the Croatian Science Foundation under the Programme for Career Development of Early Career Researchers-Training of New Doctoral Students, NPOO (C3 2 R2-I1). Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.&lt;/p&gt;


The 10th Peer Review Congress

Comparative Analysis of Expert, Clinician, and Consumer Interactions With Summary of Findings Tables: A Quasi-Experimental Study

data presentation and graphical display

diversity and inclusion

quality assurance

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress) [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ) [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)

You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

Nina Vitlov,1 Nensi Bralić,1 Tina Poklepović Peričić,2 Daniel Garcia-Costa,3 Emilia López-Iñesta,4 Elena Álvarez-García,3 Francisco Grimaldo,3 Ana Marušić1
<h4>Objective</h4>
 To explore how GRADE and Cochrane experts, clinicians, and health care consumers interact with Summary of Findings (SoF) tables from systematic reviews of evidence when answering questions about table content.
<h4>Design</h4>
 A quasi-experimental study was conducted from February to June 2025, using the Read&amp;Learn platform, an online tool designed for monitoring the interaction with a word problem or a table.1,2 The sample size was calculated at 25 participants per group to detect a mean (SD) of 4 (5) points’ difference in the number of correct answers (range, 0-16), with 80% power and 5% α level. The study intervention was blurring of SoF table cells, which the participants were required to navigate and open in order to answer questions about 4 different SoF tables, alternating between the question screen and table cells. All necessary data for answering the questions were contained within the target cells, requiring no additional calculations. We measured the number of correct answers to the questions about SoF tables, total number of table cells visited, number of target table cells visited, number of nontarget table cells visited, total time spent on the quiz from beginning to end, time spent on initial reading, total time spent reading table cells, time spent on reading target table cells, and time spent on reading nontarget table cells. The study was approved by the Ethics Committee of the University of Split School of Medicine, and all participants gave informed consent.
<h4>Results</h4>
 We collected the data from 40 experts, 40 clinicians, and 40 health care consumers (Table 25-0884). There were no significant differences between experts, clinicians, and health care consumers in terms of quiz performance, with all groups achieving a similar number of correct answers. Additionally, total time spent on the quiz, initial reading, and quiz questions did not differ between the groups. Significant differences were observed in their interaction with the SoF tables. Experts visited fewer numbers of table cells in total and fewer nontarget segments and spent the least amount of time reading the nontarget segments, compared with clinicians and health care consumers. They also spent the least amount of time reading the question statement.

![](https://assets.underline.io/markdown_image/1/image/341c5dad8e21fa52c9cd92a2a6c0f0c1.png)

<h4>Conclusions</h4>
 Experts in evidence synthesis, Cochrane systematic review authors, and GRADE experts showed different interaction patterns with the SoF table, possibly reflecting their greater familiarity with the content. Future studies, including qualitative approaches, should further explore how different users approach and use the information presented in SoF tables.
<h4>References</h4>
1. Matas J, Tokalić R, García-Costa D, et al. Tool to assess recognition and understanding of elements in Summary of Findings table for health evidence synthesis: a cross-sectional study. Sci Rep. 2023;13:18044. doi:10.1038/s41598-023-45359-x
2. Sanz MT, López-Iñesta E, García-Costa D, Grimaldo F. Measuring arithmetic word problem complexity through reading comprehension and learning analytics. Mathematics. 2020;8(9):1556. doi:10.3390/math8091556
1Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia, nina.vitlov@mefst.hr; 2Department of Prosthodontics, School of Dental Medicine, University of Split School of Medicine, Split, Croatia; 3Department of Computer Science, Universitat de València, València, Spain; 4Department of Didactics of Mathematics, Universitat de València, València, Spain.
<h4>Conflict of Interest Disclosures </h4>
Nina Vitlov is funded by the Croatian Science Foundation under the Programme for Career Development of Early Career Researchers-Training of New Doctoral Students, NPOO (C3 2 R2-I1). Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.

poster

Sina Rashedi,1 Farbod Zahedi Tajrishi,2 Ashkan Hashemi,3 Isaac Dreyfus,4 Nicholas Varunok,5 John Burton,6 Seng Chan You,7 Bjorn Redfors,8,9 Gregory Piazza,1,10 Joshua D. Wallach,11 Lesley Curtis,12 Sanjay Kaul,13 David J. Cohen,14,15 Roxana Mehran,16 Mitchell S. V. Elkind,17 Flavia Geraldes,18 Joseph S. Ross,19,20 Jane A. Leopold,10 Harlan M. Krumholz,19,21 Gregg W. Stone,16 Behnood Bikdeli1,10,19
<h4>Objective </h4>
A large number of cardiovascular randomized clinical trials (RCTs) do not meet their primary outcome (neutral or negative trials). Some of these negative/neutral RCTs show positive secondary findings. We aimed to assess the proportion of cardiovascular disease– or stroke-related RCTs with negative or neutral primary results but positive secondary findings among all negative/neutral cardiovascular RCTs published in the highest-impact medical journals and to evaluate whether these findings were pursued in subsequent confirmatory RCTs or influenced clinical practice recommendations or regulatory decisions.
<h4>Design</h4>
 We searched PubMed to identify cardiovascular RCTs published in 3 major clinical journals (NEJM, The Lancet, and JAMA) between 2014 and 2019. Negative/neutral RCTs with positive secondary findings were defined as those that did not meet their (co)primary outcome(s) (P &gt; .05) but showed positive effects in prespecified subgroup analyses for the primary outcome and/or for prespecified secondary outcomes among the entire study population. Secondary outcomes and subgroup analyses were considered if they were reported as prespecified in the trial publications. We investigated whether these positive secondary findings were examined in subsequent confirmatory RCTs via PubMed searches or directly resulted in changes in either professional society recommendations or regulatory authority endorsements in the US or Europe through December 2024. Two investigators independently performed the literature search and data extraction, with discrepancies resolved by a third investigator.
<h4>Results </h4>
Overall, 435 cardiovascular RCTs were identified; the median time from the index trial to December 2024 was 7.5 (IQR, 5.5-9.5) years. Of these 435 RCTs, the primary outcome(s) were not met in 174 trials (40.0%), of which 93 (53.4%) had at least 1 positive secondary finding. Among these trials, 51 (54.8%) had only positive secondary outcomes, 20 (21.5%) had only positive findings in subgroup analyses, and 22 (23.7%) had positive findings in both secondary outcomes and subgroup analyses. Among the 93 negative/neutral RCTs with positive secondary findings, 14 (15.1%) were consequential: subsequent RCTs were performed due to positive secondary findings for 2 trials; the secondary results from 12 RCTs contributed to changes in professional society recommendations; and 2 trials contributed to changes in labeled indications (Figure 25-0928).

![](https://assets.underline.io/markdown_image/1/image/b487af44493837ae64ec3a334989da10.png)

<h4>Conclusions</h4>
 Although nearly half of the negative/neutral cardiovascular trials published in the highest-impact journals report positive secondary findings, only a small proportion directly influence practice or policy, and very few are followed by subsequent confirmatory RCTs. Our findings caution against overemphasizing positive secondary results from cardiovascular RCTs and underscore the need for subsequent confirmatory RCTs to evaluate positive secondary findings from otherwise negative/neutral trials. Further research is planned to assess the generalizability of these findings in major specialty cardiovascular journals and to identify trial- and outcome-level characteristics that influence the selection of positive secondary findings for replication in subsequent trials.
1Thrombosis Research Group, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US, srashedi@bwh.harvard.edu; 2Tulane University School of Medicine, New Orleans, LA, US; 3Program for the Care and Study of the Aging Heart, Department of Medicine, Weill Cornell Medicine, New York, NY, US; 4Division of Cardiology, Department of Medicine, University of California Los Angeles, Los Angeles, CA, US; 5Vanderbilt University Medical Center, Nashville, TN, US; 6University of Southern California, Los Angeles, CA, US; 7Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea; 8Department of Cardiology, Sahlgrenska University Hospital, Gothenburg, Sweden; 9Cardiovascular Research Foundation, New York, NY, US; 10Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; 11Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US; 12Duke University School of Medicine, Department of Population Health Sciences, Durham, NC, US; 13Department of Cardiology, Cedars-Sinai Medical Center, Los Angeles, CA, US; 14Clinical Trials Center, Cardiovascular Research Foundation, New York, NY, US; 15St Francis Hospital and Heart Center, Roslyn, NY, US; 16Icahn School of Medicine at Mount Sinai, New York, NY, US; 17Department of Neurology, Columbia University Irving Medical Center, New York, NY, US; 18The Lancet, London, United Kingdom; 19YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, CT, US; 20Department of General Internal Medicine, Yale School of Medicine, New Haven, CT, US; 21Section of Cardiovascular Medicine, Yale School of Medicine, New Haven, CT, US.
<h4>Conflict of Interest Disclosures </h4>
Outside the submitted work, Behnood Bikdeli reported receiving the following support: a Career Development Award from the American Heart Association and VIVA Physicians (#938814), the Scott Schoen and Nancy Adams IGNITE Award, the Mary Ann Tynan Research Scientist Award from the Mary Horrigan Connors Center for Women’s Health and Gender Biology at Brigham and Women’s Hospital, and the Heart and Vascular Center Junior Faculty Award from Brigham and Women’s Hospital; serving as consulting expert on behalf of the plaintiff for litigation related to 2 specific brand models of IVC filters (although he has not been involved in the litigation in 2022-2025, nor has he received any compensation in 2022-2025); serving as a member of the Medical Advisory Board for the North American Thrombosis Forum (now VascuLearn Network) and on the Data Safety and Monitoring Board of the NAIL-IT trial funded by the National Heart, Lung, and Blood Institute and Translational Sciences; serving as a collaborating consultant with the International Consulting Associates and the US Food and Drug Administration in a study to generate knowledge about the utilization, predictors, retrieval, and safety of IVC filters; receiving compensation as an associate editor for NEJM Journal Watch Cardiology, as an associate editor for Thrombosis Research, and as an executive associate editor for JACC; and serving as a section editor for Thrombosis and Haemostasis without compensation.
<h4>Additional Information</h4>
 Behnood Bikdeli is a co–corresponding author (behnood.bikdeli@yale.edu).

Replication and Impact of Positive Secondary Findings in Negative or Neutral Cardiovascular Trials

Christof Manuel Schönenberger,1Malena Chiaborelli,1 Ala Taji Heravi,1 Lukas Kübler,2 Pooja Gandhi,3 Zsuzsanna Kontar,4Julia Hüllstrung,1 Mona Elalfy,1 Jan Glasstetter,1 Dmitry Gryaznov,5 Belinda von Niederhäusern,6 Anette Blümle,7 Jason W. Busse,8 Szimonetta Lohner,4Sally Hopewell,9 Alexandra Griessbach,1Matthias Briel,1,10 Benjamin Speich1
<h4>Objective</h4>
 Making protocols of randomized clinical trials (RCTs) publicly available is important for the trustworthiness and quality of medical research.1 In a previous study, only 36% of RCTs that received ethical approval in 2012 had a publicly available protocol.2 Repeating this study with RCTs approved in 2016, we aimed to generate current evidence on the availability of RCT protocols, sources where they are shared, and changes over time.
<h4>Design</h4>
 Using a sample of RCTs receiving ethical approval in 2016 in Switzerland, Canada, Germany, or the UK,3 we investigated the number of available protocols. For all RCTs, we checked if a result publication was available by searching PubMed, Google Scholar, trial registries, and Google. We extracted baseline characteristics for all RCTs. Up to June 2024, we systematically searched for (1) protocols available as peer-reviewed publications, (2) protocols attached to trial registries, and (3) protocols shared with result publications of RCTs. Multiple sources per protocol were possible. All searches and data extraction were performed in duplicate. We used multivariable logistic regression to examine the association of protocol availability with trial characteristics such as sample size, drug vs nondrug interventions, multicenter vs single-center status, and RCT approval in 2016 vs 2012.
<h4>Results</h4>
 Of 347 included RCTs, 228 (65.7%) had an available protocol (Table 25-0957); 150 protocols (43.2%) were available as files on trial registries, 91 (26.2%) as supplementary material to a result publication, and 81 (23.3%) as peer-reviewed publications. Protocol availability improved over time (65.7% in 2016 vs 36.2% in 2012), particularly in industry trials (83.4% in 2016 vs 34.6% in 2012), but improved little in nonindustry trials (46.4% 2016 vs 38.1% 2012). In a regression model with pooled data from trials approved in 2012 and approved in 2016, trials with a medium sample size (100-500 participants) (odds ratio [OR], 2.12; 95% CI, 1.32-3.43; P = .001), trials with a large sample size (&gt;500 participants) (OR, 6.12; 95% CI, 3.45-11.07; P &lt; .001), and multicenter trials (vs single-center trials) (OR, 2.66; 95% CI, 1.59-4.52; P &lt; .001) showed significantly higher odds of having an available protocol. A significant interaction term indicated dependence of sponsorship and year (OR, 0.13; 95% CI 0.06-0.27; P &lt; .001).

![](https://assets.underline.io/markdown_image/1/image/b7e71ef2066ff55aa9831845fed69144.png)

<h4>Conclusions </h4>
The availability of protocols increased in RCTs approved in 2016 compared with RCTs approved in 2012. This was mainly driven by industry-sponsored trials and may be explained by updated US regulations. Since 2017, all trials investigating drugs or devices that are or are intended to be approved, licensed, or cleared by the US Food and Drug Administration must have a publicly available protocol. Efforts to further improve protocol availability should be continued, especially in nonindustry-sponsored RCTs.
<h4>References</h4>
1. Chan AW, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257-266. doi:10.1016/S0140-6736(13)62296-5
2. Schönenberger CM, Griessbach A, Taji Heravi A, et al. A meta-research study of randomized controlled trials found infrequent and delayed availability of protocols. J Clin Epidemiol. 2022;149:45-52. doi:10.1016/j.jclinepi.2022.05.014
3. Gryaznov D, Odutayo A, von Niederhäusern B, et al. Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to Spirit Recommendations (ASPIRE) study and associated projects. Trials. 2020;21(1):896. doi:10.1186/s13063-020-04808-y
1Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, christofmanuel.schoenenberger@usb.ch; 2Department of Biomedicine, University Hospital Basel, University of Basel, Basel, Switzerland; 3University Health Network, Toronto, Ontario, Canada; 4MTA-PTE Lendület “Momentum” Evidence in Medicine Research Group, Department of Public Health Medicine, Medical School, University of Pécs, Pécs, Hungary; 5Viatris Innovation GMBH, Allschwil, Switzerland; 6Roche Pharma AG, Grenzach-Wyhlen, Germany; 7Clinical Trials Unit, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany; 8Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada; 9Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK; 10Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
<h4>Conflict of Interest Disclosures</h4>
 None reported.
<h4>Funding/Support</h4>
 This project was supported by the Swiss Federal Office of Public Health (to Matthias Briel). Christof Manuel Schönenberger was funded by the Janggen Pöhn Foundation and the Swiss National Science Foundation (MD-PhD grant number 323530_221860).
<h4>Role of the Funder/Sponsor</h4>
 The funders had no role in the conduct of the study, the analysis of the data, or the writing of the abstract.
<h4>Acknowledgment</h4>
 We thank all participating research ethics committees from Germany (Freiburg), Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St Gallen, Frauenfeld, and Zurich), Canada (Hamilton), and the UK (National Health Service Health Research Authority) for their support and cooperation.

Public availability of Randomized Clinical Trial Protocols: A Repeated Cross-Sectional Study

Carolina Grana,1,2,3 Lina Ghosn,1,2,3 Carolina Riveros,1,2,3 Philippe Ravaud,1,2,3 Isabelle Boutron1,2,3
<h4>Objective</h4>
 The World Health Organization (WHO) requests that “key outcomes are to be made publicly available within 12 months of study completion by posting to the results section of the primary clinical trial registry.”1 In some regions, this requirement is also mandated by law2; however, evidence indicates low adherence.3 The extent to which journal editors support this request remains unclear. This study aimed to assess adherence to the WHO request among randomized clinical trials (RCTs) published in high-impact journals and to examine whether journal editors encourage this practice in their recommendations to authors.
<h4>Design</h4>
 We conducted a cross-sectional study and reported our findings according to the STROBE guideline. On April 18, 2025, we searched PubMed using the terms date-create: 01/01/2024 to 01/01/2025 AND journal: [Name of the journal], applying the “randomized controlled trial” filter. We selected the 5 journals with the highest impact factors in the “medicine, general &amp; internal” category and in selected medical specialties based on the 2023 Clarivate Journal Citation Reports (Web of Science). We included RCTs evaluating pharmacologic, biologic, or medical device interventions. Data were extracted in duplicate from each trial publication and corresponding primary registry entry. We recorded the publication date, the trial’s primary completion date, and results status in the registry. The primary outcome was the proportion of RCTs that at the time of publication had results posted in the primary registry within 12 months of trial completion. The analysis was primarily descriptive, comparing the timing of trial completion, results posting, and publication.
<h4>Results</h4>
 Preliminary results (publications up to June 2024) identified 185 RCTs published across 18 journals. Among these, 142 (77%) evaluated pharmacologic interventions and 108 (58%) were multinational. Trial sites included at least 1 European Union country in 102 trials (55%), the US in 94 (51%), and the UK in 74 (40%). All 185 trials were registered, and 163 (88%) registrations were prospective. A total of 100 trials (54%) were published more than 12 months after their primary completion date. Among these, at the time of publication, only 23 (23%) had adhered to WHO requirements by having results posted in the trial registry within 12 months of completion. Conversely, 83 trials (45%) were published within 12 months of completion, and 45 of these (54%) had results posted in the registry when the trial report was published. Completion date was not reported for 2 trials (1%). None of the 18 journals assessed requested public posting of trial results in registries. 
<h4>Conclusions</h4>
 Preliminary findings from trials included in this study, all published in high–impact factor journals, revealed low adherence to the WHO call for timely posting of results to the registries.
<h4>References</h4>
1. WHO statement on public disclosure of clinical trial results. World Health Organization. Accessed January 10, 2025. <a href="https://www.who.int/news/item/09-04-2015-japan-primary-registries-network">https://www.who.int/news/item/09-04-2015-japan-primary-registries-network</a>
2. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 2011;364(9):852-860. doi:10.1056/NEJMsa1012065
3. DeVito NJ, Bacon S, Goldacre B. Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. Lancet. 2020;395(10221):361-369. doi:10.1016/S0140-6736(19)33220-9
1Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAe, Centre for Research in Epidemiology and Statistics (CRESS), Paris, France, isabelle.boutron@aphp.fr; 2Centre d’Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, Paris, France; 3Cochrane France, Paris, France.
<h4>Conflict of Interest Disclosures </h4>
Isabelle Boutron is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.
<h4><a id="_idTextAnchor108"></a>Additional Information</h4>
 Carolina Grana (carolinaestela.granapossamai@aphp.fr) and Lina Ghosn (lina.elchall@aphp.fr) are co–corresponding authors.

Adherence to the WHO Statement on Public Disclosure of Clinical Trial Results by Trials Published in High Impact Factor Journals

Ioana Alina Cristea,1,2 Florian Naudet,2,3,4 Guillaume Cabanac,4,5 John P. A. Ioannidis2,6,7,8
<h4>Objective </h4>
An older, field-wide analysis1 of studies published between 2005 and 2017 estimated the prevalence of prospectively registered randomized controlled trials (RCTs) at 21%. Recent estimates indicated higher rates, eg, approximately 59% for rheumatology RCTs published between 2009 and 2022 in 5 International Committee of Medical Journal Editors (ICMJE) journals.2 There are no recent field-wide estimates of the prevalence of prospective registration across clinical specialties and journals and of the related outcome reporting bias (ie, discrepancies between registered and published outcomes). We report these across a randomly selected sample of recently published RCTs.
<h4>Design </h4>
We conducted a retrospective cohort study, reported following the STROBE guidelines. We searched PubMed with the Cochrane Highly Sensitive Search Strategy3 on January 15, 2025. RCTs evaluating health-related interventions and outcomes (per the ICMJE definition) published in 2024 were included. Search results were randomized using Entrez Programming Utilities and the Linux shuf command, and a sample of 1720 records was selected. One researcher (I.A.C.) manually checked titles and abstracts to select RCTs and extracted trial registration information from publications reporting on primary outcomes (POs). Prospective registration was defined as submission date to registry preceding recruitment or study start or postdating by less than 30 days. We favored actual vs estimated study start dates when available in publications or registries. For prospectively registered trials, 1 researcher (I.A.C.) extracted all POs registered and declared in publications, and tabulated all discrepancies regarding outcome domain, measurement, or time point.
<h4>Results </h4>
Of 324,177 records identified, 1720 were screened, leading to 136 eligible RCTs. Six trials were excluded (inaccessible full text [n = 3]; Chinese language [n = 3]). Eighty-two of 130 publications (63%) reported study start dates, with a median (IQR) of 2020 (2) (range, 2013-2023). Sixty-nine of 130 RCTs (53% [95% CI, 44%-62%]) were prospectively registered (67 in ICMJE-compliant registries), including 4 registered while recruiting. Sixty-one of 130 RCTs (47% [95% CI, 38%-56%]) were declared nonregistered (n = 4), were registered retrospectively (n = 27), indicated a registration number for another trial (n = 2), or did not mention registration (n = 28). Registration delays were a median (IQR) of 632 (533) (range, 49-1524) days. For 24 of 60 nonprospectively registered trials, journal instructions to authors explicitly required prospective registration or compliance with ICMJE or Declaration of Helsinki registration requirements. Of 69 prospectively registered trials, 6 did not declare POs in publications and 2 insufficiently specified POs in the registries, while 37 (54% [95% CI, 41%-66%]) had no substantive discrepancies between registry and publication. The remaining 24 of 69 RCTs (35% [95% CI, 24%-47%]) included 1 (n = 17), 2 (n = 6), or 3 (n = 1) types of discrepancies (Table 25-1102).
<a href="/wp-content/uploads/2025_conference/tables/prc25-1102t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1102t1.png"> </a>
<h4>Conclusions </h4>
In a random sample of RCTs published in 2024 across clinical specialties, approximately half were prospectively registered and approximately one-third of these contained substantial changes between registered and reported primary outcomes.
<h4>References</h4>
1. Trinquart L, Dunn AG, Bourgeois FT. Registration of published randomized trials: a systematic review and meta-analysis. BMC Med. 2018;16(1):173. doi:10.1186/s12916-018-1168-6
2. Mongin D, Buitrago-Garcia D, Capderou S, et al. Prospective registration of trials: where we are, why, and how we could get better. J Clin Epidemiol. 2024;176. doi:10.1016/j.jclinepi.2024.111586
3. Lefebvre C, Glanville J, Briscoe S, et al. Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions, Version 6.5. Cochrane; 2024
1Department of General Psychology, University of Padova, Padova, Italy, ioanaalina.cristea@unipd.it; 2Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US; 3Université Rennes, CHU Rennes, Inserm, Centre d’investigation clinique de Rennes (CIC1414), service de pharmacologie clinique, Institut de recherche en santé, environnement et travail (Irset), UMR S 1085, EHESP, 35000, Rennes, France; 4Institut Universitaire de France (IUF), Paris, France; 5Université de Toulouse, IRIT (UMR 5505 CNRS), Toulouse, France; 6Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, US; 7Department of Epidemiology &amp; Population Health, Stanford University School of Medicine, Stanford, CA, US; 8Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, US.
<h4>Conflict of Interest Disclosures</h4>
 John P. A. Ioannidis is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.
<h4>Funding/Support</h4>
 Ioana Alina Cristea is supported by a European Research Council (ERC) Starting Grant (grant agreement: 101042701; <a href="https://cordis.europa.eu/project/id/101042701">https://cordis.europa.eu/project/id/101042701</a>), funded by the European Union (EU). Florian Naudet received funding from the French National Research Agency, the French Ministry of Health, and the French Ministry of Research and is a work package leader in the OSIRIS project (Open Science to Increase Reproducibility in Science) and for the doctoral network MSCA-DN SHARE-CTD (HORIZON-MSCA-2022-DN-01 101120360), funded by the EU. Guillaume Cabanac received funding from the Institut Universitaire de France. John P. A. Ioannidis is supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford University.
<h4>Role of the Funder/Sponsor </h4>
The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the abstract.
<h4>Additional Information </h4>
John P. A. Ioannidis is a co–corresponding author (jioannid@stanford.edu).

Prevalence of Prospective Registration and Primary Outcome Discrepancies in Recently Published Randomized Controlled Trials

Christopher C. Winchester,1 Mark J. Rolfe,1 William T. Gattrell,2 Patricia Logullo,3 Keith Goldman,4 Amy Price,5 Paul Blazey,6 Esther J. van Zuuren,7 Niall Harrison8
<h4>Objective </h4>
The ACcurate COnsensus Reporting Document (ACCORD) guideline supports the reporting of biomedical studies involving any consensus method.1 We evaluated its uptake by bibliometric analysis and qualitative and quantitative assessment of implementation activities 1 year post publication.
<h4>Design</h4>
 An implementation plan2 was established by the ACCORD Steering Committee before guideline publication, including developing an explanation and elaboration (E&amp;E) document, writing journal editorials, presenting at congresses, and encouraging journals and publishers to include ACCORD in author instructions. Guideline views and downloads (via PLOS Medicine,1 the publishing journal) and citations (Dimensions, plus additional references from the Steering Committee’s reference libraries) were obtained for the period from January 23, 2024, to January 22, 2025. Likely journal and publisher beneficiaries of ACCORD were identified by a literature search to determine the 10 journals that published the most consensus studies (Web of Science for 2020-2023) and other publications at the discretion of the Steering Committee. Relevant journal and publisher stakeholders were contacted directly or indirectly (eg, via journal websites). Activities were evaluated against the implementation plan2; journal and publisher responses were described.
<h4>Results</h4>
 In the 1 year following publication, the ACCORD guideline was viewed 13,752 times, downloaded (full-text PDF) 3790 times, and cited 105 times (5 citations [4.8%] were in articles by ACCORD authors, including the ACCORD E&amp;E3). Sources of guideline citations included consensus studies (67.6% [71 of 105]), reviews (11.4% [12 of 105]), study protocols (8.6% [9 of 105]), other reporting standards and guidelines (6.7% [7 of 105]), and editorials (3.8% [4 of 105]). Of the journals contacted by ACCORD Steering Committee members (between August 24, 2023 [preprint publication], and January 22, 2025), BMJ Open, British Journal of Dermatology, and Journal of Clinical Epidemiology mandated the use of ACCORD for consensus studies, while the British Journal of Sports Medicine, Internal Medicine Journal, and Pragmatic and Observational Research encouraged its use. PLOS has confirmed its intention to include the guideline in their author instructions. Although a further 4 journals have expressed interest in ACCORD and 1 publisher has offered to raise awareness via a webinar, they are yet to commit to its inclusion in author instructions; 2 journals and publishers have not responded to inquiries.
<h4>Conclusions </h4>
One year after publication, the ACCORD guideline has been cited and used to inform the design and reporting of consensus-based research. However, journal and publisher adoption has been limited, which may affect long-term uptake. Given the resources invested in developing reporting guidelines and their potential to improve reporting, the role of biomedical journals and publishers in their adoption warrants wider discussion.
<h4>References</h4>
1. Gattrell WT, Logullo P, van Zuuren EJ, et al. ACCORD (ACcurate COnsensus Reporting Document): a reporting guideline for consensus methods in biomedicine developed via a modified Delphi. PLOS Med. 2024;21:e1004326. doi:10.1371/journal.pmed.1004326
2. Gattrell WT, Hungin AP, Price A, et al. ACCORD guideline for reporting consensus-based methods in biomedical research and clinical practice: a study protocol. Res Integr Peer Rev. 2022;7:3. doi:10.1186/s41073-022-00122-0
3. Logullo P, van Zuuren EJ, Winchester CC, et al. ACcurate COnsensus Reporting Document (ACCORD) explanation and elaboration: guidance and examples to support reporting consensus methods. PLOS Med. 2024;21:e1004390. doi:10.1371/journal.pmed.1004390
1Oxford PharmaGenesis, Oxford, UK, chris.winchester@pharmagenesis.com; 2Independent Medical Communications Professional, Oxfordshire, UK; 3Bodleian Libraries, University of Oxford, Oxford, UK; 4Medical Affairs + Health Impact, AbbVie, North Chicago, IL, US; 5Dartmouth Institute for Health Policy &amp; Clinical Practice (TDI), Geisel School of Medicine, Dartmouth College, Hanover, NH, US; 6School of Kinesiology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada; 7Leiden University Medical Centre, Leiden, the Netherlands; 8OPEN Health Communications, London, UK.
<h4>Conflict of Interest Disclosures </h4>
Christopher C. Winchester is an employee, director, and shareholder of Oxford PharmaGenesis; a director of Oxford Health Policy Forum CIC; a trustee of the Friends of the National Library of Medicine; and an associate fellow of Green Templeton College, University of Oxford. Mark J. Rolfe is an employee of Oxford PharmaGenesis. William T. Gattrell was an independent medical communications professional at the time of this study and is currently an employee of Bristol Myers Squibb. Keith Goldman is an employee of AbbVie. Niall Harrison is an employee of OPEN Health Communications.
<h4>Funding/Support </h4>
Medical writing, editorial, and project management support for this abstract were provided by Oxford PharmaGenesis.
<h4>Acknowledgments</h4>
 Medical writing support was provided by Alison Chisholm, with editorial support from Jenny Thorp, and administrative support was provided by Mehraj Ahmed, Ryan Gamble, and Jessica Miller.
<h4>Additional Information </h4>
No authors were reimbursed for participating in the initiative.

Challenges in achieving uptake and journal endorsement of the ACcurate COnsensus Reporting Document (ACCORD) guideline

Aidan G. Cashin,1,2 Harrison J. Hansford,1,2 Miguel A. Hernán,3,4,5 Sonja A. Swanson,3,4,6 Hopin Lee,7,8 Matthew D. Jones,1,2 Issa J. Dahabreh,3,4,5,9 Barbra A. Dickerman,3,4 Matthias Egger,10,11,12 Xabier Garcia-Albeniz,3,13 Robert M. Golub,14 Nazrul Islam,15,16 Sara Lodi,3,17 Margarita Moreno-Betancur,18,19 Sallie-Anne Pearson,20 Sebastian Schneeweiss,21 Melissa K. Sharp,22 Jonathan A. C. Sterne,12,23,24 Elizabeth A. Stuart,25 James H. McAuley1,2
<h4>Objective</h4>
 When randomized trials are unavailable or not feasible, data from observational studies can be used, if assumptions hold, to answer causal questions about the comparative effects of interventions by emulating a hypothetical pragmatic randomized trial (target trial). The reporting of studies that emulate a target trial is inconsistent and often incomplete, limiting transparency and reproducibility. To address this knowledge gap, we developed consensus-based guidance for reporting analyses of observational data that aim to estimate causal effects by explicitly emulating a target trial.
<h4>Design</h4>
 The TARGET (Transparent Reporting of Observational Studies Emulating a Target Trial)1 guideline was developed using the Enhancing the Quality and Transparency of Health Research (EQUATOR) framework.2 This included (1) a systematic review3 of reporting practices in published studies that explicitly aimed to emulate a target trial, (2) a 2-round online survey (August 2023-March 2024; 18 expert participants from 6 countries) to generate agreement on the importance of candidate items selected from previous research and to identify additional items, (3) a 3-day expert consensus meeting (June 2024; 18 panelists) to refine the scope of the guideline and draft the checklist, and (4) an internal and external review and piloting activity with stakeholders and potential users (n = 108; September 2024-February 2025). The checklist was then refined based on feedback and finalized.
<h4>Results</h4>
 The 21-item TARGET checklist was organized into 6 sections (abstract, introduction, methods, results, discussion, other information) (Table 25-0970). The TARGET guideline was intended to be general, with a focus on nonrandomized studies of interventions explicitly emulating a parallel-group, individually randomized target trial, with adjustment for baseline confounders. Key recommendations were to (1) summarize the causal question and the reason for emulating a target trial; (2) clearly specify the target trial protocol (ie, causal estimand, identifying assumptions, data analysis plan) and how these components were mapped to the observational data; and (3) for each causal estimand, report the estimate obtained and its precision, along with findings from additional analyses to assess the sensitivity of estimates to assumptions and design and analysis choices.

![](https://assets.underline.io/markdown_image/1/image/1044ea19b2369c6f6dadcd84058920d2.png)

<h4>Conclusions</h4>
 The TARGET guideline provides recommendations on reporting of studies explicitly emulating a target trial. Use of the guideline should facilitate transparency in reporting to improve peer review and help researchers, clinicians, and other readers interpret and use the results.
<h4>References</h4>
1. Hansford HJ, Cashin AG, Jones MD, et al. Development of the TrAnsparent ReportinG of observational studies Emulating a Target trial (TARGET) guideline. BMJ Open. 2023;13(9):e074626. doi:10.1136/bmjopen-2023-074626
2. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217. doi:10.1371/journal.pmed.1000217
3. Hansford HJ, Cashin AG, Jones MD, et al. Reporting of observational studies explicitly aiming to emulate randomized trials. JAMA Netw Open. 2023;6(9):e2336023. doi:10.1001/jamanetworkopen.2023.36023
1School of Health Sciences, Faculty of Medicine &amp; Health, University of New South Wales, Sydney, Australia, a.cashin.neura.edu.au; 2Centre for Pain IMPACT, Neuroscience Research Australia, Sydney, Australia; 3CAUSALab, Harvard T.H. Chan School of Public Health, Boston, MA, US; 4Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, US; 5Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, US; 6Department of Epidemiology, University of Pittsburgh, Pittsburgh, PA, US; 7University of Exeter Medical School, Exeter, UK; 8IQVIA, London, UK; 9Richard A. and Susan F. Smith Center for Outcomes Research, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, US; 10Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland; 11Centre for Infectious Disease Epidemiology and Research, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa; 12Department of Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK; 13RTI Health Solutions, Barcelona, Spain; 14Northwestern University Feinberg School of Medicine, Chicago, IL, US; 15Oxford Population Health, Big Data Institute, University of Oxford, Oxford, UK; 16Faculty of Medicine, University of Southampton, Southampton, UK; 17Department of Biostatistics, Boston University School of Public Health, Boston, MA, US; 18Clinical Epidemiology &amp; Biostatistics Unit, Murdoch Children’s Research Institute, Royal Children’s Hospital, Parkville, Melbourne, VIC, Australia; 19Department of Paediatrics, The University of Melbourne, Parkville, Australia; 20School of Population Health, Faculty of Medicine &amp; Health, UNSW Sydney, Sydney, Australia; 21Division of Pharmacoepidemiology, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; 22Department of Public Health and Epidemiology, School of Population Health, RCSI University of Medicine and Health Sciences, Dublin, Ireland; 23NIHR Bristol Biomedical Research Centre, Bristol, UK; 24Health Data Research UK South-West, Bristol, UK; 25Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, US.
<h4>Conflict of Interest Disclosures </h4>
Miguel A. Hernán is an advisor to ProPublica and Adigens Health, in which he owns equity, and a member of ADIA Lab’s advisory board. Issa J. Dahabreh is a consultant to Moderna on work related to target trial emulation. Sebastian Schneeweiss is participating in investigator-initiated grants to Brigham and Women’s Hospital from Boehringer Ingelheim, Takeda, and UCB unrelated to the topic of this study; is an advisor to and owns equity in Aetion, a software manufacturer; and is an advisor to Temedica, a patient-oriented data generation company.
<h4>Funding/Support</h4>
 This work received no direct funding. The TARGET consensus meeting was supported through proceeds from a course on causal inference (December 4-7, 2023; Sydney, Australia). Aidan G. Cashin was supported by an Australian National Health and Medical Research Council (NHMRC) Investigator Grant (2010088). Harrison J. Hansford was supported by an Australian NHMRC Postgraduate Scholarship (2021950), a PhD Top-Up Scholarship from Neuroscience Research Australia, and was a Neuroscience Research Australia PhD Pearl sponsored by Sandra Salteri, AO. Miguel A. Hernán was funded by a National Institutes of Health (NIH) grant (R37 AI102634). Issa J. Dahabreh was supported by a Patient-Centered Outcomes Research Institute (PCORI) award (ME-2021C2-22365) and an NIH award (R01HL136708). Barbra A. Dickerman was supported by a grant from the NIH (R00 CA248335). Matthias Egger was supported by grants from the NIH (R01 AI152772-01, 5U01-AI069924-05) and the Swiss National Science Foundation (32FP30-174281). Nazrul Islam was supported by grants from the National Institute for Health and Care Research (NIHR; HDRUK2022.0313) and the UK Office for National Statistics (2002563). Margarita Moreno-Betancur was supported by an Australian NHMRC Investigator Grant (2009572). Sebastian Schneeweiss was supported by the NIH (R01-HL141505, R01-AR080194) and the US Food and Drug Administration (HHSF223201710146C). Jonathan A. C. Sterne was supported by the NIHR Bristol Biomedical Research Centre and by Health Data Research UK. Elizabeth A. Stuart was supported by grants from the NIH (R01 MH126856) and PCORI (ME-2020C3-21145). James H. McAuley was supported by an Australian NHMRC Investigator Grant (2010128).
<h4>Role of the Funder/Sponsor </h4>
The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation review, or approval of the abstract; and decision to submit the abstract for publication.
<h4>Additional Information </h4>
Aidan G. Cashin and Harrison J. Hansford are co–first authors.
<h4>Acknowledgment </h4>
We thank and acknowledge the contributions of all participants of the internal and external piloting who were not compensated for their contributions.

Transparent Reporting of Observational Studies Emulating a Target Trial: The TARGET Guideline

Mette B. Engmose,1,2 An-Wen Chan,3 Kerry Dwan,4 Carsten Hinrichsen,5 Asbjørn Hróbjartsson,1,2 David Moher,6 Matthew J. Page,7 Larissa Shamseer,8 Lesley A. Stewart,4 Camilla H. Nejstgaard1,2
<h4>Objective </h4>
Systematic review protocols often adhere incompletely to the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015.1 Reporting guidelines need to be regularly updated to incorporate methodological developments, author feedback, and changed research context. We aimed to inform the update of PRISMA-P 20152 by exploring protocol authors’ practices and experiences with the reporting guideline.
<h4>Design </h4>
In a cross-sectional study, we investigated adherence to PRISMA-P 2015 and interviewed protocol authors. We randomly sampled 100 systematic review protocols from May 2021 to May 2024: 50 protocols published in PubMed-indexed journals and 50 protocols uploaded to or registered in OSF/PROSPERO. Two authors independently assessed the 26 PRISMA-P 2015 items as fully, partially, or not reported (or not applicable). We analyzed protocol adherence overall and at the item level. We invited 43 corresponding authors from our sample of 100 systematic review protocols for interviews on their experiences with using PRISMA-P 2015. The selection of authors was based on maximum variation and snowball sampling and personal network. We used a piloted, semistructured interview guide involving 3 predefined themes: level of experience, views on using PRISMA-P 2015, and reflections on the guideline’s strengths and weaknesses. We applied framework analysis to the interview transcripts. The study was reported according to Standards for Reporting Qualitative Research (SRQR).3
<h4>Results </h4>
The PubMed-indexed protocols fully adhered to a median of 60% (range, 36%-80%) of PRISMA-P 2015 items. The equivalent median for OSF/PROSPERO protocols was 45% (range, 25%-88%). In both types of protocols, items not adhered to in more than 25% of the protocols were related to protocol amendments (item 4), role of the funder (item 5c), methods for planned summary other than quantitative (item 15d), meta-bias(es) (item 16), and confidence in the cumulative evidence (item 17) (Table 25-1033). In both types of protocols, 11 items partially adhered to in more than 25% of the protocols were lacking, eg, descriptions of procedures for data selection from multiple reports of the same study (item 11c). Of the 43 invited authors, 15 (9 men, 6 women) participated in the interviews. From the predefined themes, several suggestions for the PRISMA-P 2015 update emerged. Some suggestions regarded adding or modifying existing content, eg, to report conflicts of interest or how to report data synthesis when no meta-analysis was planned; other suggestions were more generic, eg, to add links to the Elaboration &amp; Explanation paper.

![](https://assets.underline.io/markdown_image/1/image/8a19657504ae87562517e7d38d9665ea.png)

<h4>Conclusions</h4>
 Adherence to PRISMA-P 2015 was inadequate in 50 systematic review protocols from PubMed-indexed journals and especially in 50 protocols from OSF/PROSPERO. Aspects often not adhered to were protocol amendments, role of the funder, methods for planned summary other than quantitative, meta-bias(es), and confidence in the cumulative evidence. The interviewed authors suggested several modifications of the guideline. Findings from this study will inform the update of PRISMA-P 2015.
<h4>References</h4>
1. Frost AD, Hróbjartsson A, Nejstgaard CH. Adherence to the PRISMA-P 2015 reporting guideline was inadequate in systematic review protocols. J Clin Epidemiol. 2022;150:179-187. doi:10.1016/j.jclinepi.2022.07.002
2. Nejstgaard CH, Shamseer L, Chan A-W, et al. Updating the PRISMA-P reporting guideline for systematic review protocols. OSF. April 10, 2024. <a href="https://osf.io/2znc5/">https://osf.io/2znc5/</a>
3. O’Brien BC, Harris IB, Beckman TJ, et al. Standards for reporting qualitative research: a synthesis of recommendations. Acid Med. 2014;89(9):1245-1251. doi:10.1097/acm.0000000000000388
1Cochrane Denmark &amp; Centre for Evidence-Based Medicine Odense (CEBMO), Department of Clinical Research, University of Southern Denmark, Odense, Denmark, mengmose@health.sdu.dk; 2Open Patient data Exploratory Network (OPEN), Odense University Hospital, Odense, Denmark; 3Department of Medicine, Women’s College Research Institute, University of Toronto, Toronto, Ontario, Canada; 4Centre for Reviews and Dissemination, University of York, York, UK; 5National Institute of Public Health, University of Southern Denmark, Odense, Denmark; 6Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; 7Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia; 8Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada.
<h4>Conflict of Interest Disclosures </h4>
An-Wen Chan, Kerry Dwan, Asbjørn Hróbjartsson, David Moher, Matthew J. Page, Larissa Shamseer, Lesley A. Stewart, and Camilla H. Nejstgaard are members of the steering group for the update of the PRISMA-P 2015 reporting guideline. All other authors declare no conflicts of interest related to financial interests, activities, relationships, and affiliations, including employment, affiliation, funding and grants received or pending, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. An-Wen Chan and David Moher are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract.
<h4>Funding/Support </h4>
This project did not receive any external funding; therefore, design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation were not influenced by any funder or sponsor.

Author Practices and experiences With PRISMA-P 2015

Cath Chamberlain,1 Peter Craig,2 Luis Gabriel Cuervo,3 Omar Dewidar,4 H. N. Ellingwood,5 Elizabeth Ghogomu,6 Billie-Jo Hardy,7 Tanya Horsley,8 Sonya Faber,9 Cindy Feng,10 Damian Francis,11 Sarah Funnell,12 Alison Krentel,9 Janet Jull,12 Elizabeth Kristjansson,13 Julian Little,9 Loveline Lum Niba,14 Tamara Kredo,15 Zack Marshall,16 Lawrence Mbuagbaw,17 Michael Johnson Mahande,18 Stuart Nicholls,19 Miriam Nkangu Nguilefem,9 Ekwaro Obuku,20 Oyekola Oloyede,21 Ebenezer Owusu-Addo,22 Kevin Pottie,23 Jacqueline Ramke,24 Alison Riddle,6 Anita Rizvi,13 Janet Hatcher Roberts,9 Larissa Shamseer,25 Melissa Sharp,26 Janice Tufte,27 Peter Tugwell,28 Xiaoqin Wang,29 Laura Weeks,30 Charles Wisonge,31 Luke Wolfenden,32 Taryn Young,33 Vivian Welch6
<h4>Objective</h4>
 Advancing health equity requires improvements in the conduct, analysis, and reporting of health research. Many observational studies are relevant to health equity because they use data that can advance our understanding of health inequities, including the systemic structures that perpetuate them. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for the reporting of observational studies is highly endorsed by journal editors; however, the consideration of health equity has not been explicitly addressed in the STROBE statement. As a result, editors have been limited in their ability to promote the reporting of health equity data and considerations. The STROBE-Equity extension provides actionable guidance for equity-relevant observational studies to fill this gap.
<h4>Design </h4>
The development of STROBE-Equity adhered to a published protocol, informed by the Enhancing the Quality and Transparency of Health Research (EQUATOR) Network’s methodological framework including a scoping review of existing guidance, methods study, key informant interviews, and a global online survey. The project team was diverse across multiple domains, including lived experience of health inequity, gender, career stage, age, ethnicity, Indigenous background, disciplines, place of residence, and different decision-making roles. We recruited team members from our international network of patients, researchers, editors, and practitioners who had prior involvement in observational studies or expertise in reporting guidelines, biostatistics, knowledge translation, and patient engagement. We applied an integrated knowledge translation approach to (1) assess the reporting of health equity in published observational studies through a scoping review of available guidance, methodological review of observational studies, and key informant interviews; (2) seek wide international input on draft items; and (3) establish consensus amongst knowledge users and researchers through a 2-day consensus meeting. The development of the reporting guideline occurred between January 2020 and February 2024, with consensus achieved in May 2024.
<h4>Results </h4>
Our process identified 10 extension items for the STROBE statement to offer guidance for reporting of health equity (Table 25-1136). The items involve the following: (1) facilitating the identification of health equity relevance, (2) describing the purpose of the study in relation to health equity, (3) describing team composition, (4) engagement with individuals experiencing inequities relevant to the condition of interest, (5) sampling and recruitment of relevant populations, (6) participant selection, (7) analytical methods, (8) describing participant flow for relevant populations, (9) describing the characteristics of study participants, and (10) interpretation of study findings in relation to health equity.
<a href="/wp-content/uploads/2025_conference/tables/prc25-1136t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1136t1.png"> </a>
<h4>Conclusions </h4>
The use of the STROBE-Equity extension alongside the original STROBE statement aims to promote transparent reporting of health equity data and considerations in observational research. Journal editors endorsing the use of this guideline can help advance health equity through improved reporting, ultimately contributing to research that informs more equitable policies and interventions.
1University of Melbourne, Australia; 2MRC/CSO Social and Public Health Sciences Unit, School of Health and Wellbeing, University of Glasgow, UK; 3Pan American Health Organization, Washington, DC, US; 4University of Toronto, Canada; 5Carleton University, Canada; 6Bruyère Health Research Institute, Canada; 7Dalla Lana School of Public Health, University of Toronto, Canada; 8Royal College of Physicians and Surgeons, Canada; 9School of Epidemiology and Public Health, University of Ottawa, Canada; 10Dalhousie University, Canada; 11PhD/School of Health and Human Performance, Center for Health and Social Issues, Georgia College, Milledgeville, GA, US; 12Queens University, Canada; 13School of Psychology, University of Ottawa, Canada; 14Department of Public Health, Faculty of Health Sciences, The University of Bamenda, Cameroon; 15South Africa Medical Research Council, South Africa; 16Department of Community Health Sciences, Cumming School of Medicine, University of Calgary and O’Brien Institute for Public Health, University of Calgary, Canada; 17McMaster University, Canada; 18Kilimanjaro Christian Medical Centre, Kenya; 19Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Canada; 20Africa Centre for Systematic Reviews and Knowledge Translation, Kampala, Uganda; 21Sefako Makgatho Health Sciences University, South Africa; 22Bureau of Integrated Rural Development, College of Agriculture and Natural Resources, Ghana; 23Dalhousie University, Canada; 24International Centre for Eye Health, London School of Hygiene &amp; Tropical Medicine, London, UK; 25Knowledge Translation Program, Li Ka Shing Knowledge Institute, Unity Health Toronto, Canada; 26Department of Public Health &amp; Epidemiology, School of Population Health, Ireland; 27Catalyst for Positive Change, US; 28Department of Medicine, Faculty of Medicine, University of Ottawa, Canada; 29University of Ottawa Heart Institute, Canada; 30Canada’s Drug Agency, Ottawa, Canada; 31South Africa Medical Research Council, South Africa; 32University of Newcastle, Australia; 33Stellenbosch University, South Africa; vwelch@campbellcollaboration.org. 
<h4>Conflict of Interest Disclosures</h4>
 Kevin Pottie, Lawrence Mbuagbaw, Vivian Welch, and Peter Tugwell declare they are co-convenors of the Cochrane Health Equity Thematic Group. No other disclosures were reported. 
<h4>Funding</h4>
 This work was funded by the Canadian Institutes of Health Research. The funder had no role in the project design, implementation, or decision to publish. Vivian Welch is funded by a Public Health Agency of Canada and Canadian Institute of Health Research Applied Public Health Chair award. Kevin Pottie is funded by a Dalhousie University Faculty of Medicine Chair in Family Medicine.
<h4>Acknowledgment</h4>
 The authors thank all the respondents to the global survey.

Development of a reporting guideline on health equity in observational research (STROBE-Equity)

Stephana Julia Moss,1 Juliane Kennett,2 Jeanna Parsons Leigh,3 Niklas Bobrovitz,4 Henry T. Stelfox5
<h4>Objective </h4>
To map the evidence for factors (eg, research practices) associated with the reproducibility of methods and results reported in health sciences research.
<h4>Design </h4>
Five bibliographic databases were searched from January 2000 to May 2023, followed by supplemental searches of high-impact journals and relevant records. We included health science records of observational, interventional, or knowledge synthesis studies reporting data on factors related to research reproducibility. Factors were operationalized as modifiable or nonmodifiable aspects of study conduct relating to individual-, study-, or institutional-level practices, methods, and processes that could impact the reproducibility of research methods or results.1 Reproducibility was operationalized by 2 mutually exclusive categories: (1) methodological reproducibility (ie, the ability to exactly repeat the methods, including study procedures and data analysis) and (2) results reproducibility (ie, obtaining corroborating results using the same or similar methods).2 We included studies that used surrogate measures for reproducibility (eg, type 1 or 2 error rates) if they (1) explicitly stated their aim to investigate the reproducibility of research and (2) rationalized their choice of surrogate measure.3 Data were coded using inductive qualitative content analysis, and empirical evidence was synthesized with evidence and gap maps. Study risk of bias was assessed using the Quality in Prognostic Studies risk-of-bias tool. Statistical tests of the association between factors and reproducibility outcomes were summarized as reported in the included articles.
<h4>Results </h4>
Our review included 148 primarily biomedical and preclinical (n = 62) and clinical (n = 71) studies. Factors were classified into 12 modifiable (eg, sample size and power) and 3 nonmodifiable (eg, publication year) categories. Of 234 reported evaluations of factors, 76 (32%) assessed methodological reproducibility and 158 (68%) assessed results reproducibility. The most frequently reported factor was transparency and reporting (38 of 234 assessments [16%]). A total of 155 factors (66%) were evaluated for statistical associations with reproducibility outcomes (Table 25-0858). Statistical associations were most frequently conducted for analytical methods (24 of 26 reporting significance [92%]), sample size and power (21 of 23 reporting significance [91%]), and participant characteristics and study materials (10 of 12 reporting significance [83%]). Risk-of-bias assessments found low risk of bias for study participation, factor measurement, and statistical analysis, and high risk of bias for confounding.

![](https://assets.underline.io/markdown_image/1/image/b16148b83c9aa6aaf32a839e2bc713d9.png)

<h4>Conclusions </h4>
Our review identified a large body of literature consisting primarily of observational studies of factors associated with the reproducibility of health sciences research. The data suggest that reproducibility may be improved by implementing more stringent statistical testing procedures and thresholds, sample size and power calculations, and improved transparency and completeness of reporting. Experimental studies are needed to test interventions to improve reproducibility. Factors identified in this study with consistent observational support should be prioritized for experimentation. Factors that affect reproducibility in health and social care services and population and public health need to be identified given the paucity of data in these areas.
<h4>References</h4>
1. Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341ps12-341ps12. doi:10.1126/scitranslmed.aaf5027
2. Niven DJ, McCormick TJ, Straus SE, et al. Reproducibility of clinical research in critical care: a scoping review. BMC Med. 2018;16:1-12. doi:10.1186/s12916-018-1018-6
3. Clemens MA. The meaning of failed replications: a review and proposal. J Econ Surveys. 2017;31(1):326-342. doi:10.1111/joes.12139
1Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada, sj.moss@dal.ca; 2Department of Critical Care Medicine, University of Calgary, Calgary, Alberta, Canada; 3Faculty of Health, Dalhousie University, Halifax, Nova Scotia, Canada; 4Department of Emergency Medicine, University of Calgary, Calgary, Alberta, Canada; 5Faculty of Medicine &amp; Dentistry, University of Alberta, Edmonton, Alberta, Canada.
<h4>Conflict of Interest Disclosures</h4>
 None reported.
<h4>Funding/Support</h4>
 This work was funded by the Canadian Institutes of Health Research.
<h4>Role of the Funder/Sponsor </h4>
The funding body had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; or decision to submit the abstract for presentation.

Factors Associated with the Reproducibility of Health Sciences Research: A Systematic Review and Evidence and Gap Map

Ayu Putu Madri Dewi,1 Nicholas J. DeVito,2 Gowri Gopalakrishna,1,3 Inge Stegeman,4,5 Mariska Leeflang1
<h4>Objective</h4>
 To improve research reproducibility, referees of grant proposals could assess the expected reproducibility of a proposed project. A simple Open Science checklist may assist referees in assessing whether referees are more capable of predicting the reproducibility of research proposals with an Open Science checklist than without one.1
<h4>Design</h4>
 This is a nonrandomized controlled trial and a comparative prediction study using a minimum of 100 granted research proposals and their corresponding published outputs (α = 5%, power = 80%, correlation = 0.6).2 The study focuses on empirical research with quantitative data across diverse fields. A mock grant review was started on April 7, 2025, with a minimum of 4 referees divided into 2 groups: one using an Open Science checklist and the other not. Each proposal will be assessed under both conditions (Figure 25-0934). The checklist, developed in a previous study3 and adapted for grant proposal evaluation, serves as the intervention. Referees are blinded to final study outcomes to minimize bias. The primary outcome is predictive accuracy, assessed using the area under the curve (AUC) from receiver operating characteristic (ROC) analysis. Referees’ predictions on the actual reproducibility of the published studies will be compared. Open Science scores are numerical (0-14), while reproducibility outcomes are binary (yes or no). The AUC from logistic regression will be used and visualized in ROC curves. Comparative predictive accuracy will be calculated for both groups. Sensitivity analyses will explore variations in checklist scoring and study reproducibility across different research fields. A database including grant proposal data, Open Science scores, reproducibility status, and analytical code in R will be shared in an open repository. We hypothesize that referees without the checklist will predict reproducibility no better than random chance (with an AUC of approximately 0.50), while those using the checklist will achieve an AUC of approximately 0.65, indicating improvement while acknowledging that the checklist is not a perfect tool.

![](https://assets.underline.io/markdown_image/1/image/17200ac9fe2e7dc26b84dba55df1facc.png)

<h4>Results</h4>
 As of June 5, 2025, a total of 15 mock grant reviewers have completed 89 mock reviews across 60 unique grant proposals: 46 reviews in the checklist group and 43 in the control group. Twenty-nine proposals were reviewed in both groups, forming a fully paired subset. Analysis is ongoing.
<h4>Conclusions</h4>
 We will share our results with funding agencies to explore integrating the Open Science checklist into grant review processes to enhance reproducibility.
<h4>References</h4>
1. Parker TH, Griffith SC, Bronstein JL, et al. Empowering peer reviewers with a checklist to improve transparency. Nat Ecol Evol. 2018;2(6):929-935. doi:10.1038/s41559-018-0545-z
2. Fraser H, Bush M, Wintle BC, et al. Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process. PLoS One. 2023;18(1):e0274429. doi:10.1371/journal.pone.0274429
3. Dewi APM, Rethlefsen M, Schroter S, et al. Measuring the efficacy of an intervention to improve reproducibility of scientific manuscripts: study protocol for a randomized controlled trial. OSFHOME. October 31, 2024. Accessed July 2, 2025. <a href="https://osf.io/b5g6y">https://osf.io/b5g6y</a>
1Amsterdam UMC, Epidemiology and Data Science Department, Amsterdam, The Netherlands; 2Bennett Institute for Applied Data Science, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK, nicholas.devito@phc.ox.ac.uk; 3Department of Epidemiology, Faculty of Health, Medicine, and Life Sciences Maastricht University, The Netherlands; 4Department of Otorhinolaryngology and Head &amp; Neck Surgery University Medical Center Utrecht, The Netherlands; 5Brain Center, University Medical Center Utrecht, The Netherlands.
<h4>Conflict of Interest Disclosures </h4>
None reported.

Next from The 10th Peer Review Congress

Replication and Impact of Positive Secondary Findings in Negative or Neutral Cardiovascular Trials

Similar lecture

Transparent Reporting of Observational Studies Emulating a Target Trial: The TARGET Guideline