United States

Background: Although ulnar collateral ligament reconstruction (UCLR) is commonly performed in Major League Baseball (MLB) pitchers, few studies have evaluated advanced analytical data in the context of return to play (RTP) after injury. Traditional postoperative outcomes have not reliably predicted players return to their preoperative performance level. Pitch movements and release extension, defined as the raw horizontal and/or vertical movement of a pitch, and distance off the pitcher’s mound a pitcher releases the pitch are all key elements professional pitchers rely upon for effectiveness. Existing literature has demonstrated a varying degree of impact of some objective advanced measurements, such as spin rate and pitch movements, using a pitching tracking system that has been recently implemented and used for analytical data collection in professional baseball. The present study is one of the first to explore the impact of UCL reconstruction on horizontal and vertical pitch movement as well as pitch release extension. The study is also the first comparison of those advanced measurements between MLB pitchers who returned with sustained performance and those who were unable to return to preinjury performance levels for sustained time.
Methods: 113 MLB pitchers who underwent UCLR between 2016 and 2022 were identified via public record. The identified cohort was separated into those who returned with sustained levels of play, defined as throwing &gt;500 pitches per season for multiple seasons following UCLR, and those who did not. The Statcast system was utilized to collect horizontal movement, vertical movement, and release extension for 4FB, 2FB, CU, and SL in the preinjury year as well as in each of the 2 years after following UCLR.
Results: 70 pitchers (62%) returned to the MLB level after UCLR. There was significant change in vertical movement for 4FB preoperatively compared to 1-year postoperatively (p&lt;0.05). Pitchers who returned to the MLB with sustained level of performance demonstrated a statistical increase (p&lt;0.05) in release extension for all pitch types in the first postoperative season when compared to those who returned without longevity, but the observed changes in release extension was only maintained in 4FB in the second postoperative season.
Conclusion: MLB pitchers who underwent UCLR and returned to the MLB level demonstrated a significant change in pitch release extension with all pitch types two years after surgery. The MLB pitchers who returned to the MLB level after UCLR and achieved longevity averaged a greater release extension postoperatively than those who did not.

AMA Research Challenge 2024 

Major League Baseball Pitchers with Greater Release Extensions After Ulnar Collateral Ligament Reconstruction Have Better Career Longevity.

Welcome to the **largest national, multi-specialty research** event featuring research from more than 750 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 

[![](https://assets.underline.io/markdown_image/1/image/20e1e77aa954efeac7a15d84c85eb383.jpg)](https://underline.io/events/468/sessions?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/068522d81aca0b68a369787c7dfccd1b.jpg)](https://underline.io/events/468/posters?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/d3971ece79d5e5575b178bdc2c8bbfae.jpg)](https://underline.io/events/468/help-desk)
 

#### Event Details

The 2024 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize presented by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in winner selection process. The winner will be announced on February 20, 2025 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

[![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)](https://www.laurelroad.com/)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

**There is no registration fee for this event! **

If you have used the AMA Research Challenge platform previously, please click the ‘Log-in’ button below. If you do not remember your password, please click “forgot password” and follow prompts to reset your password. After resetting your password, please refresh your webpage to access the platform.

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals. 

**Please use this link [here](https://www.ama-assn.org/member-benefits/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters and score posters in the semifinals (an AMA member exclusive opportunity).

Background: Although ulnar collateral ligament reconstruction (UCLR) is commonly performed in Major League Baseball (MLB) pitchers, few studies have evaluated advanced analytical data in the context of return to play (RTP) after injury. Traditional postoperative outcomes have not reliably predicted players return to their preoperative performance level. Pitch movements and release extension, defined as the raw horizontal and/or vertical movement of a pitch, and distance off the pitcher’s mound a pitcher releases the pitch are all key elements professional pitchers rely upon for effectiveness. Existing literature has demonstrated a varying degree of impact of some objective advanced measurements, such as spin rate and pitch movements, using a pitching tracking system that has been recently implemented and used for analytical data collection in professional baseball. The present study is one of the first to explore the impact of UCL reconstruction on horizontal and vertical pitch movement as well as pitch release extension. The study is also the first comparison of those advanced measurements between MLB pitchers who returned with sustained performance and those who were unable to return to preinjury performance levels for sustained time.
Methods: 113 MLB pitchers who underwent UCLR between 2016 and 2022 were identified via public record. The identified cohort was separated into those who returned with sustained levels of play, defined as throwing >500 pitches per season for multiple seasons following UCLR, and those who did not. The Statcast system was utilized to collect horizontal movement, vertical movement, and release extension for 4FB, 2FB, CU, and SL in the preinjury year as well as in each of the 2 years after following UCLR.
Results: 70 pitchers (62%) returned to the MLB level after UCLR. There was significant change in vertical movement for 4FB preoperatively compared to 1-year postoperatively (p<0.05). Pitchers who returned to the MLB with sustained level of performance demonstrated a statistical increase (p<0.05) in release extension for all pitch types in the first postoperative season when compared to those who returned without longevity, but the observed changes in release extension was only maintained in 4FB in the second postoperative season.
Conclusion: MLB pitchers who underwent UCLR and returned to the MLB level demonstrated a significant change in pitch release extension with all pitch types two years after surgery. The MLB pitchers who returned to the MLB level after UCLR and achieved longevity averaged a greater release extension postoperatively than those who did not.

poster

Abstract Title
Education, Language, and Cultural Concordance Influence Patient-Physician Communication in Orthopaedics

Background

Disparities in healthcare outcomes are pronounced among racial and ethnic minorities in the United States, particularly in orthopaedics where only 4.7% of faculty are from underrepresented minority (URiM) groups. Previous research highlights that cultural competency and humility in healthcare can reduce these disparities and improve patient satisfaction. However, there is limited data on the impact of patient-physician cultural and language concordance in orthopaedic care.

Methods

A convenience sample of patients and orthopaedic surgeons from an urban tertiary medical center was surveyed between August 2021 and May 2023. A self-administered questionnaire, available in English and Spanish, assessed demographic information, language proficiency, treatment satisfaction, and the importance of cultural and language understanding by clinicians. The cohort included 394 patients, with data analyzed using descriptive statistics, ANOVA, Pearson’s correlation, chi-square tests, and logistic regression.

Results

A total of 394 patients completed the questionnaire, with 35% male and 65% female, and an average age of 54 years. Hispanic patients reported lower English proficiency compared to Caucasian and African-American subjects (p<0.001), and educational attainment significantly influenced patients' ability to communicate with their orthopaedic surgeon (p<0.001). Hispanic and African-American patients placed higher importance on cultural understanding by their orthopaedic surgeons compared to Caucasian patients (p<0.001). Among Hispanic patients, those treated by culturally and linguistically concordant surgeons reported better communication and higher satisfaction levels. Logistic regression showed that Hispanic patients who saw culture and language concordant surgeons valued Spanish-speaking surgeons more (66% vs 45%, p=0.04). 

Conclusion

This study highlights the critical role of educational attainment and patient-physician cultural and language concordance in effective communication within orthopaedic care. The novel finding that Hispanic patients show significantly improved communication and satisfaction when treated by culturally and linguistically concordant surgeons underscores the need for increasing the number of culturally and linguistically proficient orthopaedic surgeons. Addressing these factors can enhance patient outcomes and satisfaction, particularly among Hispanic and African-American populations. Efforts should focus on improving health literacy and cultural humility training for orthopaedic surgeons to bridge the communication gap and reduce disparities in orthopaedic care.

Education, Language, and Cultural Concordance Influence Patient-Physician Communication in Orthopaedics

Importance: The United States is experiencing a public health crisis due to an exponential rise in syphilis incidence, which can cause multi-systemic manifestations, including ocular syphilis. The standard of care treatment for ocular syphilis, a neuro-syphilis equivalent, has been intravenous (IV) aqueous crystalline penicillin G 18-24 million units per day for 10-14 days, however, this is costly to both the healthcare system and patients. 
Objective: To assess the efficacy of oral doxycycline to treat ocular syphilis when compared to standard of care IV penicillin.
Design: Retrospective cohort study. 
Setting: Los Angeles General Medical Center, a large, urban academic medical center caring for underserved patients.
Participants: 32 patients with ocular syphilis. 
Interventions: Patients received either doxycycline 200mg twice daily for 28 days as an outpatient or IV penicillin for 10-14 days through shared decision making.
Main Outcomes: The main outcome measure was clinical resolution of ocular inflammation. Secondary outcome measures included visual acuity and serologic resolution (four-fold decrease in RPR titers). 
Results: Sixteen patients received oral doxycycline and sixteen patients received IV penicillin. There were no differences in age, sex, race, HIV infection, CD4 count, or treponemal test positivity between the cohorts. There was no difference in the rates of ocular inflammation resolution among patients who were treated with doxycycline or penicillin (p=0.62). Median visual acuity was better among patients receiving oral doxycycline than IV penicillin at presentation and final follow-up. Visual acuity change between presentation and final visit were similar between groups. Among patients with 9 months of follow-up, 4 out of 4 patients receiving doxycycline and 7/7 patients received IV penicillin achieved a four-fold crease in RPR titers. 
Conclusions: Oral doxycycline for the treatment of ocular syphilis may be safe and effective in a carefully selected subset of patients. An outpatient oral regimen has many benefits, most crucially in its avoidance of inpatient admissions, which are costly to the healthcare system and hazardous to the patient. Further prospective studies are required to evaluate the clinical and serologic effects of treating ocular syphilis with oral doxycycline.

Outpatient Oral Doxycycline Therapy for Ocular Syphilis

Background

Electronic health record (EHR) notes are critical to medical decision making but have become unwieldy to manually sift through due to their many aims: communication, compliance, and billing. Consequently, clinicians spend more time navigating EHRs than with patients, leading to burnout. 

Prior analysis focuses on static completed notes, and few have explored granular reading and writing activities. Clinicians describe time-varying purposes of EHR use: rapid sense-making of new patients and re-familiarization with existing patients. We introduce a framework to dynamically retrieve relevant notes at different points during a patient’s visit to streamline discovery of relevant history and satisfy changing information needs.

Methods

We analyze 48,192 note-writing sessions in the high-acuity emergency department (ED). We design a binary classification task of whether to retrieve a past note during the next session given the current writing context. Audit logs, which track reading and writing activity in the EHR, provide labels for machine learning. We encode patient and clinician information, note metadata, and a bag of words representation for note text. We train logistic regression models to predict the top ten most relevant notes for each session. 

For quantitative evaluation, we measure precision@10, recall@10, and AUC. Additionally, two physicians conducted chart review for five random patients. Each clinician was given the patient's age, sex, and chief complaint, and assigned relevance for past notes. We compared their relevance judgments with the predicted relevant notes.

Results

Our learned models’ quantitative performance surpassed that of deterministic baselines and achieved an AUC of 0.963. For the qualitative evaluation, for four out of five visits, the top ranked predicted note surfaced during the first session of the patient’s visit was determined to be directly helpful to the patient’s current presentation by both clinicians. For three of four visits, showing only the top ranked predicted note was sufficient information that both clinicians needed to proceed with patient care.

Conclusion

We present a novel framework that updates retrieval suggestions dynamically as clinical context changes, thus helping clinicians find relevant information more quickly at different points during the patient’s visit. It also surfaces difficult-to-find information and suggests patterns a clinician may not have initially deemed relevant, prompting them to reassess a diagnosis they initially anchored on and evaluate potential biases. Demonstrating that our methods perform well in the demanding ED setting is a promising proof of concept that can translate to other clinical settings and data modalities.

Conceptualizing Machine Learning for Dynamic Information Retrieval of EHR Notes

Abstract Title: 
Sex Disparities in a Telestroke Network: From Treatment Times to Outcomes

Background: The structure of the stroke care delivery system in rural South Carolina has been changed by the Medical University of South Carolina tele-stroke program with >40 spoke sites with improved treatment times and outcomes overall but disparities at the level of primary stroke centers and acute stroke ready hospitals have not been well studied. The aim of this study is to retrospectively analyze data from the tele-stroke network for sex differences in stroke care processes at spoke sites to identify disparities in treatment times and outcomes between men and women. 

Methods: Data were collected and analyzed for 21,549 patients between 1/1/2012 and 12/31/2022 for whom a telestroke was paged out. Statistical analysis was completed using SPSS version 28 (IBM corporation, Armonk, NY). Variables with continuous outcomes were assessed using independent samples median tests and categorical variables were assessed using X².

Results: Of 10,713 (49.7%) males and 10,836 (50.3%) females, females were older (68.29±15.5 vs 66.09±13.7, p<0.01) and more likely to be African American (20.4% vs16.8%, p<0.05) as compared to males. Females were more likely to get thrombolytic therapy (29.06% vs 27.2%, p<0.05) but there was no difference in endovascular therapy between both sexes. Treatment times showed that door-to-registration timing (F: 18±27 vs M: 17±26, p<0.05) and door-to-telestroke page timing (F: 22±27 vs M: 20±26, p<0.05) was significantly higher in females compared to males with ultimately door to needle time having the expected disparity between men and women (F: 59±36 vs M: 55±35, p<0.05). Women also had significantly higher mean discharge NIHSS (M: 4.17±6.20 vs F: 4.61±6.70 (p<0.001) and mRS score (M:2.24±1.93 vs F:2.43±1.95 (p<0.001)). More males were discharged home (62.4% vs 60.4%, (p<0.001), and more females were discharged to nursing homes (8.1% vs 6.2%, p<0.001) and hospice (3.9% vs 2.9%, p<0.001) post admission.

Conclusion: Females are older at stroke onset, have longer treatment times, and have poorer outcomes and discharge dispositions. Our findings highlight the opportunities to ameliorate sex disparities in developing telestroke networks in the United States. Future studies should target these disparities to improve outcomes.

Sex Differences in a Telestroke Network: Form Risk Factors to Outcomes

Introduction: Solid-organ transplant recipients (SOTRs) have a higher burden of gynecological cancers as compared to the general population and are likely to develop high-risk oncogenic HPV types due to immunosuppressive therapy. The objective of this study is to evaluate HPV vaccine awareness and uptake among SOTRs, and current attitudes toward the vaccine amongst SOTR care providers.

Methods: This cross-sectional prospective study surveyed SOTR patients and their medical providers on HPV awareness and vaccination (IRB ID: APP-23-00356) at the USC Dermatology clinic and Keck Hospital between April 2023 to May 2024. 

Results: Forty percent (36/91) of patients completed the survey, of whom 18 (50%) knew that HPV is preventable with vaccination and twenty-two (61.1%) participants believed they were at an increased risk for HPV infection. Four (11.1%) participants said their providers recommended the HPV vaccine and four patients (11.1%) reported having received the vaccine. 

Fifty-nine percent of providers (17/ 29) completed the survey, of whom seven (41.2%) providers recommend the HPV vaccine for their SOTR patients. Five (29.4%) providers discuss HPV vaccination status with their SOTR patients, and two (11.8%) stated that they offer the vaccine in their clinic.

Conclusions: Despite a majority of patients believing themselves to be at increased risk for HPV infection, only a minority of patients reported receiving HPV vaccine. Correspondingly, few SOTR care providers offered the HPV vaccine to patients. Our survey highlights the need to increase HPV vaccine access and improve provider guidance for HPV vaccination in SOTRs.


HPV Awareness and Vaccination Among Solid Organ Transplant Patients and Providers

This research investigates the role of time since trauma (TST) in refining trauma team activation (TTA) criteria at a level I trauma center. A retrospective analysis of 1,261 patients presenting after motor vehicle accidents from 2016 to 2021 was conducted, focusing on the relationship between TST and post-emergency department (ED) disposition. The study found that TTA rates decreased with increasing TST, yet patients with a TST of two or more hours showed higher rates of admission, ICU utilization, and surgical interventions. These findings challenge the assumption that TTA need diminishes over time and suggest that current criteria may benefit from refinement to account for longer TST.

Time Delay in Motor Vehicle Accident Arrival Time: A Critical Analysis of Trauma Team Activation

Abstract Title
A Survey on Disabilities and Disability Awareness in General Surgery Residents in the US

Background
Disabilities pose challenges for medical professionals, including surgical residents, impacting their training, performance, and overall well-being. Despite national statistics highlighting the high prevalence of disabilities among adults in the US, there remains a notable gap in understanding how these conditions impact surgical residency programs, particularly regarding the disclosure of disabilities and the utilization of accommodations. This study compared disability prevalence among general surgery residents with national benchmarks, explored disclosure barriers, and assessed disability policies in residency programs.

Methods
This study utilized a descriptive cross-sectional survey conducted among 198 general surgery residents across 323 programs located in urban, suburban, and rural settings. Participants completed a detailed 33-item questionnaire that encompassed demographic data, disability status, practices of disclosure, usage of accommodations, and awareness levels regarding disability policies and training within their residency programs. The survey was administered anonymously via email over two months, from February to April 2024.

Results
Among the respondents, 10.6% reported having physical disabilities, while 20.2% reported mental disabilities; the majority, 68.7%, reported no disabilities. Notably, 62.9% of residents with disabilities had not disclosed their status to program directors, primarily citing fears of discrimination and concerns over the burden of proof. Residents expressed significant apprehension regarding the transparency of accommodation requests and potential biases in evaluations. Moreover, a substantial portion (53.3%) reported receiving no prior training on interacting with individuals with disabilities, and 75.3% were unaware of their residency program's specific disability policies. These findings underscore critical deficiencies in disability
awareness, policy communication, and support mechanisms within surgical residency programs across the United States.

Conclusion
This study highlights pervasive gaps in disability disclosure and support within US surgical residency programs, emphasizing the urgent need for reform. Despite about 1 in 3 surgical residents reporting a disability, disability support seems lacking, impacting the health and wellbeing of residents. Recommendations include integrating comprehensive disability-related training into residency curricula, enhancing transparency of disability policies during the application process, and establishing clear protocols for residents to request accommodations. Addressing these challenges is crucial not only for promoting inclusivity and reducing stigma but also for enhancing professional success and overall well-being of surgical residents with disabilities. Such improvements have the potential to elevate standards of patient care and enhance surgical outcomes. Future research should aim to expand the scope of investigation to include a larger and more diverse sample, as well as explore intersectional factors influencing the accommodation needs of residents with disabilities.

A Survey on Disabilities and Disability Awareness in General Surgery Residents in the US

Background
Antisocial Personality Disorder (ASPD) is one of the most stigmatized psychiatric diagnoses in the public sphere, contrasting with Major Depressive Disorder (MDD) and Generalized Anxiety Disorder (GAD), which are among the least stigmatized. ASPD stigma extends to healthcare, where many providers hold negative attitudes towards patients with ASPD, affecting their willingness to work with this population. Understanding whether medicine perpetuates this stigma is crucial for early intervention in medical training. Given the role of language in perpetuating stigma, as evident in studies on Opioid Use Disorder, this study explored how language contributes to negative attitudes about ASPD in healthcare, an area largely overlooked in stigma research, Psychiatry, and medical education. It aimed to assess medical education resources for stigmatizing language and compare how they reflect public stigma patterns, where ASPD is more stigmatized than MDD and GAD.

Methods
Seven widely used medical education resources were analyzed for their depiction of ASPD, MDD, and GAD. Since many medical students favor third-party resources for studying and board exams, the analysis included Boards and Beyond, AMBOSS, First Aid, First Aid for Psychiatry Clerkship, and APA, as well as in-house materials from Warren Alpert Medical School and the DSM-5. The analysis focused on two stigma components identified by Phelan and Link: labeling and status loss, particularly through clinical pessimism. Fisher's Exact Test compared ASPD to MDD and GAD across labeling, treatment difficulty, and treatment omission.

Results
The analysis revealed significant stigma toward ASPD, with 85.7% (n=6) of the materials using labels like "bad," "manipulative," and "deceitful." Notably, "bad" was used as a mnemonic. In contrast, none of the materials for MDD or GAD employed such labels (p = 0.046). Regarding status loss, 42.9% (n=3) of ASPD materials emphasized treatment difficulty, compared to none for MDD or GAD (p = 0.031). Additionally, 71.4% (n=5) of ASPD resources omitted information on available treatments, while 42.9% (n=3) did so for MDD and GAD. Although the difference in treatment omission was not statistically significant (p = 1.0), the trend still reflects a bias.

Conclusion
The findings suggest that medical education promotes ASPD stigma, mirroring the disproportionate stigma seen in the general public. The use of stigmatizing language indicates that medical education may perpetuate negative attitudes among healthcare providers. Addressing this stigma is crucial to improving care and outcomes for individuals with ASPD. Further research is needed to explore causality and develop strategies to reduce stigma in medical training.

Stigma in Psychiatry Education: A Comparative Analysis of ASPD, MDD, and GAD

ChatGPT and other large language models (LLM) present a promising avenue for enhancing medical education due to its advanced text comprehension and generation capabilities. Motivated by the skyrocketing use of ChatGPT and the lack of curricular mention or discussion on how to best utilize LLMs in medical school, this study investigates the integration of ChatGPT into medical education through a single-session workshop.
Methods
Fifty-nine students from the Emory School of Medicine, 80% first- and second-year medical students and 20% physician assistant students, voluntarily participated in a ChatGPT workshop—a one hour session with learning objectives to understand the framework and construction of a LLM (ChatGPT) and develop comfort in using ChatGPT within medical contexts by generating relevant responses to medical queries in simulated scenarios. Thirty minutes was allocated toward a lecture style presentation and another thirty minutes to hands-on use of ChatGPT. Surveys administered before, immediately after, and eight weeks post-workshop assessed participants' comfort with and likelihood of using ChatGPT, as well as their perceptions of its utility in their future medical careers, ranked on a Likert scale.
Results
ChatGPT usage rose from “never used” and “rare” usage for 76% of participants to usage at least one time per week for 82% of participants at 8 weeks post-workshop, with 23% using ChatGPT four or more days per week. Survey results revealed a significant increase in participants' comfort with using ChatGPT immediately after the workshop, with this improvement persisting at the eight-week follow-up (p<0.05). Moreover, participants reported a significantly higher likelihood of utilizing ChatGPT in their future medical practice after attending the workshop (p<0.05) with the greatest proportion of individuals, 64%, using ChatGPT for explanations of pathophysiology.
Conclusion
This workshop highlights the effectiveness and need of education and training in fostering understanding and utilization of AI technologies like ChatGPT. Despite initial hesitancy, participants demonstrated increased comfort and willingness to incorporate ChatGPT into their medical education and practice. The study underscores the potential of integrating ChatGPT into medical curricula to enhance learning experiences, promote interdisciplinary collaboration, and foster critical thinking skills among students. Additionally, it emphasizes the need for further research to explore the long-term impact and ethical considerations of integrating AI technologies into medical education. Overall, integrating ChatGPT into medical education holds significant implications for the future of healthcare training and practice.

Integrating ChatGPT into Medical Education: A Workshop-Based Approach

Background
Clinical simulations have an important role in medical education, allowing students to gain realistic clinical experience in a didactic manner to improve patient interaction and clinical reasoning skills. However, access to simulations is often limited by resources including physical space in simulation centers and time with standardized patients. ChatGPT is a large language model (LLM) artificial intelligence software that is freely accessible online, that has garnered significant attention in the medical community for its ability to generate relatively accurate medical information. We evaluated the ability of ChatGPT to create high-fidelity virtual clinical simulations and provide simulated patient information and feedback across several clinical scenarios.

Methods
We prompted ChatGPT to generate seven clinical scenarios (fractured arm/intimate partner violence, acute appendicitis, aortic dissection, acetaminophen overdose, acute seizure/metastatic lung cancer, COPD exacerbation, and tension pneumothorax) and create interactive simulations including simulated patient history, physical exam discoveries, lab findings, and imaging descriptions. Each scenario was concluded with feedback and repeated in triplicate to assess consistency. Simulation scenarios were graded for fidelity, evaluated by accuracy, realism, and consistency on a 5-point Likert scale (1-2 = poor, 2-3 = low, 3-4 = moderate, 4-5 = high) by three independent physicians holding the academic rank of Full Professor. A one-sample Wilcoxon signed-rank test was used to assess simulation performance based on a threshold criteria of greater than three out of five points. Fleiss’ Kappa was used to assess interrater reliability. All hypothesis testing was performed using a significance level of p<0.05.

Results
Of the seven simulations evaluated, four were rated high fidelity, two were rated borderline high fidelity, and one was rated moderate fidelity regarding simulation accuracy, realism, and consistency. The mean overall score was 4.0, indicating overall high accuracy, realism, and consistency, significantly higher than hypothesized threshold of 3.0 (p < 0.001). There was a high degree of interrater agreement (ordinal weighted Fleiss κ, 0.763; 95%CI: 0.25-1.0, p<0.05).

Conclusion
Overall, ChatGPT was able to consistently generate accurate and consistent virtual clinical simulations. Our study highlights a framework for a novel use case of LLMs in medical education and demonstrates its utility in providing increased accessibility to simulations for medical students. Although LLMs have known limitations in information accuracy, with the rapid advancement of artificial intelligence technology and advent LLMs that can be trained on specific resources, the use of LLMs such as ChatGPT for further simulation-based medical education represents a promising frontier for medical education.

Premium content

Downloads

Next from AMA Research Challenge 2024

Education, Language, and Cultural Concordance Influence Patient-Physician Communication in Orthopaedics

Similar lecture

Factors predictive of season-ending upper-extremity injuries in MLB pitchers and the implementation of the pitch clock