United States

Background
Implicit bias, or unconscious attitudes towards minoritized groups, among healthcare providers can contribute to disparities in care, health status, and mortality for minoritized patients. One approach to mitigating the negative effects of these biases on patient care is by implementing anti-racism/anti-bias training into medical education. 

Methods
Rising third year medical students participated in a student-led implicit bias and microaggression training program, including experiential learning through standardized patient (SP) encounters and self-reflection via student-led debrief sessions. The SP encounters simulated instances of xenophobia and perceived language barriers in a hospital setting, in which students were expected to address microaggressions in real time utilizing the VITALS (Validate, Inquire, Take time, Assume the best, Leave opportunities, Speak up for others) framework. Participants completed surveys on attitudes regarding implicit bias prior to participating in the program (T1), after the VITALS video presentation (T2), and after the SP encounter and debrief sessions (T3); surveys were adapted from the VITALS study. 

Results
200 students participated in the program of which 135 linked survey sets were collected for all three time points for data analysis. Results showed that from T1 to T3, there were significant improvements in empathy with targets of bias, comfort in sharing experiences and knowledge and skills to counteract microaggressions. There were no differences by race/ethnicity, language exposure, or religious adherence.

Conclusion
Our implicit bias training, equipped students with the tools and practice needed to interrupt microaggressions in the clinical setting and helped improve their understanding and likelihood of engaging in anti-racism allyship.

AMA Research Challenge 2024 

Efficacy of Standardized Patient Encounters to Reinforce Implicit Bias Training in the Clinical Setting

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

EMNLP 2022

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

poster

Welcome to the **largest national, multi-specialty research** event featuring research from more than 750 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 

[![](https://assets.underline.io/markdown_image/1/image/20e1e77aa954efeac7a15d84c85eb383.jpg)](https://underline.io/events/468/sessions?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/068522d81aca0b68a369787c7dfccd1b.jpg)](https://underline.io/events/468/posters?searchGroup=all)
 
[![](https://assets.underline.io/markdown_image/1/image/d3971ece79d5e5575b178bdc2c8bbfae.jpg)](https://underline.io/events/468/help-desk)
 

#### Event Details

The 2024 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize presented by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in winner selection process. The winner will be announced on February 20, 2025 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

[![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)](https://www.laurelroad.com/)

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

**There is no registration fee for this event! **

If you have used the AMA Research Challenge platform previously, please click the ‘Log-in’ button below. If you do not remember your password, please click “forgot password” and follow prompts to reset your password. After resetting your password, please refresh your webpage to access the platform.

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals. 

**Please use this link [here](https://www.ama-assn.org/member-benefits/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters and score posters in the semifinals (an AMA member exclusive opportunity).

Background
In recent years, active learning methods have gained attention for improving material retention and teamwork, especially in education. Despite such benefits, learner engagement remains a challenge, prompting exploration into innovative solutions like game-based instruction. By using game principles, educators can sustain learner interest and encourage participation. Building on these principles, the Texas Health Presbyterian Dallas Internal Medicine Residency program implemented a “House Cup” system where points were awarded for educational activities to promote friendly competition and active learning.

Methods
Residents were evenly distributed into six teams. Points were earned based on criteria including: number of admissions on call days, attendance at lectures, Medical Knowledge Self Assessment Program (MKSAP) review, and the number of procedures performed. Data from January through March of 2022, 2023, and 2024 was collected. Surveys were utilized to gather information on the time spent on MKSAP preparation before and after implementation. Qualitative variables were analyzed with chi-square tests; quantitative variables with ANOVA or Kruskal-Wallis. Statistical significance was determined at an alpha level of 0.05.

Results
Prior to the “House Cup” system (pre-group), there was an average of 5.30 admissions (± 1.15). Following implementation (post-group), admissions rose to an average of 7.87 (± 1.68). Lecture attendance remained stable, with a median of 83 (IQR: 70.75-93) in the pre-group and 85 (IQR: 77-93) in the post-group. MKSAP questions completion and review time in the post-group (mode 51+ questions, 4-5 hours) increased compared to the pre-group (mode 0 questions, 0-1 hours). Number of procedures performed in 2024 (54) surpassed both 2022 (7) and 2023 (16).

Conclusion
Our aim was to assess the impact of game-based learning techniques on material retention, engagement, and teamwork within resident education. Integrating game-based learning into educational activities led to increased participation in call day admissions, lecture attendance, MKSAP review, and number of procedures performed. Implementing game-based learning resulted in improved performance across all metrics except lecture attendance, which was already high. These promising results benefit our residency program and offer insights for others seeking to incentivize and motivate residents


Level Up: Exploring the Effectiveness of Game Based Learning in Graduate Medical Education

Abstract Title: Association Between Utilization of Gender-Affirming Care and Insurance Payer, State, and State Policy

Background: Gender-affirming surgery/hormone therapy (GAS/GAHT) play an essential role in improving quality of life for many transgender and gender-diverse (TGD) individuals. Despite increased demand for GAS/GAHT, TGD individuals face many obstacles to care. We investigated the association between utilization rates of GAS/GAHT and insurance payer, state, and state-level nondiscrimination policies, which prohibit health plans from precluding care based on gender identity or expression. With the disparate legislative climate, it is imperative for stakeholders to understand how these factors influence national utilization rates of GAS/GAHT.

Methods: We retrospectively analyzed a cohort of TGD individuals of all ages to characterize GAS/GAHT utilization between 2010-2020 using the PearlDiver-Mariner Database, which contains all-payer claims for 151 million patients in all 50 states and DC. We used binary logistic regressions to evaluate associations between insurance payers (private, Medicaid, Medicare) and GAS/GAHT, while testing for effect modification by individual states. Furthermore, we designed a difference-in-differences analysis to compare GAS/GAHT utilization among those who did and did not live in states that passed policies prohibiting healthcare access discrimination by insurance payers.

Results: 77,366 individuals were included in the study. The distribution of insurance payers was as follows: 77% private, 16% Medicaid, 7% Medicare. GAS/GAHT utilization was highest among the privately insured and lowest among the Medicare subgroups. Individuals covered by Medicaid (OR=0.78,95% CI=0.73-0.83) or Medicare (OR=0.69,95% CI=0.63-0.75) were less likely to undergo GAS compared to the privately insured. Similar differences in GAHT utilization were observed. Effect modification by state was evident in both outcomes. For example, differences in GAHT utilization rates by insurance payers were smaller among those living in Illinois (38% private, 37% Medicaid, 33% Medicare) compared to Idaho (35% private, 15% Medicaid, 23% Medicare). Quasi-experimental analysis suggested that nondiscrimination policies were associated with increased national all-payer treatment utilization, especially GAS, which more than doubled after implementation.

Conclusion: Our study highlights three main findings: 1) Across all states, individuals covered by Medicaid/Medicare received GAS/GAHT less often compared to the privately insured, 2) Some individual states had minimal insurance payer discrepancies in GAS/GAHT utilization, and 3) While these insurance payer differences in utilization remain at the national level, individuals living in states that implemented nondiscrimination policies experienced an increase in GAS/GAHT post-implementation. Overall, nondiscrimination policies implemented in 21 states and DC increased GAS/GAHT utilization, but challenges at the insurance payer and individual state level continue to threaten TGD individuals’ healthcare and lives.

Association Between Utilization of Gender-Affirming Care and Insurance Payer, State, and State Policy

Abstract Title
Analyzing Recruitment Methods to Increase Diversity in Biobank Collections

Background
Biospecimen collection has played an integral role in research by storing valuable data and tissue samples from both patients with diseases, such as cancer and HIV, as well as healthy control specimens for comparison. The results of these studies shed light on the pathogenic processes of diseases, provide risk stratification, validate new diagnostic tools, and develop targeted pharmacotherapies. Including diverse samples from various races, ethnicities, and backgrounds is essential to increase the generalizability of the results of these studies. Traditional university research participants are typically recruited from the student body and surrounding area, which may not accurately represent the demographics of the local population. Our objective was to create a tissue bank from healthy individual blood samples that is representative of the San Antonio, Texas population for use in future research and analyze traditional university research recruitment methods.

Methods
Traditional methods of university recruitment at UT Health San Antonio using campus fliers, posters, emails, and class announcements were implemented. 311 individuals were screened and a total of 101 participants were enrolled. Participants self-reported a medical history without significant health conditions and provided basic demographic information. Tissue samples were acquired from blood draws and then allocated for storage in the repository. Participant demographic data was compared to the 2020 U.S. Census San Antonio population using the chi-squared test. 

Results
There was a statistically significant difference (p < 0.05) from the expected San Antonio demographics and the recruited participants of racial groups (“White/Caucasian”, “Black or African American”, “Asian”, “Native Hawaiian and Other Pacific Islander”, and “Other”) and ethnic groups (“Hispanic/Latino” and “Not Hispanic or Latino”) except for “American Indian/Alaskan Native” (Expected: 2, Actual 2). There was an overrepresentation of participants who identified as “White/Caucasian” (Expected: 61, Enrolled: 78), “Asian” (Expected: 5, Enrolled: 14), and “Not Hispanic or Latino” (Expected: 37, Enrolled: 58). There was an underrepresentation of those who identified as “Black or African American” (Expected: 10, Enrolled: 3) and “Hispanic/Latino” (Expected: 64, Actual 42). Most participants reported “White/Caucasian” (83%).

Conclusion
Advertising research opportunities in universities is an effective way of increasing enrollment at the cost of diversity. The consequences of utilizing tissue samples from a limited demographic could have significant impacts for minorities. To increase the recruitment of the underrepresented, researchers suggest the use of social media, radio advertisements, health fairs, flea markets, churches, community centers, and translated fliers.

Analyzing Recruitment Methods to Increase Diversity in Biobank Collections

Introduction and Background: The need for cultural competency in healthcare is more critical than ever, as healthcare systems must effectively meet the needs of diverse populations. A significant gap exists in providing culturally competent hair care, particularly for patients with textured or curly hair. Currently, BUMC-P lacks inclusive hair care options for these patients, impacting their overall care experience. Inspired by the University of Iowa's successful initiative to develop equitable hair care options for hospitalized patients, this project aims to bridge that gap. Our goal is to enhance patient inclusion by addressing unique hygiene needs and elevating resident cultural competency, ultimately leading to more inclusive and respectful patient care.

Methods/Approach: 72 hair and 72 beard kits were created that included hair oil, wide-toothed combs, satin bonnets, and single-blade razors. Kits were modeled after the University of Iowa initiative. Each kit includes a QR code with which patients complete a patient satisfaction survey. RNs will distribute kits on select BUMC-P inpatient floors to patients with a BMAT score of 4. Survey data will be reviewed at 6 intervals. Kits will be built by residents accompanied by interactive educational sessions on culturally-sensitive healthcare. Pre and post surveys will be used to assess resident cultural competency.

Results: This quality improvement project is ongoing. By providing culturally appropriate hair and skin care products for hospitalized patients, we aim to increase patient inclusion, which may in turn increase patient satisfaction, a key metric in Medicare reimbursement. We anticipate that collaborating with trainees on the design and necessity of these hair and beard kits, trainees will forge personal connections with patients that will improve cultural sensitivity and promote trainee well-being.

Discussion/Conclusions: The Inclusive Hair, Inclusive Care project aims to improve patient sense of inclusion, provider-patient relationships, and resident cultural competency.


The Inclusive Hair, Inclusive Care Project: Enhancing Patient Inclusion and Resident Cultural Competency Through Inclusive Hygiene Products


Background:
Conjunctivitis is a familiar presentation in a pediatrician’s office, but sometimes the cause is less common. Ranging from viral, bacterial, allergic, and nonimmune etiologies, it is our job to take a detailed history and perform a thorough physical exam to determine the best therapy possible while preserving antibiotic stewardship and improving our patients’ outcomes.

Case Presentation:
An 18 year old female presented to clinic for evaluation of worsening right eye redness, swelling, and blurred vision over the span of two weeks. Yellowish drainage in the morning caused her eye to “crust shut”, and though she used warm compresses to wipe it away, the drainage returned quickly. She had rhinorrhea with congestion, but denied fevers. Lubricating eye drops had not been helpful in alleviating her symptoms, and eye movement caused pain as well. When asking about social history, she admitted to being sexually active with inconsistent use of condoms. She did have vaginal discharge. She was afebrile, and physical exam was notable for significant conjunctival injection with copious purulent drainage and mild swelling around her eye. A urine GC/Chlamydia and pregnancy test were ordered along with a culture from the affected eye. The patient was started on ofloxacin 0.3% solution and amoxicillin/clavulanate 875-125 mg while waiting for confirmatory diagnosis and evaluation by an ophthalmologist. Her urine was positive for Gonorrhea the following day, and eventually her eye culture revealed Group A S. pyogenes and beta lactamase negative N. gonorrhoeae. The ophthalmologist agreed on continuing her ofloxacin 0.3% solution while stopping amoxicillin/clavulanate, and going to her pediatrician’s office to receive Rocephin 1000 mg intramuscularly. The patient’s symptoms vastly improved after her antibiotic therapies were completed, and was recommended to follow up with ophthalmology to monitor for any vision changes.

Discussion:
There are different causes for conjunctivitis, viral being the most common in children, however there are cases of conjunctivitis that providers must address in a timely fashion to prevent blindness. Gonococcal conjunctivitis is a severe condition that requires urgent evaluation and treatment by an ophthalmologist due to concerns for corneal perforation leading to scarring and blindness. It is most commonly seen in neonates due to maternal infections, however pediatricians cannot forget that N. gonorrhoeae is the most common cause for bacterial conjunctivitis in sexually active teenagers and adults. Even though this patient was only being seen for conjunctivitis, one must never forget to ask about sexual history for this very reason.

An Unexpected Diagnosis of Conjunctivitis in an 18 Year Old Female

Unveiling Trends and Disparities: A Comprehensive Analysis of Adolescent Births in California - Navigating the Challenges and Prioritizing Adolecent Maternal Health

Background: The adolescent birth rate in the United States, measured as births per 1,000 females aged 15 to 19 years, has steadily declined since 1991, reaching a historic low of 16.7 per 1,000 females in 2019. Despite this positive trend, teenage pregnancy continues to pose significant social and health challenges, manifesting in heightened rates of maternal high school dropout, birth complications, infant mortality, adolescent incarceration, and chronic childhood medical conditions. This study aims to examine recent trends in teen birth prevalence, identifying associated risk factors and disparities in the state of California.

Methods: Conducted as a retrospective cohort study, we analyzed 372,352 teen births out of the 5,137,376 resident live births (7.25%) documented in the California Birth Statistical Master Files (BSMF) spanning from 2007 to 2016. Age at birth was categorized as 15 to 19 years, with a reference category of 20 to 29 years. The study investigated various birth outcomes, including low birth weight (LBW), very low birth weight (VLBW), preterm birth (PTB), very preterm birth (VPTB), small-for-gestational-age birth (SGA), large-for-gestational-age birth (LGA), and cesarean delivery (CD). Utilizing descriptive analysis, simple linear regression, and multivariate logistic regression (MLR), we calculated adjusted odds ratios (AORs) with 95% confidence intervals (CIs) to explore associations. The MLR analysis specifically delved into predictors of teen pregnancy, referencing mothers aged 20 to 29 years.

Results: While the teenage pregnancy rate exhibited a linear decline (p < 0.001) from 40.1 in 2007 to 15.8 in 2016 (per 1,000 females), certain maternal characteristics witnessed an increase over time. Adjusted odds ratios revealed that women with less than a high school education and those with a high school diploma had a 9.0-fold and 3.3-fold higher prevalence of teen births, respectively, compared to women with some college or an associate degree. Hispanic women were twice as likely to experience teen births compared to White women. Teen mothers were nearly 50% more likely to rely on Medi-Cal (public health insurance) and 46% more likely not to receive first-trimester prenatal care. Importantly, they were 43% more likely to be underweight.

Conclusion: Our findings emphasize the need to address racial and ethnic disparities and improve birth outcomes among adolescent births. Policymakers should prioritize ensuring pregnant adolescents have access to high-quality maternal health services, encompassing both pre and postnatal care, to safeguard the health of both mothers and infants in the years ahead.

Unveiling Trends and Disparities: A Comprehensive Analysis of Adolescent Births in California

Abstract Title
A Heart Under Siege: The Uncommon Tale of Salmonella Endocarditis 
Background
Infective endocarditis (IE) is an infection of the innermost layer of the heart, typically caused by bacteremia localized with platelets, fibrin, and inflammatory cells that form a vegetation on a heart valve. It is generally caused by gram-positive bacteria such as Staphylococcus aureus, Streptococci viridians, Enterococci, and coagulase-negative Staphylococci, with gram-negative pathogens accounting for a small fraction of IE cases. This case report highlights the presentation, diagnostic work-up, treatment, and outcome of a patient with the rare condition of Salmonella infective endocarditis.

Case Presentation
A 59-year-old male presented to the emergency department with decreased appetite, weakness, and altered mental status. He was initially diagnosed with pneumonia and Salmonella bacteremia and started on an antibiotic regimen. His hospital course was further complicated by a CT scan revealing a splenic abscess and a brain MRI showing multiple cerebral infarctions, both caused by septic emboli. Interventional radiology and neurology were consulted for these complications. A transesophageal echocardiogram (TEE) showed a 1.0 cm mitral valve vegetation. Infectious disease was consulted for antibiotic recommendations to manage the bacteremia while the patient awaited mitral valve replacement, as recommended by cardiothoracic surgery.
The patient, with a significant past medical history of heart failure, diabetes, hypertension, and 30-year smoking, initially presented with tachypnea, marked leukocytosis, and elevated lactic acid levels. Blood cultures revealed Salmonella non-typhi, prompting a switch to Ceftriaxone. The patient developed encephalopathy and altered mentation, with a brain MRI showing multiple tiny infarcts. A diagnosis of Salmonella infective endocarditis with septic embolization was confirmed via TEE.
A multidisciplinary approach involving interventional radiology, infectious diseases, neurology, and cardiothoracic surgery led to the decision for mitral valve replacement and concurrent coronary artery bypass grafting. Postoperatively, the patient developed atrial fibrillation, managed with an amiodarone drip and eventual cardioversion. He was discharged with an anticoagulant and follow-up plan.
At follow-up visits, the patient was stable, compliant with medications, and reported no new symptoms. His INR was closely monitored, and he was educated on the importance of medication adherence following mitral valve replacement. He was referred to occupational and physical therapy, as well as neurology, for his history of cerebral infarctions.

Discussion
This case underscores the importance of clinical suspicion, immediate symptom treatment, interdisciplinary collaboration, and tailored therapeutic interventions in managing rare cases of infective endocarditis. This report serves as a guide for clinicians encountering similar pathologies, highlighting the critical role of prompt microbiological investigations and tailored treatment regimens in improving patient outcomes.



A Heart Under Siege: The Uncommon Tale of Salmonella Endocarditis

Introduction
In recent years, hospitals have seen an overuse of emergency room (ER) facilities. This problem is particularly worse among the pediatric population as guardians approach the ER for low acuity reasons too. Thus, there is a need to explore the reasons behind this practice and review interventions to reduce it.
Methods 
Based on a registered protocol, studies were systematically searched in PubMed, using a developed search strategy with critical words by all four reviewers. Studies were included if they discussed emergency room overuse in pediatric populations, behavioral interventions to decrease this, causes of overuse or quality improvement projects assessing this problem; were published after 2014; were available as free full text; were Randomized Controlled Trials (RCTs), qualitative studies, case controls/cohort studies, descriptive studies; and were in English. The bias assessment was done using CASP Tool Kits and scored internally by one reviewer and checked by two more. Data extraction was done by two reviewers and then independently checked for completeness and accuracy by the other two.
Results
Of 240 studies found by initial database search, 48 were selected based on inclusion criteria and data was extracted from only 23 studies after all parameters were checked. Almost all the studies were from 2017-2022. 11 of the studies were conducted in the North America (Canada and US). There were 7 RCT and 7 experimental studies and 6 qualitative studies. The mean sample size for studies dealing with patient population was 1028 after excluding one outlier. Causes like gastroenteritis, fever, asthma and trauma were primary reasons for presentation. Younger children, patients without PCP, those coming in office hours and those from low education households overused ER more often. CT scan was the most overused imaging technique across studies. The relationship between physician prescription habits and ER overuse was studied across multiple studies, specially under the lens of over ordering chest X-ray plus antibiotics in bronchiolitis cases. Using tailored social resources handout, managing toddler behavior, better triage models and using video discharge instructions post-ER visit was associated with a much notable decrease in child hospitalization and ER visit frequency.
Conclusion
Although numerous studies discuss components of ER overuse among the pediatric population, there is still a scant number of studies that discuss interventions and their efficacy. Therefore, more studies need to be conducted introducing new interventions, specifically behavioral ones based on the existing studies talking about the causes, protocols, overused parameters and demographics.
Although numerous studies discuss components of ER overuse among the pediatric population, there is still a scant number of studies that discuss interventions and their efficacy. Therefore, more studies need to be conducted introducing new interventions, specifically behavioral ones based on the existing studies talking about the causes, protocols, overused parameters and demographics.

Nudging towards efficiency: Reducing emergency room usage in pediatric population

Title: Accuracy Benchmark Highlights AI's Potential to Transform Medical Education and Curricula Development

Background: The impressive capabilities of modern large language models (LLMs) like ChatGPT have recently been demonstrated in the context of the United States Medical Licensing Examination (USMLE). The National Board of Medical Examiners (NBME) released a study showing that these models can pass actual USMLE exams, with numerous studies showcasing improvements with successive models. However, a comprehensive analysis of these models', including ChatGPT 4 Omni, performance across specific medical content domains remains absent, limiting our understanding of their full potential in medical education. Furthermore, establishing a benchmark for accuracy is crucial to underline the reliability of these models in providing accurate and dependable information.

Methods: Utilizing 750 clinical vignette-based multiple-choice questions (MCQs) provided by medical schools, the performance of successive iterations of the leading LLM, ChatGPT—namely, ChatGPT 3.5 (GPT-3.5), ChatGPT 4 (GPT-4), and ChatGPT 4 Omni (GPT-4o)—were evaluated across various USMLE disciplines, clinical clerkships, and clinical skills such as diagnostics and management. Accuracy assessments were conducted using a standardized protocol, with rigorous statistical analyses employed to compare and analyze the reliability of these models in responding to the MCQs.

Results: GPT-4o achieved a remarkable correct response rate of 90.4%, significantly outperforming both GPT-4 (81.1%), GPT-3.5's (60.0%), and the medical student average of 59.3% (95% CI: 58.3%-68.3%). In preclinical disciplines, GPT-4o performed best in Social Sciences (95.5%), Microbiology (92.3%), and Immunology (92.9%). GPT-4's highest accuracy was in Behavioral and Neuroscience (86.5%), and GPT-3.5's highest was in Behavioral and Neuroscience (76.9%). In Clinical Clerkship categories, GPT-4o attained perfect scores in Family Medicine (100%) and Internal Medicine (100%), while GPT-4's highest was in Internal Medicine (95.5%), and GPT-3.5's highest was in Neurology (69.5%).

Conclusion: This study highlights the exceptional performance of the ChatGPT series across USMLE preclinical and clinical disciplines, showcasing the transformative potential of AI in medical education. The advancements seen in ChatGPT 4 Omni underscore its utility as an educational tool, suggesting a future where AI enhances learning and competency in medical fields. This study benchmarks groundbreaking accuracy, ushering in newfound reliability in the accuracy of AI. As AI integration in education grows, our findings emphasize the necessity for a formal curriculum to guide proper usage, ensuring oversight and validation to maximize benefits while maintaining educational standards.


Accuracy Benchmark Highlights AI's Potential to Transform Medical Education and Curricula Development

INTRODUCTION 

American football is renowned for being one of the most contact-intensive sports, frequently resulting in head injuries, with over 40% of retired National Football League (NFL) players showing evidence of long-term sequelae. Contrastingly, epidemiological data on head injuries in rugby, a sport with shared origins and similarities to American football, is lagging. Rugby has grown significantly in the United States over the past few decades. Despite the level of contact rivaling that of football, players wear little to no protective gear. Given the rising popularity of rugby in American sports, it is crucial to examine the rates and mechanisms of injury to better protect these high-performance athletes. 

METHODS 

The National Electronic Injury Surveillance System (NEISS) was queried for injuries related to playing rugby while using body part codes for head injuries between 2004-2023. NEISS offers estimates of national injuries reported to emergency departments across the United States. Narratives were screened for relevance, specific diagnosis, and mechanisms of injury. Non-cranial head injuries were excluded from the narratives, and nonspecific narratives only indicating a ‘head injury’ or ‘head trauma’ were only included if a loss of consciousness was noted. Weighted survey analysis determined incidence rates and 95% confidence intervals. Weighted binomial regression was used to analyze relationships between age and other binary variables. 

RESULTS

A total of 910 narratives produced a national estimate of 29,250 emergency department (ED) visits for traumatic brain injuries (TBI) in the United States from playing rugby between 2004-2023. Of all the estimated cases of TBI, 63.3% were concussions (Figure 1), of which 17.6% were accompanied by a loss of consciousness. The majority of those who sustained a TBI were male (70%), and the average age was 20 years (Figure 2). Male rugby players with TBI were older on average than females (P = 0.04). A total of 565 narratives (national estimate: 17,781) indicated the mechanism of injury, with tackles (27.4%) resulting in more injuries than any other observed mechanism.

The prevalence of head injuries among rugby players is a pressing issue, compounded by the lack of comprehensive research on injury frequency and on-field signs like motor incoordination. These injuries pose immediate and long-term risks, particularly for younger, inexperienced athletes who may have poorer tackling form. Addressing these challenges requires ongoing research, education, and improved injury prevention strategies, such as tailored protective equipment and training in safe play practices. Investing in these measures enhances player safety and supports the long-term health and performance of young athletes, highlighting the need for further studies into rugby players' neurologic health.



Premium content

Downloads

Next from AMA Research Challenge 2024

Level Up: Exploring the Effectiveness of Game Based Learning in Graduate Medical Education

Similar lecture

Leveraging Bias in Pre-trained Word Embeddings for Unsupervised Microaggression Detection