United States

OBJECTIVE
Despite rising popularity and performance, studies evaluating the use of LLMs for clinical decision support are lacking. Here, we evaluate ChatGPT 3.5 and 4’s capacity for clinical decision support in radiology via the identification of appropriate imaging services for two important clinical presentations: breast cancer screening and breast pain.
METHODS
We compared ChatGPT&#39;s responses to the American College of Radiology (ACR) Appropriateness Criteria for breast pain and breast cancer screening. Our prompt formats included an open-ended (OE) and a select all that apply (SATA) format. Scoring criteria evaluated whether proposed imaging modalities were in accordance with ACR guidelines. Three replicate entries were conducted for each prompt, and the average of these was used to determine final scores.
RESULTS
Both ChatGPT 3.5 and 4 achieved an average OE score of 1.830 (out of 2) for breast cancer screening prompts. ChatGPT 3.5 achieved a SATA average percentage correct of 88.9%, compared to ChatGPT 4’s average percentage correct of 98.4% for breast cancer screening prompts. For breast pain, ChatGPT 3.5 achieved an average OE score of 1.125 (out of 2) and a SATA average percentage correct of 58.3%, as compared to an average OE score of 1.666 (out of 2) and a SATA average percentage correct of 77.7%. 
DISCUSSION
Our results demonstrate the eventual feasibility of using LLMs like ChatGPT for radiologic decision making, with the potential to improve clinical workflow and responsible use of radiology services. More use cases and greater accuracy are necessary to evaluate and implement such tools.


AMA Research Challenge 2023

Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot

Welcome to the **largest national, multi-specialty research** event featuring research from nearly 700 medical students, residents, fellows, and international medical graduates.

Use the buttons below or navigation menu to access the event.
 


[![](https://assets.underline.io/uploads/markdown_image/1/image/3e33a8c754cc49610b30950617d813e7.png)](https://underline.io/events/434/sessions?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/551404d429c4e274f9a54cbdd7081918.png)](https://underline.io/events/434/posters?searchGroup=all)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/677d92f566b880cebd36905fc738fc10.png)](https://underline.io/events/434/lounge)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/c7392833c7261c3874c5b43fa2f4df19.png)](https://underline.io/events/434/help-desk)
 

#### Event Details

The 2023 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. **Log in at any time throughout the event to:**

* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research
* **Attend education sessions** with physician experts on key research topics

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges, and compete to win a $10,000 grand prize sponsored by Laurel Road. 

### Grand Prize Sponsor

AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on February 4, 2024 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use this link [here](https://www.ama-assn.org/about/events/ama-research-challenge) to register for free.**

Registration Is Required

Join the AMA Research Challenge poster symposium and semifinals to explore top research posters, score posters in the semifinals (an AMA member exclusive opportunity) and attend education sessions with experienced physicians to learn tips to advance your research and career. 

OBJECTIVE
Despite rising popularity and performance, studies evaluating the use of LLMs for clinical decision support are lacking. Here, we evaluate ChatGPT 3.5 and 4’s capacity for clinical decision support in radiology via the identification of appropriate imaging services for two important clinical presentations: breast cancer screening and breast pain.
METHODS
We compared ChatGPT's responses to the American College of Radiology (ACR) Appropriateness Criteria for breast pain and breast cancer screening. Our prompt formats included an open-ended (OE) and a select all that apply (SATA) format. Scoring criteria evaluated whether proposed imaging modalities were in accordance with ACR guidelines. Three replicate entries were conducted for each prompt, and the average of these was used to determine final scores.
RESULTS
Both ChatGPT 3.5 and 4 achieved an average OE score of 1.830 (out of 2) for breast cancer screening prompts. ChatGPT 3.5 achieved a SATA average percentage correct of 88.9%, compared to ChatGPT 4’s average percentage correct of 98.4% for breast cancer screening prompts. For breast pain, ChatGPT 3.5 achieved an average OE score of 1.125 (out of 2) and a SATA average percentage correct of 58.3%, as compared to an average OE score of 1.666 (out of 2) and a SATA average percentage correct of 77.7%. 
DISCUSSION
Our results demonstrate the eventual feasibility of using LLMs like ChatGPT for radiologic decision making, with the potential to improve clinical workflow and responsible use of radiology services. More use cases and greater accuracy are necessary to evaluate and implement such tools.


poster

Background: The EMBARK trial is a phase 3, randomized, study evaluating use of enzalutamide with or without concomitant leuprolide in non-metastatic castrate sensitive prostate cancer patients. Trial enrollment relied upon conventional imaging which under detects metastatic disease in comparison to PSMA-PET imaging. The aim of this post-hoc retrospective analysis was to describe the PSMA-PET findings and survival outcomes in a patient population representative of the EMBARK trial. 

Methods: This was a post-hoc, retrospective analysis of 5 prospective studies of PSMA-PET conducted at UCLA from 2016 to 2021 that included patients with recurrent PCa. Patients were included if they met eligibility EMBARK trial enrollment at the time of their PSMA scan as per the following: Rising PSA above 1.0ng/mL (post RP and SRT) or 2.0ng/mL >nadir (post-RT), PSA doubling time ≤9 months, Serum testosterone ≥150 ng/dL. PSMA PET findings (PROMISE miTNM stage, number of lesions) and survival outcomes were collected from clinical reports. Descriptive statistics (median, range) and Kaplan-Meier modeling (median, Confidence intervals) were used.

Results: From 2002 patients screened, 146 patients were included in the analysis. Median pre-scan PSA levels and PSA doubling time in patients who underwent RP (n=85), dRT (n=22), post-RP SRT (n=39) and overall (n=146) were 2.4 ng/mL (1-72.1), 8.3 ng/mL (2.37-202), 2.58 ng/ml (1.1-32.3), 2.7 ng/mL (1-202), and 3.5 months (1-9), 3.3 months (1-9), 4.4 months (1-9), 3.7 (1-9) months, respectively. PSMA-PET was positive in 80%, 41%, 85%, 83% of RP, dRT, RP + SRT and overall populations, respectively. Poly-metastatic disease (≥5 lesions) was found in 19%, 32%, 21% and 21% of cases, respectively. PSMA-PET detected disease only localized to the prostate fossa (miT+N0M0) in 7%, 32%, 0% and 9%. PSMA-PET detected any distant metastatic disease (miTxNxM1) in 33%, 41% 56% and 40% respectively. PSMA-PET detected metastatic nodal disease (N1 and/or M1a) in 58% 32% 74% and 39% and osseous disease (M1b) in 19% 23% 31% and 23% cases. Twenty-eight patients received androgen receptor-axis therapy (ARSi) or androgen deprivation therapy (ADT) following PSMA-PET had available outcome data. Recurrence free survival in ARSi compared to the ADT cohort was 53 months vs 22 months (p=0.008).
Conclusion: In this “EMBARK-like” cohort of patients with high-risk nmCSPCa and PSA DT<9 months, PSMA-PET detected M1 disease in 40% of patients, over one-fifth of cases presented with 5 or more lesions and those receiving ARSi had significantly improved recurrence free survival. Further studies are needed to assess its independent prognostic value and use for treatment guidance.

PSMA PET findings in an “EMBARK-like” cohort of patients with high-risk non-metastatic hormone-sensitive prostate cancer: A single center post-hoc retrospective analysis

BACKGROUND:
Rheumatoid arthritis (RA) is a chronic autoimmune disease impacting about 1.3 million Americans, with women much more likely to be affected than men. The development of this systemic inflammatory disease is influenced by both genetic and environmental factors. Certain environmental factors, such as diet, can be more readily controlled by individuals, and therefore may serve as more attainable therapeutic targets for preventing and mitigating RA and its symptoms. 

Various foods and nutrients are associated with inflammatory biomarkers1,2,3,17, and thus have been linked with increased or decreased risk of RA. However, these associations have been inconsistent across studies likely due to overall systematic bias, measurement error associated with individual food groups and nutrients, and limited investigations evaluating the relationship between overall dietary patterns and RA in larger sample sizes4,5,6,12. Furthermore, few studies that have examined the impact of overall dietary patterns on RA but have had conflicting results and/or generally involved smaller sample sizes. 

OBJECTIVE:
Our objective is to evaluate the association between dietary quality, as measured by the 2015 Healthy Eating Index (HEI-2015), and risk of incident rheumatoid arthritis (RA) in postmenopausal women.

METHODS: 
109,591 postmenopausal women without baseline RA were prospectively followed as part of the Women’s Health Initiative (WHI) for an average of 8.1 years. Baseline diet was measured using a food frequency questionnaire (FFQ) and diet quality was evaluated using the HEI-2015. Multivariable Cox proportional regression analysis was performed evaluating the association of diet quality with self-reported physician diagnosed RA after adjusting for age, race, education status, income, and body mass index (BMI). 

RESULTS:
During 1,241,532 person-years of follow-up, 5,823 incident RA cases were identified. Compared to quartile 1, quartiles 2, 3, and 4 of the HEI-2015 scores were associated with a reduced RA risk of 3%, 13%, and 18%, respectively (p-trend <0.001). A higher score for total fruits (p-trend=.006), whole fruits (p-trend<.001), total vegetables (p-trend=.004), dark green vegetables (p-trend<.001), whole grain (p-trend=.004), total dairy (p-trend=.004), and seafood/plant protein (p-trend=.015) consumption was significantly associated with lower rates of incident RA independent of age and confounders. Conversely, a higher score for consumption of percent saturated fat (p-trend=.001) was significantly associated with higher rates of incident RA (Table 3).

CONCLUSIONS:
A higher quality diet rich in fruits, vegetables, whole grain, dairy, and seafood/plant protein, is associated with a lower risk of developing RA in postmenopausal women. Meanwhile, a less healthy diet with higher intake of percent saturated fat is associated with higher risk of RA.

Association of Diet Quality with Risk of Incident Rheumatoid Arthritis in the Women’s Health Initiative

Background/Purpose: The purpose of this study was to investigate factors associated with objective cognitive impairment in patients diagnosed with Systemic Lupus Erythematous (SLE). The prevalence of subjective complaints are common, and previous studies have showed a general decline in immediate memory and total memory. 
Methods: This study was conducted using baseline data from ClearMEMory (Clearing Lupus Fog with Memantine). Seventy-nine participants who reported neuropsychiatric symptoms and came for screening were included. The dependent variable was the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), a brief, age-normalized, individually administered battery to measure cognitive decline or improvement. Potential predictive variables were sex, race, ethnicity, age, ESR, anti-dsDNA antibodies, the polysymptomatic distress scale (PSD) and the hospital anxiety and depression scales (HADS-A, HADS-D). Regression models were run to determine covariates that effect the RBANS total and the RBANS immediate memory. The models examined HADS scores between 6-12 to determine if there was a statistically significant difference when the scores went from normal to depressed. The models used PSD values 12-16 which are values that cross the threshold for diagnosis of fibromyalgia.
Results: Participant’s median age was 51 years, and they were predominantly female (96%), non-Hispanic (94%) and White (70%) (Table 1). We used linear regression to model the RBANS total score. A model including HADS-D, PSD, and race was the most significant (p = 0.0004) (Table 2). As the HADS depression score increased from 6 to 12, the RBANS total decreased by 3.95 points and as the PSD total increased from 12 to 16, the RBANS total decreased by 2.35 points on average. Non-Whites, when compared to Whites had a decrease in the RBANS total by 7.77 points. The most significant model for RBANS immediate memory included the same factors (p = 0.0004, Table 3). 
Conclusion: Results support race, depression, and polysymptomatic distress as significant contributors to the risk of cognitive impairment in patients with SLE. Disease activity did not appreciably affect cognition in this participant sample. As participant scores crossed thresholds for depression or fibromyalgia diagnosis, measurable declines in cognition, particularly immediate memory, were observed. Further studies with different measures of cognition are necessary to understand the impact of race in these models.

Predictors of Cognitive Impairment in Individuals with Systemic Lupus Erythematosus

**Scoring is only possible in the semifinals hall**

**Abstract:**



Risk of Lupus Death According to Rural Urban Residence

VEGF, Neuregulin, and Postoperative Vasoplegia Following Cardiopulmonary Bypass Surgery

Background
Lung cancer is the leading cause of cancer deaths worldwide. The increasing use of minimally invasive surgical approaches has expanded the scope of treatment options for patients with lung cancer. However, these operations are not risk-free as there have been several reported cases of intraoperative catastrophes, with major hemorrhage from the pulmonary artery being the most common. The objective of this study was to develop a computer vision-based approach to reliably identify the pulmonary artery during robotic right lower lobectomies. 

Methods
Twelve patients with biopsy-proven Stage I non-small-cell lung cancer of the right lower lobe underwent robotic lobectomy with mediastinal lymph node dissection. Complete videos of each operation were obtained, and video fragments of the pulmonary artery were identified by two board-certified thoracic surgeons. Annotation masks of the pulmonary artery were then created using Computer Vision Annotation Tool, an open-source web-based annotation tool. The labeled data was next used to train a state-of-the-art instance segmentation algorithm called Mask Regional-Convolutional Neural Network (Mask R-CNN) using an 80:20 random sample split from ten cases for training and validation, respectively. Two cases were utilized for generalization testing. Three custom performance metrics commonly utilized in deep learning-based instance segmentation were used to evaluate our approach: Intersection over Union (IoU), Average Precision at IoU threshold of 50% (AP50) and at 75% (AP75). 

Results
Annotation masks of the pulmonary artery were created in 4,139 images across twelve cases of robotic right lower lobectomy. 2,893 and 684 annotated images across ten (83.3%) cases were used for training and validation, respectively; and 261 and 301 annotated images from two (16.7%) cases were used for generalization testing. A mean IoU of 87.7% was achieved in the validation dataset. The AP50 attained by our model was 96.7% and AP75 was 82.3%. In generalizability testing, when the model was tested on data never exposed to it, it was able to partially generalize with a mean IoU of 86.0%. 

Conclusion
Our study shows that our Mask R-CNN model was able to identify surgical anatomy during robotic lobectomy with high accuracy in the validation dataset. While still at an early stage, more variable labeled data and collaborative efforts are needed to improve the generalizability of our model. We envision such a computer vision system to be valuable in early surgeon training and incorporate future real-time augmentation of video feedback to improve outcomes.






Using Artificial Intelligence for Automatic Detection of the Pulmonary Artery During Robotic Right Lower Lobectomies

Background: Endoscopy is commonly used to diagnose upper gastrointestinal (GI) system disorders. However, excess mucus can significantly impair visualization, and reducing mucus intra-procedurally can increase procedure time. Several studies have evaluated the efficacy of pre-medication with mucolytic agents prior to gastroscopy to improve the diagnostic yield of endoscopy and reduce procedural time. Here, we systematically review recent evidence assessing the efficacy of mucolytic agents, including N-acetyl cysteine (NAC), in improving visibility in upper GI endoscopy.
Methods: Studies were identified by interrogating the PubMed database using the search terms "endoscopy" and "N-acetyl cysteine" for the period of 2016-2023. Only randomized control trials were included in the search. Records written in languages other than English were excluded. Ten (10) records matching these criteria were retrieved. Title and abstract screening resulted in five (5) records, which were subsequently extracted.
Results: The studies were conducted in the United Kingdom, Chile, Portugal, Thailand, and the Czech Republic. Four (4) studies consisted of randomized, double-blinded, placebo-controlled trials, while the remaining study was a randomized, non-blinded controlled trial. Four (4) of the studies showed that pre-administration of NAC (600 mg) plus simethicone (100 mg) significantly improved the visibility of the upper GI tract (esophagus, stomach, and duodenum) when compared to placebo control. The only study that failed to show an improvement in visibility scores was attributed to sub-threshold doses of NAC (400 mg) and simethicone (20 mg). Although none of the studies included an arm consisting of NAC alone, one study comparing simethicone +/- NAC showed superior visibility when NAC was administered at a high dose (1000 mg), while another study comparing simethicone +/- NAC failed to show improvement when NAC was administered at a moderate dose (600 mg). Lastly, one study found that the procedure time and volume of mucosal cleansing solution required were significantly reduced when NAC and simethicone were administered together.


Conclusion: Our survey of recent randomized control trials supports that pre-medication with NAC (600-1000 mg) and simethicone (100 mg) significantly improves the visibility of the upper GI mucosal surfaces during upper endoscopic procedures and, therefore, may improve diagnostic yield in settings where visibility is impaired. This approach may be particularly useful in lower-resource settings where less sophisticated endoscopic tools are utilized.

Efficacy of N-Acetyl Cysteine in the Prevention of Post-Operative Atrial Fibrillation and Acute Kidney Injury Post-Cardiac Surgery: A Systematic Review

Background: Estimated Glomerular Filtration Rate (eGFR) is the standard method for determining kidney function and chronic kidney disease (CKD) stage classification. In the Spina Bifida (SB) population, eGFR estimates from a creatinine-based equation may not accurately represent kidney function due to the lower muscle mass in this population. We hypothesized that estimating equations utilizing Cystatin-C (CysC) would yield different CKD classifications than Creatinine (Cr) – based equations for the adult SB population. 

Methods: We conducted a retrospective chart review of all patients seen in an adult urology clinic with a diagnosis of spina bifida from 2004-2021 that were managed in our urology clinic with creatinine and cystatin C values available. All patients were > 18 years old. A cross-sectional study of patients with a CysC and concurrent Cr measurement was performed; the most recent lab values were utilized if multiple existed. Three adult eGFR equations were utilized. Patients were staged based on CKD classification (GFR > 90, GFR 60-90, GFR <60). A Chi-square test was conducted to determine the difference in CKD classification based on each equation.

Results: A total of 105 SB patients (63% female) were included in cross-sectional analysis. The mean age was 33 ± 6.5 years. Medical comorbidities including diabetes (12.4%), hypertension (28.6%), obesity (34.2%) were noted. Table 1 shows the difference in eGFR and CKD classifications between equations. On Chi-square analysis, CKD classifications were significantly different by the type of equation used (p = 0.007). Pair-wise comparison showed that significance was driven by the difference between cystatin C and creatinine equations (p = 0.002) and the cystatin C and combined equations (p = 0.05). 

Conclusion: In an adult SB population, CysC-based equations result in lower eGFRs and different CKD classifications. Determining an appropriate eGFR measurement is essential to assessing and managing renal function in these medically complex patients.

Variability in Cystatin C and Creatinine-Based equations for Chronic Kidney Disease Staging in Adult Spina Bifida Patients

Background:
Financial difficulties have been associated with cancer patients delaying or deferring necessary care, resulting in later presentation with more advanced disease and worse clinical outcomes. Various sociodemographic factors have been linked to financial toxicity endured by cancer patients and negative clinical outcomes, including race, baseline financial status, region of residence, and insurance status. The association between specific financial factors and various clinical outcomes in particular forms of cancer requires further investigation.

Methods:
A cross-sectional study was conducted assessing 488,853 prostate cancer patients from the Surveillance, Epidemiology, and End Results (SEER) Program between the years of 2010 and 2018. The association between annual household income (HHI) and outcomes of prostate cancer were examined using ANOVA and Chi-square analyses comparing clinical measures based on categorical HHI groupings.

Results:
The average age across all HHI classifications was 66.63 years. Analysis demonstrated that a significantly higher percentage of African American patients fell within the <=35k and 35-55k income categories compared to other races (p<0.001). Patients with a lower HHI had higher PSA levels (12.10 vs. 10.90, 10.61, and 10.37 for <=35k, 35-55k, 55-75k, and >=75k, respectively, p<0.001) and the highest rate of disease metastasis to bone (6.3%, 5.6%, 5.7%, and 5.8% for <=35k, 35-55k, 55-75k, and > 75k, respectively, p<0.001) upon diagnosis. Patients in lower income brackets also demonstrated lower rates of undergoing surgical treatment (31.9%,
36.9%, 37.6%, and 35.1% for <=35K, 35-55k, 55-75k, and > 75k, respectively, p<0.001). Lastly, patients with lower HHI demonstrated progressively higher rates of cancer-specific-mortality (8.4%, 6.8%, 6.5% and 5.1% for <=35k, 35-55k, 55-75k, and >75k, respectively, p<0.001), as well as higher overall mortality (21.2%, 18.0%, 15.0%, and 11.9% for <=35k, 35-55k, 55-75k, and >75k, respectively, p<0.001).

Conclusions:
These data suggest that prostate cancer patients with lower household income are diagnosed with more aggressive disease, tend to undergo surgery at lower rates, develop more advanced disease, and endure worse clinical outcomes than those in higher income brackets. African Americans appear to be disproportionately affected compared to other races. Healthcare providers should be made aware of the clear associations between lower income and more aggressive disease at diagnosis, lower rates of surgical treatment, and worse cancer-specific and overall mortality so that adequate support may be provided regardless of financial difficulties.

The impact of household income on prostate cancer diagnosis, treatment, and outcomes

Background
Kidney stones are a common urological affliction in North America, affecting approximately 7 to 13% of the population. Abdominal pain is a frequent symptom, often prompting patients to present to the emergency department, where a CT scan confirms the diagnosis. A significant number of these patients undergo urological management in the form of temporary ureteral stent placement in the acute setting. The American Urological Association has guidelines which offer a framework for appropriate therapeutic interventions for renal colic, ranging from pain management and trial of medical expulsive therapy to surgery. This analysis seeks to evaluate the application of these guidelines at a tertiary medical center in patients who present with abdominal pain and are diagnosed with nephrolithiasis.
Methods
Chart review was conducted for patients who underwent ureteral stent placement for nephrolithiasis from January 1st to December 31st, 2022 at the University of Cincinnati Medical Center in Cincinnati, OH. Stones were classified based on size: less than 5 millimeters, between 5 and 10 millimeters and more than 10 millimeters. We assessed for infection, defined as more than 10,000 WBC/L or evidence of leukocyte esterase or nitrates on urinalysis, elevated creatinine, intractable pain, and presence of hydronephrosis on the affected side.
Results
Of the 174 patients who received stent placement in 2022 for a diagnosis of nephrolithiasis, 103 underwent the procedure emergently at the time of initial presentation. 94/103 (91.2%) had stones < 10 mm and 1 of these 94 (1.1%) received a stent in the absence of severe pain, infection, or elevated creatinine. 19/94 (20.2%) received stenting due to pain alone, and 75/94 (79.8%) had multiple indications (per AUA guidelines) for the procedure. 59 patients with a stone size < 10 mm underwent stenting after discharge and failure of medical therapy; 53 (89.8%) of these patients had at least one possible indication for stenting at the time of initial presentation.
Conclusion
Peer-reviewed guidelines offer a valuable framework to improve patient care by correlating therapy with appropriate indications. The AUA’s 2016 guidelines for surgical stone management encourage a trial of medical therapy for stones < 10 mm if not accompanied by infection or renal failure. Our findings suggest that these guidelines are effectively preventing unnecessary surgical intervention, as evidenced by the low rate of stenting without appropriate indications (1.1%), but may be driving overly conservative decision making, as a significant number of patients were discharged with medical therapy who may have benefited from emergent stenting.

Premium content

Next from AMA Research Challenge 2023

PSMA PET findings in an “EMBARK-like” cohort of patients with high-risk non-metastatic hormone-sensitive prostate cancer: A single center post-hoc retrospective analysis

Similar lecture

Deep Transfer Learning for Breast Cancer Detection Using 2D Screening Mammography