United States

Introduction: 
There are multiple factors that can impact a student’s success in a surgical clerkship, including preclinical curriculum and grades, previously completed clerkships, clerkship sequence and schedule, length of clerkship, work-study balance, and quality of education during the clerkship. These factors make it difficult for medical educators to assess the effectiveness of their preclinical and clinical curricula. This is the first systematic review to date that has analyzed the factors that negatively or positively influence NBME Surgery Subject Exam scores. 


Methods: 
Computer systematic literature searches of the Embase and Pubmed databases were performed. The studies included in this systematic review were papers testing variables that affect NBME subject exam performance of medical students in their third-year surgical clerkships.


Results: 
The literature search resulted in 2,823 articles, of which 17 were selected for analysis. The studies included in this review were grouped into three categories based on variables assessed: preclinical factors, third-year clerkship scheduling or structure, and surgical clerkship educational methods.
Analysis shows that the quality of medical students’ preclinical education plays a strong factor, as USMLE Step 1 scores and second-year NBME scores have a strong positive correlation with exam scores, while shortened preclinical curricula negatively impact third-year NBME exam scores. While the length of the surgery clerkship has varied significance based on multiple studies, the sequence of clerkships showed a consistent correlation with third-year NBME exam scores. Taking medicine clerkship prior to the surgery clerkship showed significant improvement in third-year NBME exam scores across multiple studies. The quality of clinical education received at clerkships is also very important; multiple studies of different educational models have shown that increased student engagement through presentations, small-group case-based learning, and direct resident interaction resulted in increased test scores. Additionally, the student-perceived educational quality along with preceptor-perceived clinical competency via clerkship evaluations and OSCE are positively correlated with NBME subject exam scores.


Conclusion: 
This systematic review looked at multiple factors that influence student success of the NBME Surgery Subject Examination and should be a useful tool for medical educators and students looking to increase NBME Surgery Subject Examination grades at their institution.


AMA Research Challenge 2022

Factors Influencing NBME Surgical Shelf Exam Scores: A Systematic Review

poster

Welcome to the largest national, multi-specialty research event **featuring research from 750 medical students, residents and fellows, and international medical graduates.**


#### Event Details
The 2022 American Medical Association Research Challenge includes a poster symposium hall and a semifinals hall with the top scored research. Log in at any time throughout the event to:
* **View posters and presentations** on a variety of topics and specialties
* **Score posters in the semifinals** to help decide the top five finalists (AMA member exclusive opportunity)
* **Connect with participants** and provide feedback to help advance their research

At the conclusion of this event, the AMA will announce the top five finalists who will advance to the AMA Research Challenge finals, present their research to an elite panel of judges and compete to win a $10,000 grand prize sponsored by Laurel Road.
 

#### Attend Live Forums
**Oct. 20, 6 — 7 p.m. Central**
 **How to plan a career in academic medicine** 
 Learn how to plan a career in academic medicine and get your questions answered. [Access Here](https://underline.io/events/363/lounge)
 
 **Speakers**
![](https://assets.underline.io/uploads/markdown_image/1/image/d5adb133dfc58eb0a6d469e9cedcd5d4.jpg)
Ricardo Correa, MD, EdD
![](https://assets.underline.io/uploads/markdown_image/1/image/5f9437ab87861538bf02b9ca7c0d926e.jpg)
Stuart M. Greenstein, MD
![](https://assets.underline.io/uploads/markdown_image/1/image/05b1b637c99286d558c371f8e8b185ce.jpg)
Stephanie Strohbeen, MD
 
 **Oct. 22, 11 a.m. — Noon Central**
 **How to get published**
 Learn tips to get your research published and get your questions answered. [Access Here](https://underline.io/events/363/lounge)
 
 **Speakers**
![](https://assets.underline.io/uploads/markdown_image/1/image/8ce6235e957d31cb5e6f9097cc6f6972.jpg)
Jessica Li, MD/PhD Candidate
![](https://assets.underline.io/uploads/markdown_image/1/image/9950ef2e0237f54cb3d8280334d42491.jpg)
Charles Lopresto, DO

### Sponsor
AMA would like to thank Laurel Road for sponsoring a $10,000 grand prize for the winner of the AMA Research Challenge. Laurel Road is not involved in the judging or scoring process to select the winner. The winner will be announced on Dec. 7 during the AMA Research Challenge finals. Payment of the grand prize will be directly from Laurel Road. See the AMA Research Challenge Official Rules for terms and conditions.

![](https://assets.underline.io/uploads/markdown_image/1/image/611e331d4a0eba0920f240cfc04b4d77.png)

**There is no registration fee for this event! **

Register to explore research in a variety of topics. AMA members get exclusive access to score posters in the semifinals.

**Please use the link below to register for free if you have not already done so:**

https://www.ama-assn.org/about/events/ama-research-challenge

Registration Is Required

Welcome to the largest national, multi-specialty research event featuring research from 750 medical students, residents and fellows, and international medical graduates.


Background
Pre-clinical medical students at our institution are trained in sterile technique by volunteer scrub technicians in a simulated operating room experience. Standardized patients (SPs) have long been recognized for their contributions to medical education, but have yet to be broadly integrated into simulations that go beyond role play scenarios, such as scrub training. Their unique skill set makes them strong candidates to lead hands on simulation-based training sessions.
Methods
Pre-clinical medical students were invited by email to sign up for one of two optional scrub training workshops in our institution’s simulation center. One session was led by a scrub technician and a senior medical student, as it has been in the past. The other training was led by specially trained standardized patients. Students were asked to complete anonymous pre and post-surveys which collected demographics, attitudes towards simulation, attitudes towards the training itself, and a brief quiz on aseptic hand hygiene. Descriptive statistics were calculated for all variables and t-tests were used to assess the relationship between groups. 
Results
Both the standardized patient group and the scrub tech group saw significant increases in their pre and post scores (0.7 to 0.9; p<0.01 for both groups). There was no appreciable difference when the post-scores of both groups (0.9 and 0.9) were compared to one another. Similarly, there was no significant difference between students’ perceived quality of the training nor their confidence in scrubbing in following the training (p=0.0805 and p=0.619, respectively). Overall, both groups saw a significant increase in their scrubbing confidence (p < 0.01).
Conclusion
Our results suggest that scrub training led by trained standardized patients is comparable to those led by scrub technicians. Both trainings were successful, as evidenced by increased quiz scores and increased scrubbing confidence. Importantly, learners did not perceive a qualitative difference between the two trainings. In the future, SP-led scrub training may lead to more flexibility in scheduling, and ultimately increased learner participation. Although already heavily utilized in medical education, scrub training may represent another area where SPs can meaningfully contribute.

Standardized Patients as Effective as Scrub Technicians at Scrub Training Medical Students

Abstract Title: Considerations for designing and implementing a multi-institution undergraduate medical education experience

Background: Cancer Care Experience (CCE) is an extracurricular program for undergraduate medical students to explore oncology. With the transition to virtual learning during the pandemic, program leaders offered CCE to students at Duke University School of Medicine and Penn State College of Medicine as a dual-institutional program. To our knowledge, there are few other dual-institution programs discussed in the literature, with most focusing on graduate medical education. Therefore, we aimed to assess 1) virtual learning effectiveness, 2) student perspectives on a multi-institutional model, and 3) the feasibility of implementing a multi-institutional program. 

Methods: An electronic voluntary Qualtrics survey of Likert-type statements was distributed to participants at the end of the program. Percent agreement to statements was determined and reported as a frequency. A Friedman Test (p<0.05) with post hoc analysis was conducted to assess preference for different program modalities.

Results: Seventeen (26%) students completed the voluntary survey. Students indicated that participation expanded their understanding of oncology care (100.00%) and raised their comfort in communicating bad news (88.20%). Interestingly, students felt comfortable learning on the online platform (93.80%), but few agreed that it was effective for engagement in large group sessions (37.50%). There was a statistically significant difference in preference among models of instruction (χ2(4)=38.315, p<0.001), with the multi-institution hybrid model ranked highest. Comparisons between multi-institution hybrid and single-institution in-person (Z=-2.718, p=.007), single institution virtual (Z=-3.602, p<.001), and single-institution hybrid (Z=-2.829, p=0.005) demonstrated a preference for the multi-institutional hybrid model.

Conclusion: Most students agreed that the virtual platform was convenient, engaging, and had few limitations. They agreed that a multi-institutional program was valuable and would recommend it to others. From an implementation perspective, there was a benefit of reaching more students and having access to additional faculty and resources. This approach allowed individuals from geographically distant institutions to participate, highlighting inter-institutional strengths, and facilitating broader educational and mentorship opportunities. Our study demonstrates the benefits of multi-institution programs and hopefully will encourage more collaboration in the future to host multi-institutional programs.

Considerations for Designing & Implementing a Multi-Institution Undergraduate Medical Education Experience

Background
Physical examination (PE) is an essential component of high-quality medical care that can aid in diagnosis, guide workup, and impact management decisions. Incomplete or deficient PE is a common source of diagnostic error and contributes to adverse patient outcomes. PE is one of many skills that should be developed and cultivated during medical training, but direct observation of, and feedback regarding, PE is limited. The coronavirus disease 2019 (COVID-19) pandemic led to a marked increase in telemedicine visits and a decline in in-person appointments, posing new challenges to performing the PE. Furthermore, while PE skills are associated with competency in other clinical domains during in-person encounters, little is known about PE skills during telemedicine encounters.

Methods
Forty-five internal medicine interns (24 females and 21 males) from two academic health centers participated in a clinical skills assessment via an online platform where they were evaluated by volunteer patients and faculty preceptors. Preceptors rated resident performance in seven domains: 1) physical examination technique; 2) identification of physical signs; 3) clinical communication; 4) differential diagnosis; 5) clinical judgment; 6) management of patient concerns; and 7) maintenance of patient welfare. The Spearman correlation coefficient was used to evaluate relationships between performance in these domains. Resident self-ratings for each domain were also obtained and compared to preceptor scores.

Results
PE technique was significantly correlated with accurate identification of physical signs (r = 0.42, P = 0.004), clinical communication (r = 0.31, P = 0.036), clinical judgment (r = 0. 60, P < 0.0001), and managing patient concerns (r = 0.41, P = 0.005). Residents rated their performance more highly than did preceptors on the domains of clinical communication, clinical judgment, and managing patient concerns. Residents highly rated the feedback they received during the experience.

Conclusion
While the telemedicine PE differs from the in-person PE, our findings highlight that it remains an essential clinical skill that is associated with performance in multiple other clinical domains. Given the rise in telehealth visits during the COVID-19 pandemic and resident overestimation of their skills, there is an increased need for dedicated telemedicine PE teaching.

Assessment of graduate medical trainee clinical skills during telemedicine encounters

Background
The authors identified gaps in the Indiana University School of Medicine (IUSM) reproductive curriculum and surveyed students for interest. Respondents expressed interest in comprehensive pregnancy options counseling, which was not previously included. Using evidence-based recommendations, the authors collaborated with faculty to develop an educational session about medical shared decision-making related to pregnancy. As a result, a pregnancy options counseling panel was added to the preclinical Endocrine, Reproductive, Musculoskeletal, and Dermatology (ERMD) course.

Methods
The authors and reproductive course management team developed an optional, live 90-minute physician panel held via Zoom videoconferencing software and disseminated to students at all nine regional campuses. Three recorded lectures, fifteen minutes each, were produced by experts about abortion, adoption, and continuing pregnancy to provide background information. The panel included providers with diverse practices and opinions about abortion and reproductive healthcare. After the panel, a 19-item survey was distributed including questions asking students to rate abortion, adoption, and pregnancy continuation counseling preparedness before and after the panel on a scale of 1-5. Paired t-tests were performed to determine whether preparedness significantly improved. The study was reviewed as an exempt project by the IUSM Institutional Review Board (protocol #13987).

Results
Out 353 students second year students, 207 (57%) attended the panel. Fifty-three attendees (26%) responded to the survey; 39 (73%) identified as female, 12 (23%) as male, and 2 (3%) as nonbinary. 41 (77%) respondents were White, 6 (11%) Asian, 4 (8%) Black or African American, and 2 (4%) identified as other. The mean improvement for abortion counseling preparedness was 1.33 (p<0.001, 95% CI [1.12,1.53]). For adoption counseling the mean improvement was 0.75 with (p<0.001, 95% CI [0.52,0.98]). Finally for pregnancy continuation mean improvement was 0.87 with (p<0.001, 95% CI[0.64,1.09]).

Conclusion
Overall students rated their preparedness significantly higher after attending the panel for comprehensive pregnancy options counseling. Using a statewide physician videoconference panel modality can help provide students across a multi-campus medical school access to physician expertise and unified content. Having access to the experts from other regional campuses is a powerful resource for bridging gaps in pregnancy counseling and creating much needed standardized curriculum.

Development and Implementation of Pregnancy Options Counseling Curriculum in Preclinical Medical Education

Background
A minimally-invasive approach is the standard for most benign pelvic surgeries, which now are increasingly performed with robotic assistance. Currently, there is no standardized curriculum for robotic training among Ob-Gyn residents and no method of quantitative assessment of robot confidence at Creighton. 

Methods
PGY1-PGY3s completed a pre-survey to establish baseline comfort with robotic surgery. A Likert scale (1 - very unsatisfied or not at all confident, 5 - very satisfied or extremely confident) was used to determine self-assessed comfort with robotic components: port and instrument placement, docking, bipolar and monopolar instrumentation, camera, switching instruments, suturing, and overall comfort with surgery. Residents completed an Intuitive training module for the da Vinci Xi P8 robot. An in-person training was led by a representative and attending where residents were taught basics of robotic surgery and how to access the simulation trainer and complete modules independently. PGY1-PGY3s completed a post-survey to evaluate changes in competence and confidence in robotic training.

Results
Paired T-tests were performed using pre and post-responses. There was a significant increase in overall satisfaction of gynecologic training opportunities among all residents (p < 0.001). The average satisfaction before intervention increased from 2.58 to 4.36. PGY1s had improved confidence with basic components of robotic surgery, from 1 to 2.66 (p = 0.03). There was no difference in confidence regarding suturing and performing an entire DaVinci surgery. PGY2s had improved confidence with port placement of instruments and docking the camera & instruments at the bedside (both p = 0.01). Average confidence of port placement and docking improved from 2.5 to 4.5. There were no significant increases in PGY3s’ confidence among measured components. Among all residents, confidence increased in components of robotic surgery except for suturing and performing an entire surgery (p < 0.05).

Conclusion
Residents indicated improved satisfaction with the robot curriculum. They demonstrated improved confidence in several robot surgery components, aside from overall confidence in completing a robotic surgery. In future iterations, residents will have online and in-person training during intern orientation, simulator sessions during didactics, and simulator modules correlated to each year.

Improving Ob/Gyn Resident Confidence and Competence in Robotic-Assisted Surgery through Establishing a Robot Curriculum

As part of their holistic care of women, family physicians must provide comprehensive reproductive health services, including education about and provision of contraception. Studies have also shown that contraceptive knowledge base is inconsistent within primary care specialities. While lessons about contraception may be covered in the preclinical years, students often don’t have the chance to integrate and apply such knowledge prior to encountering patients. Studies have shown that interactive approaches and team based learning result in higher satisfaction and equivalent knowledge gain compared to standard lecture styles. Therefore, primary care clinical curriculum should contain engaging content about contraception to solidify students’ knowledge and ensure that they are comfortable discussing options with patients in the future. The aim of this poster is to share an interactive approach to integrating contraception into the clinical curriculum via a jeopardy style session during the clerkship year. 


Utilizing An Interactive Jeopardy Format to Teach About Contraception in the Family Medicine Clerkship

Purpose
Supervision must be documented for every patient encounter during residency training. This is especially indicated when a bill is submitted to a payor for services performed with the involvement of a resident. The teaching physician must include an attestation documenting the level of resident involvement in the patient encounter. When submitted to CMS the GC modifier is also utilized to indicate that a resident, under the direction of a teaching physician, is involved in a patient encounter. We retrospectively reviewed our documentation of the teaching physician in a resident ophthalmology clinic. 

Methods
All patient encounters performed by ophthalmology residents over a 5-month period (January through May 2021) were retrospectively reviewed for appropriate documentation of the teaching physician. All patient encounters were supervised by teaching physicians in an outpatient ambulatory ophthalmology clinic at the Texas Diabetic Institute, San Antonio, TX. Each electronic medical record (EMR) was reviewed for appropriate documentation of the teaching attestation and GC modifier. The data was analyzed using R software. 

Results 
A total of 1,500 patient encounters were reviewed. All records (100%) documented the teaching attestation. The GC modifier was utilized in 1,388 of patient encounters (92.5%). January (95%) and May (88%) showed the highest and lowest compliance, respectively. The GC modifier was not documented in 7.5% of patient encounters.

Conclusion
All patient encounters included appropriate documentation of the teaching attestation. Residents in clinic have direct supervision by the teaching physician and the EMR sends all notes to teaching physicians to be signed and attested. In addition, all notes are reviewed internally for compliance and are amended if not documented appropriately. Compliance with the GC modifier was significantly lower. This is attributed to inadequate understanding of proper usage and that it may not be required by all payors. With recent increase emphasis on the attestation for all notes, providers are less enthusiastic to also ensure documentation of the GC modifier due to this redundancy. Good practices aim for 100% compliance in documentation. Both teaching and resident physicians should be educated on appropriate supervision and how it should be best documented in the medical record.

Compliance with Documenting the Presence of a Teaching Physician in a Resident Ophthalmology Clinic

Detection and Diagnosis of Retinoblastoma: Can Mobile Devices be the Next Step Toward Early Intervention?

Abstract Title: DoubleVision Ophthalmoscope: A novel teaching fundoscope for medical student learners and beyond
Background: Learning to operate a standard fundoscope is an unintuitive process and recognizing retinal pathologies can be a subjective struggle requiring guesswork. While technologies exist to facilitate learning via screen projection from various ophthalmoscopes, a modality for the standard fundoscope, most utilized in medical student training, has not been well established. Our innovation aims to facilitate clearer instruction by physician preceptors while simultaneously lowering the barrier to entry for competent fundoscopy exams. Competent performance of a fundoscopic examination is a required skill per the American Council of Graduate Medical Education and is mandated in moderate and high-level neurologic workups per private and federal insurance agencies. 
Methods
The initial design includes a unique fundoscope based on similar mechanics of standard fundoscopes with a mini camera attachment and connection to monitor screen. The monitor screen will allow for improved visualization for the learner, image saving, and real-time annotation capabilities. 
Results
Through this innovative design we compare the current unintuitive status of fundoscopy training to a novel approach in training. We utilize various criteria in evaluating effectiveness including student/physician interviews and comparisons to other devices in the literature alongside biomedical design processes. We discuss in detail the product value proposition, market feasibility, ethical considerations, and global impact, as these concepts in relation to such device are scarce in the literature.
Conclusion
Clinical studies have emphasized the difficulty in the objectivity of the fundoscopic procedure. Simulations and training models lack fidelity due to high degree of variance between patients, as well as variance between patients and simulations. Residency and intern training of this technique have been difficult, but increased time with hands-on instructor led workshops have reduced this difficulty. There is a need for standardized training protocols across multiple centers, although some clinics may not have access to high end mentorship due to geographic constraints. The remote capabilities of our device would allow education to be performed globally. Prototypes of teaching ophthalmoscopes have been associated with improved confidence and more accuracy in objective measures of pathology identification. 


DoubleVision Ophthalmoscope: A novel teaching fundoscope for medical student learners and beyond

Purpose: To validate the clinical utility of Vision.app in measuring best corrected visual acuity (BCVA) in a pediatric cohort. 
Methods: OD/OS BCVA of pediatric patients was obtained by a) Landolt ring (VAApp) and b) Snellen chart (VAClinic). VAApp was measured by the phone application, Vision.app, from smartphones which used a 4-force choice procedure with a dark blue Landolt C optotype on a black background displayed at 100cm. VAClinic was the smallest line where optotypes were correctly identified with patient sitting 20 ft from an illuminated screen. The two methods were compared using Pearson correlation coefficient and Cronbach α. Vision.app sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to detect significant VA changes (≥0.3) logMAR from in-clinic baseline. A Bland-Altman plot was constructed to visualize the difference between the two methods. 
Results: A total of 49 participants completed the study with data on 98 eyes collected. Mean (SD) age of 10.22 ± 3.28 years with a 26:23 male:female ratio. Mean VA difference was -0.14 (95% CI, -0.193 to -0.087) logMAR or <1 line of difference with upper and lower limits of 0.23 and -0.51 logMAR. The Pearson correlation coefficient was 0.54 [p < 0.001, (95% CI: 0.38, 0.66)]and Cronbach α is 0.68 [p < 0.001, (95% CI: 0.52, 0.78)]. Vision.app had an 80% sensitivity, 73.5% specificity, 35.3% PPV, 95.3% NPV. 
Conclusion: In our study, Vision.app is a valid method of measuring visual acuity in children.

Premium content

Next from AMA Research Challenge 2022

Standardized Patients as Effective as Scrub Technicians at Scrub Training Medical Students

Similar lecture