United States

Introduction. The focus of the STEM Bridges Across Eastern Queens project intervention is the redesign of introductory STEM courses with the goal of increasing retention in STEM of under-represented minority students. A primary outcome is identifying implementation fidelity related to student academic performance. In the absence of treatment fidelity data, it is impossible to know the true cause of both positive and negative outcomes. Further, continued and modified interventions in response to positive outcomes is hindered by lack of fidelity data. According to a 2019 review of the implementation fidelity literature (Hill &amp; Erickson, 2019), there are at least three types of fidelity: structural, process, and dose. Structural fidelity refers to implementation adherence to factors such as staffing levels and frequency of contact, while process fidelity relates to provider-client interactions as well as client to client interactions (O’Donnell, 2008). Dose fidelity specifies the extent to which an intervention program was ultimately available to those who were meant to implement it (Dusenbury et al., 2003). This project primarily focuses on dose fidelity and the extent to which instructors actually implemented the intervention as designed. The study population included undergraduate students at a two-year or four-year college enrolled in treatment or control introductory STEM courses in Biology, Chemistry, Computer Science, Environmental Science, and/or Physics. 

Methods. The analytic sample through fall 19 comprises 6,979 unique students with course GPAs from study sections across an array of treatment- and control-assigned sections each term. Of these, 3802 or 56% were enrolled at the two-year college and 2968 or 44% were enrolled at the four-year college. Each term, lists of eligible sections from included study courses are provided to the project evaluator for random assignment to study conditions. Students self-selected their course sections without knowledge of section assignment. Faculty from included departments actively engaged in course redesign workshops, semester-embedded faculty development opportunities, and professional development supports which have been expanded over time, as per faculty requests. 

Results. Treatment and control section instructors completed implementation fidelity surveys to determine the extent to which each section is offered in compliance with its random assignment. Compliance fidelity was scored as full implementation, partial implementation, or no implementation, and it was noted where data were unavailable. For most course sections, treatment implementation was full or partial and control sections were no implementation or ‘no data’, which were considered as no implementation in analysis. 

Analysis of semester-to-semester retention within study courses shows significantly higher rates for treatment than control students between successive semesters from the fall 2018 to spring 2020 term. Students in treatment-assigned sections scored higher than their control classmates. 

Conclusions and Discussion. While intervention implementation fidelity has been an important consideration for studies in fields such as medicine and mental health treatment, usage in higher education research has been limited. Yet as our study reveals, there are practical advantages in the information they can provide to various higher education stakeholders, including faculty who are planning and teaching effective course enhancements, and administrators who are devising strategic student retention initiatives. The implementation fidelity surveys were developed for faculty to remain accountable to the project’s objectives but eventually to better capture a snapshot of just-in-time teaching during the academic semester and track the changes in teaching practice and mindset after implementation. These surveys provide quantitative data, including compliance scores to better assess redesign activities, and qualitative data such as faculty perspectives on successful curriculum enhancements, what evidence-based teaching methods are being utilized, and how student peer mentors engage with their classmates and instructors through open-ended reflections. Over the course of this project, the results of implementation surveys have evolved into a formative needs-assessment as they communicated increasing requests from faculty participants for opportunities to improve their pedagogy and to collaborate with their colleagues across both campuses to develop and share teaching practices and resources. Figure 1 shows the addition of faculty interventions across the life of the project, all of which have stemmed from faculty requests. 

Acknowledgements. This research was supported by the U.S. Department of Education Grant PO31C160208. We would like to thank the faculty, peer mentors, and students involved with this project. 

References
Borrelli, B. (2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry, 71, 552-563.
Dusenbury, L., Brannigan, R., Falco, M. &amp; Hansen, W.B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research: Theory &amp; Practice, 18 (2), 237-256.
Hill, H.C. &amp; Erickson, A. (2019). Using implementation fidelity to aid in interpreting program impacts: A brief review. Educational Researcher, 48 (9), 590-598.
O’Donnell, C.L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78 (1), 33-84.

13th Conference on Understanding Interventions 2021

Impact of Course Redesign Implementation on Student Success and Ongoing Pedagogical Practices

surveys

implementation

undergraduate

pedagogy

faculty development

The annual UI conference has grown from a meeting to a community of more than 6,000 researchers, program officers, evaluators, program directors, training faculty, and others within STEM, and now operates with a mission to mutually inform scholarship and practice, develop and enhance the community interested in educational interventions, and provide professional development for all STEM persons. UI held its first Virtual conference in December of 2020 in response to Covid-19 restrictions that forced the cancellation of the original March 2020 in person meeting. Born out of necessity, the virtual conference introduced a format that provided pre conference availability of symposia, Deeper Dives and Poster presentations. The 13th Annual Understanding Interventions Conference will continue with that design and we foresee future conferences adopting aspects of that format in reaching a virtual audience as well as those who attend the conferences in-person.

Attendees will experience plenaries, workshops, symposia, and poster presentations on hypothesis-based research on interventions and initiatives that broaden participation in STEM careers. The conference will continue to facilitate the dialogue among education, behavioral, and social science researchers, evaluators, and faculty in STEM fields who participate in intervention programs. Graduate students in these fields are strongly encouraged to attend. Workshops, sessions, and posters will communicate effective strategies from successful STEM intervention programs, present results from empirical research studies, and synthesize the translation of research findings into practice on STEM learning, undergraduate research, graduate education, and student retention.

The cost to attend the conference is $150. There are special rates for post-doctoral fellows and graduate students, as well as conference grants that can be provided for those with need. Registration for the conference is open, with additional information to come.

The annual UI conference has grown from a meeting to a community of more than 6,000 researchers, program officers, evaluators, program directors, training faculty, and others within STEM, and now operates with a mission to mutually inform scholarship and practice, develop and enhance the community interested in educational interventions, and provide professional development for all STEM persons. 

Introduction. The focus of the STEM Bridges Across Eastern Queens project intervention is the redesign of introductory STEM courses with the goal of increasing retention in STEM of under-represented minority students. A primary outcome is identifying implementation fidelity related to student academic performance. In the absence of treatment fidelity data, it is impossible to know the true cause of both positive and negative outcomes. Further, continued and modified interventions in response to positive outcomes is hindered by lack of fidelity data. According to a 2019 review of the implementation fidelity literature (Hill & Erickson, 2019), there are at least three types of fidelity: structural, process, and dose. Structural fidelity refers to implementation adherence to factors such as staffing levels and frequency of contact, while process fidelity relates to provider-client interactions as well as client to client interactions (O’Donnell, 2008). Dose fidelity specifies the extent to which an intervention program was ultimately available to those who were meant to implement it (Dusenbury et al., 2003). This project primarily focuses on dose fidelity and the extent to which instructors actually implemented the intervention as designed. The study population included undergraduate students at a two-year or four-year college enrolled in treatment or control introductory STEM courses in Biology, Chemistry, Computer Science, Environmental Science, and/or Physics. 

Methods. The analytic sample through fall 19 comprises 6,979 unique students with course GPAs from study sections across an array of treatment- and control-assigned sections each term. Of these, 3802 or 56% were enrolled at the two-year college and 2968 or 44% were enrolled at the four-year college. Each term, lists of eligible sections from included study courses are provided to the project evaluator for random assignment to study conditions. Students self-selected their course sections without knowledge of section assignment. Faculty from included departments actively engaged in course redesign workshops, semester-embedded faculty development opportunities, and professional development supports which have been expanded over time, as per faculty requests. 

Results. Treatment and control section instructors completed implementation fidelity surveys to determine the extent to which each section is offered in compliance with its random assignment. Compliance fidelity was scored as full implementation, partial implementation, or no implementation, and it was noted where data were unavailable. For most course sections, treatment implementation was full or partial and control sections were no implementation or ‘no data’, which were considered as no implementation in analysis. 

Analysis of semester-to-semester retention within study courses shows significantly higher rates for treatment than control students between successive semesters from the fall 2018 to spring 2020 term. Students in treatment-assigned sections scored higher than their control classmates. 

Conclusions and Discussion. While intervention implementation fidelity has been an important consideration for studies in fields such as medicine and mental health treatment, usage in higher education research has been limited. Yet as our study reveals, there are practical advantages in the information they can provide to various higher education stakeholders, including faculty who are planning and teaching effective course enhancements, and administrators who are devising strategic student retention initiatives. The implementation fidelity surveys were developed for faculty to remain accountable to the project’s objectives but eventually to better capture a snapshot of just-in-time teaching during the academic semester and track the changes in teaching practice and mindset after implementation. These surveys provide quantitative data, including compliance scores to better assess redesign activities, and qualitative data such as faculty perspectives on successful curriculum enhancements, what evidence-based teaching methods are being utilized, and how student peer mentors engage with their classmates and instructors through open-ended reflections. Over the course of this project, the results of implementation surveys have evolved into a formative needs-assessment as they communicated increasing requests from faculty participants for opportunities to improve their pedagogy and to collaborate with their colleagues across both campuses to develop and share teaching practices and resources. Figure 1 shows the addition of faculty interventions across the life of the project, all of which have stemmed from faculty requests. 

Acknowledgements. This research was supported by the U.S. Department of Education Grant PO31C160208. We would like to thank the faculty, peer mentors, and students involved with this project. 

References
Borrelli, B. (2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry, 71, 552-563.
Dusenbury, L., Brannigan, R., Falco, M. & Hansen, W.B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research: Theory & Practice, 18 (2), 237-256.
Hill, H.C. & Erickson, A. (2019). Using implementation fidelity to aid in interpreting program impacts: A brief review. Educational Researcher, 48 (9), 590-598.
O’Donnell, C.L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78 (1), 33-84.

poster

Interest plays a major role in STEM motivation. Our research is based in introductory biology classes for pre-health majors, where students start out with high levels of interest. The course is challenging, however, and we test interventions (based in expectancy-value theory [Eccles & Wigfield, 2002]) to help underrepresented students perform better. Both competence beliefs and values are critically important, but we intervene on value, and test competence beliefs as moderators. Different outcomes reflect different motivational processes, and we use different measures to study students' performance in gateway science classes and persistence in STEM. We know that these value-focused interventions can promote students' performance and engagement in the targeted biology classes (Harackiewicz et al., 2016). We have now begun to analyze long-term follow-up data from biology students in order to shed light on intervention processes and inform the design of future interventions (Harackiewicz & Priniski, 2018).

Using Long-term Follow-up Data to Understand and Refine Utility-Value Interventions in the Biomedical Sciences

Introduction. The Historically Black Colleges and Universities Undergraduate Program (HBCU-UP), through Targeted Infusion Projects (TIPs), provides support to achieve short-term, well-defined goals to improve undergraduate STEM education. In the current TIP at Xavier University of Louisiana (XULA), two freshman lab courses have been redesigned as Course-based Undergraduate Research Experiences (CUREs). Merits of early research experiences are well documented (Dolan, 2016), yet fewer than 10% of XULA Biology freshmen have research opportunities in their first year. The XULA TIP aims at bridging this gap by infusing two authentic research projects in two foundational labs in Biology. The first, the ‘Yeast ORFan’ project, in collaboration with Juanita College, PA, aims at studying yeast ‘ORFans’ (genes of unknown function). The second, in collaboration with the USDA SRRC, LA, is centered on the phenotypic and genotypic characterization of an Aspergillus (fungal mold) species known to damage food crops. Although CUREs are gaining attention for their ability to offer research opportunities in a classroom setting (Dolan, 2016), few undergraduate minority-serving institutions have established CUREs as part of their curricula. There is, therefore, a great need to study CUREs and their impacts on underrepresented minority (URMs) students’ attitudes towards Biology, learning, retention, and their desire to pursue careers in STEM in general. The XULA CURE infusion project is the first of its kind at an HBCU (XULA) and the results of this intervention/newer way of teaching will enhance our knowledge in the area of science education.
Methods. Two established courses (Biol 1210L and Biol 1220L), meeting once/week for 1 credit hour and required for all Biology majors, were redesigned as CUREs. The student population consisted of ~ 240 students per semester; ~ 85% were URMs and ~ 70% were female. The overarching goal was to introduce students to scientific competencies using interdisciplinary concepts and ‘do-able’, real-life research projects with the above-mentioned yeast and mold as model systems. Data were collected by instructors in the form of student feedback, class participation, assignments, in-class computer work and the application of knowledge in lab experiments and on quizzes. Data were also collected by the external evaluator using assessments such as the Biological Experimental Design Concept Inventory (BEDCI, Deane et al., 2014) and the ‘Science Motivation Questionnaire’ (Glynn et al., 2011).
Results. In year one (Fall 2019-Spring 2020), only the Fall 2019 semester was 100% in person since these were ‘normal’, pre-COVID-19 pandemic conditions and students gained research experience with the yeast ORFan project. Instructors noted early on that, although students enjoyed the flexible format of hands-on lab work combined with data analysis, etc., the weekly two hour session did not provide enough time to teach lab skills and research-related scientific content, allow students to practice, learn troubleshooting, reflect on experimental design, conduct experiments, record results and understand their significance. Nevertheless, the combined hard work of both the CURE faculty and students demonstrated a pattern of improved academic performance on the four common (identical) quizzes. For example, in four sections taught by the same instructor, students in one section consistently underperformed at the start of the semester (on Quizzes 1 and 2), but their scores improved steadily until, by Quiz 4, their scores were indistinguishable from those of the students in other sections (unpublished data). Students also were evaluated on lab skills, experimental work, and group presentations. Lastly, based on the external evaluator’s analysis, students showed gains in four of the five assessed areas of biology motivation (intrinsic motivation, self-efficacy, self-determination, and career motivation, unpublished data). In mid-March, 2020, the threat of COVID-19 caused the University to shut down overnight. Students were forced to leave, and some had no place to live. Many students and faculty lost their loved ones or became infected with the virus. The remaining weeks of the Spring 2020 semester were taught exclusively online; instructors conducted Zoom class meetings while developing a completely new style of teaching and communicating. Fortunately, since the courses were designed to focus on many lab skills early on, the online transition, while not optimal, still allowed for about 75-80% of the projects to be covered. Both year two semesters (Fall 2020-Spring 2021) were taught as ‘hybrid’ courses (at least 50% in person) with extremely strict COVID-19 guidelines. The XULA team developed novel strategies, including preparing in-house videos for the group that was on ‘Zoom’ to study, while the other group was in the lab. At the time that this abstract was submitted (May 2021), year two had just ended. All of our strategies, results, and outcomes will be discussed during the conference workshop or poster session. 
Conclusions and Discussion. Preliminary analysis of year two suggests that while students appreciated the efforts of the CURE team during the COVID-19 pandemic, their morale overall was down. Our students were used to seeing their instructors and advisors in person. While the hybrid model is better than being all online, most students prefer in person learning. Some have confessed that they tend to get lazy and not pay attention since Zoom recordings can be made available. However, things are looking up and XULA plans to move back to all in person instruction. Year one results have demonstrated positive trends in most evaluative areas and year three (Fall 2021-Spring 2022) should provide even more insights. In conclusion, our working hypothesis remains that providing meaningful research experiences early on to all URM Biology majors can positively impact their self-confidence and desire to stay in STEM. Findings from our study will also advance the understanding of this type of intervention and provide a roadmap for other minority-serving institutions who might want to implement it. 
Acknowledgments. 
NSF HBCU-UP TIP Program [Award Number 1912437, P.I. Shubha K. Ireland (SKI)].
The Xavier Eminent Scholar XXXVIII Professorship to SKI (Louisiana Board of Regents supported).
Dr. Jill Keeney, Biology, Juniata College, PA 16652 (collaborator/guide, the yeast ORFan project).
Dr. Jeff Cary and scientists at the USDA, SRRC, LA 70124 (collaborators/guides, the Aspergillus project).
References. 
Deane, T., Nomme, K., Jeffery, E., Pollock, C. & Birol, G. (2014). Development of the Biological Experimental Design Concept Inventory (BEDCI), CBE-Life Sciences Education, 13, 540-551. doi: 10.1187/cbe.13-11-0218 
Dolan, E. L. (2016). Course-based Undergraduate Research Experiences: Current Knowledge and Future Directions. Paper commissioned for the Committee on Strengthening Research Experiences for Undergraduate STEM Students. Board on Science Education, Division of Behavioral and Social Sciences and Education. Board on Life Sciences, Division of Earth and Life Studies. 
Glynn, S. M., Brickman, P., Armstrong, N. & Taasoobshirazi, G. (2011). Science Motivation Questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159-1176. 


Research Experiences in Classroom and their Impacts on Biology Freshmen at an HBCU 



The Continuing Umbrella of Research Experience (CURE) is a National Cancer Institute funded research training and career-development initiative that focuses on sustaining the pipeline of students from underrepresented groups in the biomedical sciences. The UMB CURE Program is a groundbreaking year-round pipeline program that prepares sixth- to 12th-grade students in West Baltimore for competitive and rewarding research, STEM, and health care career opportunities. 

UMB CURE provides holistic services to scholars and families to ensure their success. The program incorporates hands-on STEM activities and STEM curriculum to further develop scholar interest in STEM fields and careers. Mentorship, socio-emotional support, resource referrals, and academic support consistute parts of the holistic services that UMB CURE provides to scholars and families. 

Our inaugural cohort is moving into their senior year in Fall 2021.

Enhancing the STEM and Health Care Pipeline

Introduction: The lack of diversity among faculty at universities and medical schools in the United States is a matter of growing concern. At universities in the US, underrepresented minority (URM) researchers make up a mere 3.5% of the faculty in the life sciences and just 6.3% of the faculty in basic science departments at medical schools (Lambert et al., 2020). Women also remain underrepresented, accounting for only 39% of the faculty in the life sciences at universities and 35% of the faculty in basic science departments at medical schools (Association of American Medical Colleges, 2017; National Science Foundation, 2019). We hypothesize that there are a significant number of postdoctoral researchers with aspirations of becoming faculty members in academia, but many seek nonacademic careers due to uncharacterized influences. 

Methods: Based on psycho-social theories for career development and motivation, we developed and conducted a survey to examine the factors that influence commitment to or divergence from conventional academic research careers with 1284 biomedical postdocs across the United States. To further elucidate the benefits, challenges, and strategies for pursuing an academic career, 994 responses were qualitatively analyzed using both content and thematic analyses. 177 unique codes, 20 categories, and 10 subthemes emerged from the data and were generalized into two thematic areas: Life in Academia and Strategies for Success.

Results: We reveal four main outcomes that were largely undefined in the field. (1) The first two years of postdoctoral training are pivotal in the retention of women and URM postdocs in the academic research pipeline. (2) The likelihood of biomedical postdocs choosing an academic research career increases as financial security, mentorship from their PI and their sense of self-worth increase. (3) Unlike their colleagues, many URM postdocs do not expect to associate with the people that they value most in a research-intensive faculty position. (4) We also find that female postdocs reported lower research self-efficacy (belief in research abilities) and lower outcome expectations (belief in the outcomes of research efforts) than men, which also significantly predicts first-author publication rates. 

Conclusions and Discussion: Multiple factors are responsible for URM and female postdocs not choosing to pursue careers in academic research, including the low expectation of finding people of similar background or like-values in academia. For women, lower self-efficacy and outcome expectations compared to their male counterparts were strong predictors for choosing non-academic careers. Interventions addressing such factors in the early stages of postdoctoral training may prevent attrition of underrepresented groups. For advisors and institutions, this work provides a unique perspective from postdoctoral scholars on elements of the academic training path that can be improved to increase retention, career satisfaction, and preparation for the scientific workforce.

Acknowledgements: We acknowledge the National Center for Advancing Translational Sciences (Award Number UL1TR002384) for support.

Factors that affect postdoctoral trainee persistence in academia

Introduction

The Biology Scholars Program ("BSP") is an equity and inclusion program at UC Berkeley that aims to support the success of historically underrepresented undergraduates in STEM. BSP has documented student success for over 28 years and continues to cultivate student growth and inclusivity at UC Berkeley for PEER and first-generation science students (Estrada et al., 2019). While UC Berkeley is highly renowned for its prestigious and rigorous academic programs, there still exists institutional barriers that disproportionately negatively affect historically underrepresented students' persistence (Ballen & Mason, 2017). We therefore sought to understand the impact and influence of the Biology Scholars Program on UC Berkeley. In particular, we wanted to understand its influence on members of the campus community (i.e., faculty, staff, and administrators) and its effect on the larger institution. 

Methods

Data were collected via a brief one-time online survey in fall of 2020, which was distributed to participants using Qualtrics, a survey website. Participants were selected based on past attendance to BSP workshops and conferences between 2015 through 2020. Of 172 participants, 38% were staff, 35% faculty, 14% graduate students, 4% administrators and 9% "others" (i.e. alumni, postdocs, program directors). Over half of the sample self-reported as white (57%), hispanic/latinx (12%), black/african american (9%), filipino (3%), asian (3%), chinese (3%), vietnamese (2%), asian indian (2%), japanese (2%), american indian (1%), korean (1%), and declined to state (5%). Respondents self-identified as female (54%), male (34%), transgender (1%), nonbinary (2%), declined to state (1%), and did not answer (8%). The majority of respondents were between 30 - 50 years old. 

Results

The results from the one-time survey provides robust descriptive data to increase understanding of how BSP has impacted UC Berkeley. The first set of questions assessed participants' familiarity with BSP. The data show that over half of participants had experience interacting with members of the BSP community (e.g., BSP faculty and staff [68%] or BSP students [83%]). A subset of questions was dedicated to understanding participants' knowledge of BSP. In terms of correct responses, participants scored relatively high on 5 sections: selection of students (86%), treatment of students (73%), goals of BSP (85%), who participates in the program (65%) and criteria for student success (85%). These data show that participants understand how BSP functions or "works." With respect to BSP's influence on UC Berkeley members, the category most frequently selected by faculty, staff, and administrators was "mentoring" in relation to 1) influence in their own work (28%) and 2) student persistence in STEM (16%). Importantly, 81% of participants agreed that BSP has influenced the minds of UC Berkeley faculty, staff and administrations about growing student talent instead of selecting for it; a key principle of BSP. 

The Biology Scholars Program was also reported to provide a model of how to create a safe and positive environment for UC Berkeley students. In fact, 97% of participants agreed that BSP's scholars created positive "ripple" effects into UC Berkeley and 90% agreed that BSP's Inclusive Excellence Communities of Practice also created positive "ripple" effects into UC Berkeley. A number of questions were also asked about BSP's influence. Seventy-seven percent of participants reported that BSP's presence on campus changed UC Berkeley. More to this point, the program was also reported to have positively influenced perceptions of equity (75%), inclusion (81%) and bias reduction (77%). Lastly, an open-ended question asked participants to describe BSP's greatest impact on UC Berkeley and the top two most frequent descriptions were modeling how to be supportive (19%) and creating a sense of belonging or community (17%) for students.

Conclusions and Discussion

The results of this study provide insight into the experiences and perceptions of faculty, staff and administrators with BSP and its impact on UC Berkeley. Overall, results showed that the Biology Scholars Program has had a positive effect on the members of the UC Berkeley community by modelling successful mentorship, and creation of contexts that provide support, inclusivity and equity for students who develop (rather than arrive with) the skills and preparation necessary to thrive in scientific disciplines. On a larger scale, BSP has had a "ripple" like effect on UC Berkeley, influencing not only individual members of the university, but other programs on campus, that seek to adapt the BSP model to other disciplines. How these "ripples" might extend to other universities is a topic of interest and future research. 

Acknowledgments

The present research was supported by a grant from Howard Hughes Medical Institute ("HHMI") (Internal ID: 25087). We thank Missy Soto and Brook Yu for their contributions to this research.

Understanding the Biology Scholars Program's Impact on UC Berkeley

Introduction. Effective mentoring through research experiences supports student’s learning in science courses and interest in science careers. It also impacts academic performance and success by increasing independent learning (Lopatto, D. (2007). Research improves undergraduate student attitudes towards science including self-reported curiosity, science identity, confidence, it also increases communication skills, creativity and knowledge of scientific content (Laursen, S. et al, 2010). Faculty interactions in this context are particularly relevant, as these promote self-efficacy, scientific identity, and scholarly productivity (Joshi, M., Aikens, M. L., & Dolan, E. L., 2019) While the benefits of undergraduate research are well documented, the National Survey of Student Engagement (NSSE) reports that only 19% of students engage in research activities during their undergraduate education, a number that is even smaller for those attending 2-year colleges. 

Students from backgrounds underrepresented (URM) in STEM fields show larger gains than other students when engaging in research experiences (Lopatto, D. (2007). The COVID-19 pandemic further impacted the availability of research experiences for undergraduates, and data shows that this has particularly impacted URM and first generation students (Barber, P. H. et al, 2021). Therefore, the development of programs that engage URM students in remote research experiences is a priority. Here we describe an intervention that engages 2- and 4-year college students in biology-related research and present a preliminary assessment of their experiences.


Goal of Research: In an effort to support research experiences for undergraduates at 2- and 4-year colleges and to increase opportunities to engage community college students in research, an interdisciplinary undergraduate research experience was developed by two professors at the City University of New York. This pilot intervention had the goal of providing STEM undergraduates from mostly URM backgrounds with research opportunities during the 2020 distance modality period of instruction due to the COVID-19 pandemic.


Methods. Students from a 2-year and a 4-year college that had expressed interest in participating in research were recruited at both schools. A total of 15 undergraduates worked on research projects during the spring and summer semesters of 2020. The experience was designed as an interdisciplinary collaborative research project in which students of both schools worked together. The students and professors met periodically via Zoom to participate in theoretical lectures, experimental analysis of data, peer reviewed paper discussions, and hands-on activities. The students also scheduled meetings with their research partners independently of the planned overall group meetings, students were asked to organize these considering the progress and needs of their project, and their time availability. The preliminary findings of the research were presented in shared meetings, and also students submitted abstracts to local and national conferences. 

The assessment of the student experiences was performed qualitatively by the use of a 9-question open survey that evaluated the implementation of the intervention and identified strengths and areas in need of improvement. The survey was voluntary and anonymous. 12 of the participants provided responses. It is important to highlight that there are limited qualitative studies discussing close mentoring research engagement of URM students in STEM. In order to achieve depth and intersubjectivity in the data analysis, the analysis was carried out in various temporal instances. In the first stage, the data corpus was coded and analyzed based on its content using grounded theory. In a second stage, units of meaning were created, the research team categorized, filtered and reorganized based on the research objectives. In a third moment, and after the systematic reading of the codes and their context, a selection was made of the categories that specifically accounted for students' experience during the research intervention. 


Results. A total of 15 students participated in this intervention, 14 of which are URM in STEM (either African American or Hispanic). The group had 9 (60%) females students and 6 (40%) male students. The students worked in pairs with specific research projects, and met independently and with the overall group throughout the intervention period with similar frequency. As the intervention progressed, student-only research meetings were more common than whole group meetings. A total of 12 students participated in local and/or national conferences during the summer and within the 6-month period following the intervention. 

In the qualitative assessment, the students highlighted a (1)new understanding of the research process in which they understood the times and movements of the process, a (2)development of personal, academic and professional skills, and an (3)increase in their confidence. The latter resulted from their engagement in activities of different kinds. The students also expressed the importance of having a community during the COVID-19 pandemic, they expressed a (4)higher sense of belonging that prevented isolation as a result of participating in the intervention.Students expressed satisfaction with their ability to manage several projects at the same time, with limited free time and working through weekends. They also provided suggestions for future interventions, in areas where they considered there could be improvement, which included the availability of financial support and the planning of more frequent sessions. 


Conclusions and Discussions. Our intervention adds to other available evidence that indicates remote undergraduate experiences are possible and can support learning for undergraduate students (Tripepi, M., & Landberg, T. 2021; Jensen-Ryan, D. et al, 2021). This work also shows that interventions of this kind can promote remote interactions between students and faculty, as well as among students impacting positively the college experience for undergraduates. We found that students build on skills beyond the realm of the academic and professional world, with gains in personal development as well. We also think it will be important to more critically explore the impact of the overwhelming and hyperproductive academic lifestyle and how this model, which instructors share and students reproduce, can be damaging to long-term sustainable interaction with research and learning (Enciso Dominguez, G. et al 2021). Because URM STEM students already faced multiple structural barriers, to provide successful models of success requires us to further explore this dynamic to ultimately enhance interventions at the undergraduate level that aim to increase their numbers in these fields.

Supporting STEM students through remote interdisciplinary collaboration in undergraduate research

The Syracuse University CHANcE (Collaborative High Impact Activities in Natrual sCience Education) Project is a five-year program designed to provide the professional support biology and chemistry faculty need to transform their courses to include high impact practices such as active learning and inclusive pedagogy. Participation in the Project is open to all faculty in the 2 departments including tenure-earning and teaching faculty. The Project is in its third year and most of the faculty in the 2 departments are participating. To compensate faculty participants for the additional work involved in course transformation, a $5,000 grant is made available to each instructor who submits a course transformation proposal that is approved, who agrees to implement the changes specified in the proposal, and submits a final report including assessment of the implementation. Professional support is provided during both the redesign and implementation phases to help instructors develop specific learning goals, introduce specific active-learning and inclusive practices for accomplishing these goals, and assess learning outcomes. Professional assistance includes regular meetings with a CHANcE Project Education Specialist, postdoctoral fellow, graduate research assistants, peer mentors, administrative personnel, communities of practice, and semi-annual two-day professional development workshops held in January and June of each year. 
To date, over 12 courses have been or are in the process of being transformed, including general chemistry, introductory biology, ecology and evolution, cell biology, anatomy and physiology, organic chemistry laboratory, inorganic laboratory techniques, and integrative biology laboratory. Additionally, the upper-division courses “perspective in biochemistry,” “the science of GMOs,” and “personalized medicine” are being transformed. High impact practices introduced thus far in these courses include writing-to-learn, case studies, messy problems, clicker questions, interactive video lectures, transparent assignments, peer learning assistants, and assignments to highlight the contributions of BIPOC scientists. These transformed courses affect over 1000 students annually, and thereby increase our capacity for inclusion.
The faculty response to the CHANcE Project has been exceptional. The level of participation indicates that faculty care, and they stepped up. However, we found that inclusion is hard to grasp and implement because of the normativity of dominant culture. The practice of inclusion is known to touch on values, emotions, and long-held beliefs about who scientists are and who belongs. In addition, overcoming built-in thoughts about group affiliations is a difficult, nonlinear process that is not considered in faculty load. For students who have been historically excluded because of ethnicity or race, the challenge of having to develop a sense of belonging without sacrificing their cultural identity or adopting the oppressive views of the dominant group is daunting. We are able to address many of these difficult challenges in faculty thinking and practice through the semi-annual 2-day professional development workshops.


How to transform course in biology adn chemistry using high impact practices

With support from the NSF Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM), the Culturally Adaptive Pathway to Success (CAPS) program aims to build an inclusive pathway to accelerate graduation for academically talented, low-income students in Engineering and Computer Science majors at Cal State LA. The campus context primarily serves underrepresented and economically disadvantaged students in the Los Angeles area. Within the College of Engineering, Computer Science, and Technology, the average graduation rates of first-time freshman (since 2007) are: 4 years: 3.5%, 5 years: 18.5%, and 6 years: 29.4%. The CAPS program seeks to improve on these levels of student achievement by scaffolding student development with multiple supports. 

The CAPS program aims to develop social and career competence in our students via three integrated interventions: (1) Mentor+, an advising strategy that trains mentors to engage with students in relation to their academic work, and the connections between work and community. (2) Second CAPS established peer cohorts, providing a structure for social support among students and training models of peer mentorship (3) Finally, CAPS structures professional development from faculty who have been trained to support students with a holistic understanding of the antecedents of college success. 

The success of the CAPS program will ultimately be determined by the improved graduation rates of the involved scholars. Preliminary results suggest that CAPS scholars are progressing faster than non-CAPS students academically. As of Spring 2021, 86% of scholars were retained. 70% of the first cohort graduated in 4 years, and more than half of the second cohort are expected to do the same next year. Further, 100% of the retained scholars are on pace to graduate in 5 years. This is compared to average graduation rates of <4% achieving 4-year graduation and <19% achieving 5-year graduation, among comparable students in the college. The results of the study suggest that others looking to support low income underrepresented students could benefit from pairing financial support with mentors trained to consider the experiences of economically and racially diverse student populations, as well as providing infrastructure for the development of peer mentorship groups.

Two years of mentorship in Culturally Adaptive Pathways to Success

Historically, ethnic minority scholars (African Americans, Latinx, American Indians/Alaskan Natives) have been underrepresented in Science, Technology, Engineering, and Mathematics (STEM) fields (NSF, 2017). With historically underrepresented (HU) groups projected to become nearly 40% of the population by 2050, it is imperative that the STEM disciplines increase their diversity (NACME, 2014). A collateral benefit often attributed to diversity is greater creativity, with more divergent groups achieving greater amounts of creativity (Paulus, van der Zee, & Kenworthy, 2017; McLeod, Lobel, & Cox ,1996). Lack of heterogeneity in STEM education and the workforce is thought to result in missed opportunities, missed perspectives, and waste of intellectual potential (Valantine & Collins, 2015). Demographic, cognitive, and gender diversity have all been signaled as sources of strength and ideas in team settings. However, there are few studies exploring what types of team diversity actually lead to greater creativity in multidisciplinary settings. 
We designed this study to explore what type of team diversity results in greater creativity outcomes within a team-based, project-based Bioinspired Design course for undergraduate students. The Bioinspired Design course aims to remove artificially created disciplinary boundaries to extend beyond STEM by being open to undergraduates from any major, year, or disciplinary background. This study included 275 students, spanning over two years of the course, who were 60.7% female, 38.5% male, and 0.8% non-binary/other; ranged from 18 to 26 years of age, and 13.5% were HU student. Due to the unique student population and pedagogical approach of this course, we developed an integrative creativity measure that assessed the fluidity, elaboration, originality, and innovation of the bioinspired products created by the teams in the course and yielded good validity, α= 0.82. Lastly, we coded the 72 teams for different types of diversity measures that included educational status, major, ethnicity, and HU status.
Correlational data between our measures of diversity and creativity show the more diversity of educational status in the team the higher the innovation (r=0.12, p<.05) and overall creativity (r=0.17, p<.01). The data also show the higher the diversity in majors within the group, the higher scores on elaboration (r=0.12, p<.05), originality (r=.019, p<.01), innovation (r=0.18, p<.01), and overall creativity (r=0.23, p<.01). Results further affirmed that the more diverse the group in terms of ethnicity, the higher the scores in all domains of creativity, fluency (r=0.23, p<.01), elaboration (r=0.23, p<.01), originality (r=0.11, p<.05), innovation (r=0.16, p<.01), and overall creativity (r=28., p<.01). 
Our findings show correlational effects on the creativity of student output within an interdisciplinary undergraduate STEM course and diversity in terms of educational status, major, and ethnicity. Specifically, when there is higher interdisciplinary cooperation and ethnic diversity in a team, there is greater output of original ideas, innovation, and overall creativity. The results suggest that many types of diversity in scientific teams can lead to more innovative discoveries, more focus on real world issues, and supports the strength of interdisciplinary collaboration.


Effect of Team Diversity on Creativity of Bioinspired Design Inventions

We employed two approaches using regression analyses to examine the impact of the BUilding Infrastructure Leading to Diversity (BUILD) Scholars Program on first-year students at four BUILD sites on self-efficacy with respect to research. Findings indicate that first-year participation in the BUILD Scholar program significantly increases researcher self-efficacy.

Impact of Course Redesign Implementation on Student Success and Ongoing Pedagogical Practices

Downloads

Next from 13th Conference on Understanding Interventions 2021

Using Long-term Follow-up Data to Understand and Refine Utility-Value Interventions in the Biomedical Sciences

Similar lecture

How to transform course in biology adn chemistry using high impact practices

Impact of Course Redesign Implementation on Student Success and Ongoing Pedagogical Practices

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from 13th Conference on Understanding Interventions 2021

Using Long-term Follow-up Data to Understand and Refine Utility-Value Interventions in the Biomedical Sciences

Similar lecture

How to transform course in biology adn chemistry using high impact practices

Downloads