United States

The rush to online course delivery in response to COVID-19 revealed limitations of commercially available online virtual labs. Existing virtual lab experiences focus on content delivery and generally poorly support student exploration. As well, the effectiveness of virtual labs is not well understood, particularly for specific student populations including Hispanic/Latinx students. To address these issues, educators in the California State University system designed three virtual labs that focus on high impact scientific and mathematical processes, relate to real life events brought about by the pandemic, and are culturally responsive and trauma-informed. This poster shares initial results from educational research focused on student experiences in the new upper-division physics virtual lab. The primary methodology for this research is user experience (UX) testing, an interview-based method in which users (students) are asked to talk aloud as they complete tasks (virtual course/lab). Led by a student researcher, UX testing was selected to empower students to have a safe space to share their experiences with another peer to gather feedback to improve/continue virtual course/lab structure. UX testing is effective with small numbers of participants; five users is ideal for discovering problems in an existing system because the probability of users encountering a usability issue is 31% according to an average problem frequency and can find 85% of the problems (Sauro 2012). In this study, three of five students interviewed identify as Hispanic/Latinx. UX testing showed a 100% success rate and provided insights connected to course/lab resources, organization, and future approaches to virtual labs. Findings from pre/post surveys on student self-efficacy (Chen, Gully &amp; Eden 2001) and sense of belonging (Floyd-Smith, et al, 2010) are also shared. Student reflections on the method of UX testing show that the research method, itself, contributes to student sense of belonging.

PERC SM 21

Reimagined Virtual Upper-Division Physics Laboratory Experiences in Response to the COVID-19 Pandemic: Findings from User Experience Testing

user-study

trauma-informed

hispanic-serving-institutions

student-support

inclusive

culturally-responsive

user-testing

virtual-labs

user-experience

education-design

virtual-education

pandemic

covid-19

Join AAPT and Physics Educators worldwide in developing and strengthening your professional connections by attending the 2021 AAPT Virtual Summer Meeting. AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques.

Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. Attending the virtual summer meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.

Please log in with your AAPT account or register for the event if you haven't already.

If you are registered please note that you need to be logged in with your AAPT email / credentials in order to get access.

AAPT SM 21

The rush to online course delivery in response to COVID-19 revealed limitations of commercially available online virtual labs. Existing virtual lab experiences focus on content delivery and generally poorly support student exploration. As well, the effectiveness of virtual labs is not well understood, particularly for specific student populations including Hispanic/Latinx students. To address these issues, educators in the California State University system designed three virtual labs that focus on high impact scientific and mathematical processes, relate to real life events brought about by the pandemic, and are culturally responsive and trauma-informed. This poster shares initial results from educational research focused on student experiences in the new upper-division physics virtual lab. The primary methodology for this research is user experience (UX) testing, an interview-based method in which users (students) are asked to talk aloud as they complete tasks (virtual course/lab). Led by a student researcher, UX testing was selected to empower students to have a safe space to share their experiences with another peer to gather feedback to improve/continue virtual course/lab structure. UX testing is effective with small numbers of participants; five users is ideal for discovering problems in an existing system because the probability of users encountering a usability issue is 31% according to an average problem frequency and can find 85% of the problems (Sauro 2012). In this study, three of five students interviewed identify as Hispanic/Latinx. UX testing showed a 100% success rate and provided insights connected to course/lab resources, organization, and future approaches to virtual labs. Findings from pre/post surveys on student self-efficacy (Chen, Gully & Eden 2001) and sense of belonging (Floyd-Smith, et al, 2010) are also shared. Student reflections on the method of UX testing show that the research method, itself, contributes to student sense of belonging.

poster

**Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics**

As a gateway discipline to STEM majors and careers, physics plays a central role in deciding who does, and who doesn't, get to discover and pursue scientific careers. Technical and scientific professionals are rewarded with economic privilege and status which affords them both social and cultural power. Therefore, social justice will be realized only when the opportunity, and the choice, to excel in physics is open to all students. The culture and practices of the physics classroom reflect the culture of physics at large, which raises the question of how our professional physics communities can embrace, rather than inadvertently repel, a diverse group of academics that can help improve physics culture.  To this end, the 2021 Physics Education Research Conference will focus on defining inclusive spaces and practices. Additionally, we will explore strategies for eliminating exclusionary practices in our classrooms, research spaces, and professional organizations. We call on the community to propose sessions that will drive conversations related to these initiatives throughout PERC 2021.

Please download the full program [here. ](https://drive.google.com/file/d/1TRMC_SYJ3PUqXyP7sskA7LRSA8ow0eRE/view?usp=sharing)

August 4, 2021 - August 5, 2021 in a Virtual Conference Format

Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics

Using analysis of variance on a sample consisting of 1,499 US students across 21 US PhD programs, we show that there is no significant difference in the time it takes US male and female physics PhD students to complete their degree programs. This result comes in spite of a statistically significant 18 percentile point gap in median GRE-P scores between genders. Additional analyses reveal that there is no statistical difference between US students reported as White, Black/Hispanic/Multiracial/Native American, and Asian. Expanding our sample to also include 1,143 Non-US students, we find a small but significant effect of citizenship status on time to PhD completion where the average time for Non-US students to complete a physics PhD is about two months less than their US student counterparts. These results show that in spite of known gaps in standardized admissions exams between genders, these differences are not reflected in subsequent graduate school performance. Our findings reinforce the need for graduate admissions committees to go beyond quantitative metrics and conduct a holistic assessment of an applicant's potential to perform research effectively and to earn a PhD.

Time to PhD completion is no different between men and women despite score gap on physics GRE

Postsecondary STEM instruction often does not meet the diverse needs, abilities, and interests of postsecondary STEM learners. The Universal Design for Learning (UDL) framework provides ideas for designing instructional environments to support learner variation. However, existing tools do not meet the professional development needs of many postsecondary STEM instructors, or the research needs of some discipline-based education researchers. Enacting and assessing UDL in postsecondary STEM requires expertise form three stakeholder groups: postsecondary STEM instructors, experts in disability and UDL, and discipline-based education researchers. Building on prior work with UDL in postsecondary STEM, our multi-disciplinary team developed the Universal Design for Learning Instructional Practice Observation Protocol (UDL-IPOP). The UDL-IPOP has one-to-one alignment with the finest-grain descriptors (i.e., checkpoints) in the UDL framework as well as exemplar practices to explicate UDL-aligned instructional practices for postsecondary STEM. After drafting an initial version of the UDL-IPOP, we discussed it with representatives from the three relevant stakeholder groups. Here, we describe practices that aligned with the UDL framework. Then, we discuss initial impressions about which practices postsecondary STEM instructors likely already implement and ideas from disability experts about how STEM instructors could deepen their UDL practice. Overall, we find that postsecondary STEM instructors and disability experts focused on different types of learner variation. We suggest that collaborating with disability experts could provide a necessary “lens change” to better support variation in postsecondary STEM learners’ needs, abilities, and interests.

Explicating Universal Design for Learning-aligned instructional practices for postsecondary STEM

In the spring and summer of 2020, we collected data on faculty experiences during their transition to onlineteaching as a result of the COVID-19 pandemic. Data on the participants’ institutions, job security and position,and preparation time were collected, as well as a free text response to add anything the participants felt wasrelevant. In total 364 text responses were collected. Using natural language processing tools, sentiment scoreswere calculated for each response. The sentiment was found to be overall positive. Then, a machine learningmodel was created using Keras, which was trained on various data for 75% of the responses. The remaining25% were used for predicting sentiment scores, to identify which data from the survey, if any, were potentialpredictors of participant sentiment. The score predictions were used to determine if any data on participantsâinstitutions, positions, or transition time were correlated with positive or negative experiences.

Predictors of faculty sentiment on their transition to online teaching

Due to the growing concerns surrounding the COVID-19 pandemic, colleges and universities either canceled or remotely hosted their 2020 National Science Foundation Research Experience for Undergraduates (REU) programs. This study is part of a larger study examining the impact of these fully remote experiences on professional and psychosocial factors such as mentees’ sense of belonging, identity, and self-efficacy and their retention in STEM degree programs. We present a single-student case study and describe the dramaturgical analysis which centers on identifying five fundamental constructs within the data: objectives, conflicts, tactics, attitudes, and emotions. These items investigate what the participant in the remote REU program experienced, and how this experience changed the ways in which he thinks about his future career decision-making. Our analysis explored four different themes: Lack of community, mentor support, real REU experience in a virtual format, and future career decision-making. The mentee reported that this experience was highly beneficial and that he developed a sense of belonging and identity, despite working remotely – often from his own bedroom.

A case study approach to understanding a remote undergraduate research program

COVID-19 has disrupted nearly all aspects of global society, with severe impacts on many people around the world. In 2020, supported by [funding source], we began a study to document and understand how COVID-19 has affected graduate admissions in physics, with a focus on the ways in which students and departments have been impacted. In this Work in Progress paper, we report on the first year of this study, which collected data from the 2019-2020 admissions cycle. We collected survey data from Directors of Graduate Study (DGS) and from students who applied to physics graduate school in this cycle (including students who did and did not begin graduate school in 2020), as well as conducted follow-up interviews with students and faculty to better understand the breadth of experiences of individuals affected. Four themes emerged from this first year of data: shifts in international/domestic representation, program resilience as a key distinguishing feature during COVID, the breadth of student experiences in 2020, and concerns about the expectations for the future.

Tracking Graduate Admissions During COVID-19: Year One

This project aims to understand physics faculty responses to transitioning to online teaching during the COVID-19 pandemic. We surveyed 662 physics faculty following the spring 2020 term, and 258 completed a follow up survey after the fall 2020 term. We used natural language processing to measure the change in sentiment scores of one hundred physics faculty from the United States that completed an optional written prompt on both the initial and follow up surveys. These sentiment scores measured between -1 and 1 for completely negative and completely positive respectively. Sentiment scores after spring 2020 were slightly positive with a median value of 0.2347. The distribution of sentiment changes was approximately normally distributed with a mean centered near zero. Analysis suggests the average sentiment did not change from the initial to followup surveys.

Sentiment analysis of faculty responses; transition to online learning

In spring 2020, among the many significant challenges raised by the COVID-19 pandemic, many educators had to face the most arduous task: to convert well-designed in-person instruction methods to online teaching and to teach synchronously using hybrid or HyFlex models or completely asynchronously. The above teaching methods require instructor-student communication that allows the instructor to adapt his/her teaching material and teaching style to best fit their audience. However, during the COVID-19 pandemic, while some students seem to have acclimated to online teaching, many do not seem to be engaging in the virtual classroom. In the worst-case scenario, some students may not be present at all, while others may be present but not participating. With minimal immediate feedback from students, we believe that access to some background information - such as learning styles, goals, motivations, and expectations of the potential audience - will help instructors improve their online teaching methods and increase classroom comprehension. In the process, we collected data from STEM introductory-level courses in the form of student feedback surveys and personal interviews conducted online. We then used persona methodology and ethnography to document the diverse learning group in a STEM classroom. Personas are “life-like models, whose characteristics are driven by the various goals and motivations of real or potential users." We used these personas to learn how individuals’ learning experiences affect their participation, goals, motivations, and expectations during online learning. These personas can support the design of online courses with student and instructor-centered approaches that better engage undergraduate students across multiple disciplines.

Personas of STEM students completing online instructions during the COVID-19 pandemic

State regulations in response to the coronavirus pandemic impact upon the state of Arkansas permitted a full year’s worth of hybrid instruction at the University of Central Arkansas during the 2020-2021 academic year. Changes to instruction included the permission of an in-person lab, subject to social distancing, reduced classroom capacity, and other health protocols. However, these protocols induced the removal of a research-based problem-solving exercise previously available at the beginning of lab sections. Before and during this time, data for the Attitudes and Approaches to Physics Problem Solving (AAPS) survey was collected over the course of five-semesters’ worth of introductory algebra-based physics students: two semesters of normal in-person instruction (Spring 2019, 1st-semester physics, and Fall 2019, 2nd-semester physics); one semester in which normal instruction was interrupted by pandemic emergencies mid-semester, forcing fully online instruction for the remainder (Spring 2020, 1st-semester physics); and two semesters in which a hybrid format for lecture and adjusted in-person laboratory section were permitted (Fall 2020, 2nd-semester physics, and Spring 2021, 1st-semester physics). This permits comparison of first-semester and second-semester students between the pre-pandemic traditional instruction and the pandemic-induced hybrid instruction. Students’ attitudes towards physics problem solving is checked for correlation with the presence or lack thereof of a pre-lab problem solving exercise, in the institutional setting of a regional comprehensive state university in the Southern US. A feedback survey from students who took the hybrid course is also examined for perceived relevance as well as student beliefs as to whether or not the problem solving exercise would have helped if it could have been included in the laboratory sections.

Attitudes toward Physics Problem Solving: In-Person vs. Hybrid Instruction”

The move to online teaching necessitated by the COVID-19 pandemic has had profound impacts on university teaching and learning. Learning Assistants (LAs), undergraduates who operate as both instructors and students, were doubly impacted by the transition. In the Spring of 2020, LAs were critical in maintaining a collaborative and inclusive learning environment during the sudden transition to online learning. To examine how LAs were affected by the transition, we conducted an exploratory case study of experienced LAs at [Redacted] University who were teaching in a flipped, introductory-level physics course. We explore the impact on them both as students and instructors. Through an analysis of semi-structured interviews, several common themes emerged. First, the loss of structure in the environment led to increased structure in other aspects of the LAs' life. Further, increased isolation led to a decrease in interaction between the teaching staff inside and outside of the classroom. As a result, LAs have had to develop new classroom management techniques to compensate for the changed environment. At the same time, the resources available to students have changed, often compounding existing challenges faced by the LAs. In this work, we will discuss the strategies developed by two experienced LAs to deal with the challenges of transitioning to an online environment. We hope this will enable further studies focusing on the impact of online teaching on physics identity building in LAs.

Undergraduate learning assistants and the transition to online learning

The sudden transition to remote instruction left many introductory physics students with limited opportunities for meaningful interpersonal interactions with their classmates and instructors. Students reported discouragement over their ability to learn new concepts, and lack of motivation to engage in the courses. In response, we formed “Learning Pods” - online learning communities of small student groups connected through Slack and mentored in collaboration skills. Summer of 2020, Learning Pods became an organizing structure of the remotely delivered introductory calculus-based mechanics course. In subsequent terms, features of the Learning Pod Intervention varied as we scaled up from courses with ~100 students to those with 300-500 students. Based on analysis of student survey questions, we present preliminary indicators of increased student self-efficacy and improved engagement with instructors and peers. Improved self-efficacy and engagement have been correlated with improved success and persistence, especially for students from groups underrepresented in physics. We observe that explicitly incorporating collaboration as instructional material and setting up structures to facilitate communication increased student self-efficacy and interactivity. In addition, TA facilitation of synchronous activities to foster effective collaboration lowered barriers for students reaching out to them. Overall, instructor and TA efforts to increase effective collaboration and communication had a positive impact on student self-efficacy and engagement, and the more structured and explicit interventions had more impact.

Premium content

Reimagined Virtual Upper-Division Physics Laboratory Experiences in Response to the COVID-19 Pandemic: Findings from User Experience Testing

Next from PERC SM 21

Time to PhD completion is no different between men and women despite score gap on physics GRE

Similar lecture

Students' Differential Experiences with Emergency Remote Teaching in Fall 2020