United States

As a method to shorten the test time of the Force Concept Inventory (FCI), a previous study suggested the use of Computerized Adaptive Testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. As a step to develop a CAT–based version of the FCI (FCI-CAT), the previous study examined the optimal test length of the FCI-CAT such that accuracy and precision [which were measured in terms of root-mean-square error (RMSE)] of Cohen’s d would be comparable to that of the full FCI for a given class size. The objective of this paper is to address an issue in the previous study and, in so doing, to improve the FCI-CAT. We consider content balancing ensuring that the same set of concepts assessed in the original test is covered in the CAT administration for each respondent. To balance content in CAT, the percentage of items to be administered from each subgroup is defined in advance. Doing so ensures that items from each subgroup are administered. We conducted a Monte Carlo simulation to analyze how implementing an algorithm of content balancing affects the RMSE of Cohen’s d. As a result, we found that, for a class size of 40, the increase of the RMSE due to content balancing is 6%-7% for test lengths of 2-5 items and less than 1% if the test length is larger than 13 items. This result indicates that for a sufficiently large test length (say, larger than 13 items), content balancing does not compromise the accuracy and precision of the FCI-CAT. Hence, we recommend that the FCI-CAT incorporate content balancing, provided the test length is larger than 13 items.

PERC SM 21

Balancing content of computerized adaptive testing for the Force Concept Inventory

computerized adaptive testing

item response theory

assessment

**Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics**

As a gateway discipline to STEM majors and careers, physics plays a central role in deciding who does, and who doesn't, get to discover and pursue scientific careers. Technical and scientific professionals are rewarded with economic privilege and status which affords them both social and cultural power. Therefore, social justice will be realized only when the opportunity, and the choice, to excel in physics is open to all students. The culture and practices of the physics classroom reflect the culture of physics at large, which raises the question of how our professional physics communities can embrace, rather than inadvertently repel, a diverse group of academics that can help improve physics culture.  To this end, the 2021 Physics Education Research Conference will focus on defining inclusive spaces and practices. Additionally, we will explore strategies for eliminating exclusionary practices in our classrooms, research spaces, and professional organizations. We call on the community to propose sessions that will drive conversations related to these initiatives throughout PERC 2021.

Please download the full program [here. ](https://drive.google.com/file/d/1TRMC_SYJ3PUqXyP7sskA7LRSA8ow0eRE/view?usp=sharing)

August 4, 2021 - August 5, 2021 in a Virtual Conference Format

Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics

poster

We present the initial results using an assessment of calculus proficiency in the context of introductory physics (i.e., calculus transfer to physics), comparison of calculus transfer ability with physics self-efficacy, and how these attributes intersect with student gender. Although students demonstrated an improvement in their calculus transfer ability, most nevertheless displayed a decrease in their physics self-efficacy. Similarly, while women showed consistently exhibited lower physics self-efficacy than men, women demonstrated larger gains in their proficiency with calculus transfer to physics. We also discuss future work to better evaluate the interdependence of mathematics self-efficacy, physics self-efficacy, and ability to transfer calculus to physics.

Interplay between physics self-efficacy, calculus transfer ability, and gender

Research on self-efficacy -- or the confidence in one’s capability to perform a task -- has been shown to predict achievement and persistence in the sciences. Throughout a student’s experience in curricular and/or co-curricular activities, they may face threats to their self-efficacy. To investigate the threats or supports to students’ self-efficacy, we are using a mixed methods approach. This novel design creates a holistic narrative of which curricular and/or co-curricular activities are impacting a student’s self-efficacy as well as how and why these activities may be affecting their self-efficacy. We use a mixed methods approach to collect in-the-moment information about lived experiences. Quantitatively, we employ the Experience Sampling Method (ESM) using surveys of domain-specific self-efficacy. Qualitatively, we use daily reflections to investigate threats and supports toward students’ self-efficacy. Using an example, we will present the research design while focusing on building this conversation between the ESM and qualitative reflections.

Identifying Threats and Supports to Self-Efficacy Using Mixed Methods Research

This study is situated within possible selves theory to explore how senior undergraduate students in STEM develop and envision their future possible selves after graduation. We interviewed senior students at an urban institution in the US and encouraged them to think about who they will be in the future associated with their future career choices. This study presents two case studies: Ricky and Francisco. These two students discuss a variety of academic contexts through which they negotiate and explore career-related possible selves: conferences; experiences from classes and research; maintained aspirations for science; and dissatisfaction with education from high school. However, we found that in some cases academic contexts alone are not enough to help students develop possible future selves. Rather, they intersect with social contexts (social identity, out-of-school experiences) through which possible selves emerge. Our findings suggest that students might need more than just classroom-based lessons to explore their possible selves. Implications from this research can benefit the ongoing curriculum development aiming at increasing students' interest in STEM-related careers and building a meaningful professional life after graduation.

Senior undergraduate students developing and envisioning possible selves after graduation

In this study, we adapted a prior identity framework to investigate how students’ perception of learning environment (including sense of belonging, peer interaction and perceived recognition) in an introductory physics course predicts their course grades and physics motivational beliefs (including self-efficacy, interest and identity) at the end of this course. We found signatures of inequitable and non-inclusive learning environment in that female students’ mean scores for sense of belonging, peer interaction and perceived recognition were all lower than male students’ in the course. In addition, we found that female students had lower average course grades than male students. Using structural equation modeling, we found that students’ perception of the learning environment predicts their self-efficacy, interest, identity and grades at the end of the course. In particular, students’ perceived recognition, e.g., by instructors and teaching assistants, played a major role in predicting students’ physics identity, and students’ sense of belonging in physics played an important role in explaining the change in students’ physics self-efficacy. Our findings can be helpful for creating an inclusive and equitable learning environment in which all students can excel.

How learning environment predicts students’ physics grades and motivational beliefs in introductory physics courses

Physics is a degree that supports many career paths, and students often develop preferences for particular methods, such as theoretical, computational or experimental. However, it is not well understood how those preferences develop and affect students' later career decisions. We used Social Cognitive Career Theory (SCCT) as the basis for interpreting students' decision-making processes. SCCT provides a framework for connecting learning experiences, self-efficacy, and outcome expectations with students' interests, goals, and decisions. Semi-structured interviews with 8 physics students were conducted. This analysis focuses primarily on a single student to provide space to explore all three method specializations (theory, computation, and experiment) in more depth. We find that the availability of resources and learning opportunities had a significant impact on students' career choices. Theoretical and computational experiences were readily available through classwork, undergraduate research, and could be worked on at home and in peer study groups. Students lacked the ability to work on experimental physics outside of infrequent classroom opportunities and could not build peer networks that supported their experimental skill growth, which was linked to lower interest and self-efficacy in regards to experimental physics.

Access to opportunities affects physics majors interest and choice of methods specialization

Theory on teacher-student relationship development in higher education is scarce despite the empirical work linking positive relationships to important student outcomes. To advance research in this area, this study explores using Epistemic Network Analysis, a technique that characterizes patterns in data by visualizing them as connected graphs, to characterize patterns in social behaviors exhibited by Teaching Assistants in a large-enrollment algebra-based physics course for non-majors. We recorded classroom interactions between teaching assistants and students in synchronous remote active-learning problem-solving sessions and coded the interactions for the types of social behaviors exhibited by the Teaching Assistants (e.g., addressing students by name or empathizing with student difficulties) as they engage groups of 2-3 students in breakout rooms. Using a validated survey measuring a construct related to teacher-student relationship quality, we identified the two instructors with the highest-rated and lowest-rated scores and compared the connected graphs describing the patterns in their social practices. We present the graphs and generate hypotheses about which social behaviors may contribute most to building or damaging relationships between Teaching Assistants and students. Our results can guide future research in Teaching Assistant-student relationships in physics classrooms and have implications for training Teaching Assistants to build rapport with students.

Characterizing social behavior patterns in teaching assistant interactions with students

In this project, we sought to uncover the cognitive processes and skills that are involved in completing a theoretical physics project. Theoretical physics is often portrayed as a field requiring individual genius and can seem inaccessible to undergraduate students, as well as the public. We drew upon the foundations of Cognitive Task Analysis and completed semi-structured interviews with eleven theoretical physics faculty members from several different research institutions who specialized in subfields including quantum optics, biophysics, computational astrophysics, and string theory. We analyzed the processes and skills of these physicists, focusing on an analysis of idea origin, collaboration, and motivation. We used concept maps to organize these codes and portray the factors that influence the creation of project ideas. We found that motivation and collaboration are fundamental determinants of project ideas and their origins, which contradicts the “lone genius” stereotype. These findings on cognitive processes and skills can help us understand how to better prepare students to do theoretical physics research. Finally, the information gathered during this project may be useful for improving the public understanding of theoretical physics, dispelling the belief that the field requires “genius,” and making it accessible to more students. (Supported by NSF DGE-1846321 and REU-1757477)

When the light bulb turns on: motivation and collaboration spark the creation of ideas for theoretical physicists

Previous work has shown that many physics students will draw unprompted diagrams while solving physics problems. These diagrams are distinct from prompted student-generated diagrams in that they tend to be smaller and contain fewer details, an unsurprising finding given that these two types of diagrams are generated for different reasons. In this study, we expand and elaborate on reasons physics problem-solvers generate diagrams and the differences between unprompted and prompted diagrams by analyzing a set of 10 interviews with undergraduates, graduates, postdocs, and faculty in physics. Interviewees were given six physics problems and three diagramming tasks to complete, followed by a set of explicit questions about diagramming while problem solving. Participants reported that their unprompted diagrams help them: understand the situation and question; store, organize, and prioritize information; and support solution strategies; and communicate information. Participants also shared their beliefs about the limitations of diagrams, including when they are not useful or stop being useful, and difficulties with generating diagrams.

Physics Problem-Solvers on Why They Generate Unprompted Diagrams

Matrix algebra is a commonly applied mathematical tool in physics that can be used in classical mechanics, quantum mechanics, optics, and more. A recent study arising from the Research in Undergraduate Mathematics Education community suggests that students use three mathematical interpretations of matrix multiplication: linear combination, system of equations, and transformations [Larson & Zandieh, 2013], but there are no corresponding studies in PER. We have investigated student written responses [N=43] and student interviews [N=3] to examine how physics students interpret matrix equations using the three interpretations to classify their responses. In this poster, we evaluate both types of data with the primary emphasis on the student interviews. Preliminary results suggest that physics students prefer using the transformations interpretation, which corresponds to interpreting a matrix transforming, or ‘acting on’ a vector, giving a new vector.

Physics students and matrix multiplication: evaluating their interpretations

Ethical training for scientists is not effective in improving scientists' ethical reasoning. In this study, we had fourteen scientists from several disciplines participate in a fellowship designed to have the participants discuss the roles both scientific and non-scientific values play in science. We conducted pre- and post-interviews with the participants about how they viewed ethics in their research and what they thought the appropriate action would be in vignettes posing ethical dilemmas about research conduct. We analyzed these interviews by focusing on the goals and values the faculty express in their ethical reasoning. Our results show that the values scientists' invoke did change when talking about ethics in their own research, but did not change when looking at vignettes. We discuss the implications of this finding for future work regarding the reason for these shifts in reasoning.

Premium content

Balancing content of computerized adaptive testing for the Force Concept Inventory

Next from PERC SM 21

Interplay between physics self-efficacy, calculus transfer ability, and gender

Similar lecture

Comparing item response theory models for ranking incorrect response options