United States

Many researchers find advantages of assigning and supporting open inquiries in physics lab education. However, objective assessment becomes more difficult as students gain more control over their own experiments because tasks are not described in operational detail. The Assessment Rubric for Physics Inquiry (ARPI) is designed to determine students’ attainment level of inquiry knowledge and to simultaneously assess the scientific quality of the inquiry. The ARPI consists of nineteen understandings of evidence divided over six phases of inquiry. It enables the objective assessment of students’ work without the need to adapt the rubric to differing research topics. In this presentation we highlight the main features of this validated instrument and demonstrate its applicability in a fully open lab course in which natural science students conceive, design, and carry out two extended experiments using accessible Maker tools.

PERC SM 21

Using the Assessment Rubric for Physics Inquiry for open inquiries in a multidisciplinary lab course

open inquiry

lab education

assessment

Join AAPT and Physics Educators worldwide in developing and strengthening your professional connections by attending the 2021 AAPT Virtual Summer Meeting. AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques.

Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. Attending the virtual summer meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.

Please log in with your AAPT account or register for the event if you haven't already.

If you are registered please note that you need to be logged in with your AAPT email / credentials in order to get access.

AAPT SM 21

poster

**Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics**

As a gateway discipline to STEM majors and careers, physics plays a central role in deciding who does, and who doesn't, get to discover and pursue scientific careers. Technical and scientific professionals are rewarded with economic privilege and status which affords them both social and cultural power. Therefore, social justice will be realized only when the opportunity, and the choice, to excel in physics is open to all students. The culture and practices of the physics classroom reflect the culture of physics at large, which raises the question of how our professional physics communities can embrace, rather than inadvertently repel, a diverse group of academics that can help improve physics culture.  To this end, the 2021 Physics Education Research Conference will focus on defining inclusive spaces and practices. Additionally, we will explore strategies for eliminating exclusionary practices in our classrooms, research spaces, and professional organizations. We call on the community to propose sessions that will drive conversations related to these initiatives throughout PERC 2021.

Please download the full program [here. ](https://drive.google.com/file/d/1TRMC_SYJ3PUqXyP7sskA7LRSA8ow0eRE/view?usp=sharing)

August 4, 2021 - August 5, 2021 in a Virtual Conference Format

Making Physics More Inclusive and Eliminating Exclusionary Practices in Physics

In this study, we adapted a prior identity framework to investigate how students’ perception of learning environment (including sense of belonging, peer interaction and perceived recognition) in an introductory physics course predicts their course grades and physics motivational beliefs (including self-efficacy, interest and identity) at the end of this course. We found signatures of inequitable and non-inclusive learning environment in that female students’ mean scores for sense of belonging, peer interaction and perceived recognition were all lower than male students’ in the course. In addition, we found that female students had lower average course grades than male students. Using structural equation modeling, we found that students’ perception of the learning environment predicts their self-efficacy, interest, identity and grades at the end of the course. In particular, students’ perceived recognition, e.g., by instructors and teaching assistants, played a major role in predicting students’ physics identity, and students’ sense of belonging in physics played an important role in explaining the change in students’ physics self-efficacy. Our findings can be helpful for creating an inclusive and equitable learning environment in which all students can excel.

How learning environment predicts students’ physics grades and motivational beliefs in introductory physics courses

Physics is a degree that supports many career paths, and students often develop preferences for particular methods, such as theoretical, computational or experimental. However, it is not well understood how those preferences develop and affect students' later career decisions. We used Social Cognitive Career Theory (SCCT) as the basis for interpreting students' decision-making processes. SCCT provides a framework for connecting learning experiences, self-efficacy, and outcome expectations with students' interests, goals, and decisions. Semi-structured interviews with 8 physics students were conducted. This analysis focuses primarily on a single student to provide space to explore all three method specializations (theory, computation, and experiment) in more depth. We find that the availability of resources and learning opportunities had a significant impact on students' career choices. Theoretical and computational experiences were readily available through classwork, undergraduate research, and could be worked on at home and in peer study groups. Students lacked the ability to work on experimental physics outside of infrequent classroom opportunities and could not build peer networks that supported their experimental skill growth, which was linked to lower interest and self-efficacy in regards to experimental physics.

Access to opportunities affects physics majors interest and choice of methods specialization

Theory on teacher-student relationship development in higher education is scarce despite the empirical work linking positive relationships to important student outcomes. To advance research in this area, this study explores using Epistemic Network Analysis, a technique that characterizes patterns in data by visualizing them as connected graphs, to characterize patterns in social behaviors exhibited by Teaching Assistants in a large-enrollment algebra-based physics course for non-majors. We recorded classroom interactions between teaching assistants and students in synchronous remote active-learning problem-solving sessions and coded the interactions for the types of social behaviors exhibited by the Teaching Assistants (e.g., addressing students by name or empathizing with student difficulties) as they engage groups of 2-3 students in breakout rooms. Using a validated survey measuring a construct related to teacher-student relationship quality, we identified the two instructors with the highest-rated and lowest-rated scores and compared the connected graphs describing the patterns in their social practices. We present the graphs and generate hypotheses about which social behaviors may contribute most to building or damaging relationships between Teaching Assistants and students. Our results can guide future research in Teaching Assistant-student relationships in physics classrooms and have implications for training Teaching Assistants to build rapport with students.

Characterizing social behavior patterns in teaching assistant interactions with students

In this project, we sought to uncover the cognitive processes and skills that are involved in completing a theoretical physics project. Theoretical physics is often portrayed as a field requiring individual genius and can seem inaccessible to undergraduate students, as well as the public. We drew upon the foundations of Cognitive Task Analysis and completed semi-structured interviews with eleven theoretical physics faculty members from several different research institutions who specialized in subfields including quantum optics, biophysics, computational astrophysics, and string theory. We analyzed the processes and skills of these physicists, focusing on an analysis of idea origin, collaboration, and motivation. We used concept maps to organize these codes and portray the factors that influence the creation of project ideas. We found that motivation and collaboration are fundamental determinants of project ideas and their origins, which contradicts the “lone genius” stereotype. These findings on cognitive processes and skills can help us understand how to better prepare students to do theoretical physics research. Finally, the information gathered during this project may be useful for improving the public understanding of theoretical physics, dispelling the belief that the field requires “genius,” and making it accessible to more students. (Supported by NSF DGE-1846321 and REU-1757477)

When the light bulb turns on: motivation and collaboration spark the creation of ideas for theoretical physicists

Previous work has shown that many physics students will draw unprompted diagrams while solving physics problems. These diagrams are distinct from prompted student-generated diagrams in that they tend to be smaller and contain fewer details, an unsurprising finding given that these two types of diagrams are generated for different reasons. In this study, we expand and elaborate on reasons physics problem-solvers generate diagrams and the differences between unprompted and prompted diagrams by analyzing a set of 10 interviews with undergraduates, graduates, postdocs, and faculty in physics. Interviewees were given six physics problems and three diagramming tasks to complete, followed by a set of explicit questions about diagramming while problem solving. Participants reported that their unprompted diagrams help them: understand the situation and question; store, organize, and prioritize information; and support solution strategies; and communicate information. Participants also shared their beliefs about the limitations of diagrams, including when they are not useful or stop being useful, and difficulties with generating diagrams.

Physics Problem-Solvers on Why They Generate Unprompted Diagrams

Matrix algebra is a commonly applied mathematical tool in physics that can be used in classical mechanics, quantum mechanics, optics, and more. A recent study arising from the Research in Undergraduate Mathematics Education community suggests that students use three mathematical interpretations of matrix multiplication: linear combination, system of equations, and transformations [Larson & Zandieh, 2013], but there are no corresponding studies in PER. We have investigated student written responses [N=43] and student interviews [N=3] to examine how physics students interpret matrix equations using the three interpretations to classify their responses. In this poster, we evaluate both types of data with the primary emphasis on the student interviews. Preliminary results suggest that physics students prefer using the transformations interpretation, which corresponds to interpreting a matrix transforming, or ‘acting on’ a vector, giving a new vector.

Physics students and matrix multiplication: evaluating their interpretations

As a method to shorten the test time of the Force Concept Inventory (FCI), a previous study suggested the use of Computerized Adaptive Testing (CAT). CAT is the process of administering a test on a computer, with items (i.e., questions) selected based upon the responses of the examinee to prior items. As a step to develop a CAT–based version of the FCI (FCI-CAT), the previous study examined the optimal test length of the FCI-CAT such that accuracy and precision [which were measured in terms of root-mean-square error (RMSE)] of Cohen’s d would be comparable to that of the full FCI for a given class size. The objective of this paper is to address an issue in the previous study and, in so doing, to improve the FCI-CAT. We consider content balancing ensuring that the same set of concepts assessed in the original test is covered in the CAT administration for each respondent. To balance content in CAT, the percentage of items to be administered from each subgroup is defined in advance. Doing so ensures that items from each subgroup are administered. We conducted a Monte Carlo simulation to analyze how implementing an algorithm of content balancing affects the RMSE of Cohen’s d. As a result, we found that, for a class size of 40, the increase of the RMSE due to content balancing is 6%-7% for test lengths of 2-5 items and less than 1% if the test length is larger than 13 items. This result indicates that for a sufficiently large test length (say, larger than 13 items), content balancing does not compromise the accuracy and precision of the FCI-CAT. Hence, we recommend that the FCI-CAT incorporate content balancing, provided the test length is larger than 13 items.

Balancing content of computerized adaptive testing for the Force Concept Inventory

Education researchers often compare performance across race and gender on research-based assessments of physics knowledge to investigate the impacts of racism and sexism on physics student learning. These investigations' claims rely on research-based assessments providing reliable, unbiased measures of student knowledge across social identity groups. We used classical test theory and differential item functioning (DIF) analysis to examine whether the items on the Force Concept Inventory (FCI) provided unbiased data across social identifiers for race, gender, and their intersections. The data was accessed through the Learning About STEM Student Outcomes platform and included responses from 4,848 students posttests in 152 calculus-based introductory physics courses from 16 institutions. The results indicated that the majority of items (22) on the FCI were biased towards a group. These results point to the need for instrument validation to account for item bias and the identification or development of fair research-based assessments.

Bias on the Force Concept Inventory across the intersection of gender and race

Previous work has shown that item response theory may be used to rank incorrect response options to multiple-choice items on commonly used assessments. This work has shown that, when the correct response to each item is specified, a nominal response model (NRM) may be used to rank the incorrect options. We seek to determine the robustness of these results by applying the NRM to all response choices, without specifying the correct response. We apply these analyses to multiple data sets (each with more than 9,000 response sets), including pre-instruction and post-instruction responses. We find that the rankings generated without specifying the correct response are consistent with the previously published rankings for one data set; however, we find noticeable differences between rankings generated from different data sets. We provide evidence that discrepancies may result from differences in response rates for less commonly chosen responses.

Comparing item response theory models for ranking incorrect response options

Ethical training for scientists is not effective in improving scientists' ethical reasoning. In this study, we had fourteen scientists from several disciplines participate in a fellowship designed to have the participants discuss the roles both scientific and non-scientific values play in science. We conducted pre- and post-interviews with the participants about how they viewed ethics in their research and what they thought the appropriate action would be in vignettes posing ethical dilemmas about research conduct. We analyzed these interviews by focusing on the goals and values the faculty express in their ethical reasoning. Our results show that the values scientists' invoke did change when talking about ethics in their own research, but did not change when looking at vignettes. We discuss the implications of this finding for future work regarding the reason for these shifts in reasoning.

Premium content

Using the Assessment Rubric for Physics Inquiry for open inquiries in a multidisciplinary lab course

Next from PERC SM 21

How learning environment predicts students’ physics grades and motivational beliefs in introductory physics courses

Similar lecture

Assessing a flipped-lab course consisting of open-inquiry projects using Arduinos