United States

Recent science education reform calls for argumentation as one of the central practices in science classrooms. However, evaluating student argumentation proficiency is challenging. Based on an existing learning progression of argumentation and grade-appropriate objectives, we developed a measure for middle-school students in three contexts (e.g. kinetic theory of gases) of 19 constructed response items. Experts coded responses from 932 middle-school students for components of argumentation and used these coded responses to train machine learning algorithms for each item. The algorithms were validated using a cross-validation approach. We found that the machine-human agreements are robust, with a mean Cohen’s kappa=0.75, SD=0.08. We further applied a many-facet Rasch analysis and found the item difficulty in this measure is well aligned with the established learning progression. The algorithms developed in this study can be used in classrooms to automatically evaluate students’ argumentation proficiency.

AAPT SM 20

Applying Machine Learning to Automatically Assess Middle-School Students’ Argumentation

argument

automatic scoring

assessment

machine learning

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. The program featured plenary speakers, sessions, workshops and many special activities.

For more information please visit the 2020 AAPT Virtual Summer Meeting page at: https://www.aapt.org/Conferences/SM2020/index.cfm.


The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. 

technical paper

Student issues with both understanding electrical potential and difficulties with interpreting diagrams have been well explored. A major issue in student ranking of electric potentials is identification of the sign of the electric charge. An investigation of modifying traditional equipotential diagrams based on theories of visual affordances to improve students’ recognition of electric change sign when ranking electric potential was undertaken. A prior study showed equipotential diagram modifications significantly increased student gaze times at the diagrams without increasing students’ overall correctness rates. In this study modifications of color and line style were made to equipotential lines to increase visual salience of indicators of change sign. Students were randomly assigned to the traditional or modified diagrams and asked to compare electric potentials for indicated points on given diagrams. Pre- and post-test comparisons and the results of student interviews will be presented to clarify the specific issues students have reading equipotential diagrams.

Changes to equipotential diagrams to improve student ranking of electric potential

Our overall goal is to develop “diagnose-and-practice” resources to help students in our introductory physics courses improve their accuracy and fluency on math skills necessary for success. In this phase of the project we focused on developing and testing online practice exercises. In each of 11 lecture sections, all students were assigned to practice either trigonometry or linear equations using our online platform multiple times throughout the semester. We observed that on average, students in both conditions decreased in pre -to-post time on all items, with the relevant treatment group showing larger decrease for some of the relevant test items. Accuracy scores were high with no differences between conditions. We also tracked student accuracy and time during their practice throughout the semester and found student improvement in accuracy and/or time for some categories, but not for others. A new practice protocol will be implemented in AU2020 based on these results.

Online practice of trigonometry and linear equations skills for physics

We teach Optics by leveraging a coherent series of hands-on laboratory engagements that connect well with classroom discussion of mathematical modeling. These labs directly explore mathematical models contained in Optics texts, commonly utilizing liquid crystal Spatial Light Modulators (SLMs), which allow simple and direct programmatic control of phase, amplitude, or polarization across the field of a beam of light. As added bonus, their use avoids the excessive burdens associated with manual alignment and re-alignment (and re-alignment and...) that would be required for systematic studies based on traditional (fixed) optical components. Again, SLMs allow direct control of amplitude and phase modulation of beams, which is useful for teaching Fresnel Diffraction, Fraunhofer Diffraction, and Fourier Optics, as well as spatial filtering, computer-generated Holograms, Aberration Correction, Laser Modes, and much, much more (e.g., encoding information, the linear momentum, spin angular momentum, and orbital angular momentum of light beams).

The Future of Optics is Programmable

We are continuing a nationwide effort to develop a systemic understanding of the landscape of informal physics using an organizational theory perspective. We have collected surveys and interviews with facilitators, but this information is only from the perspective of the faculty or physics student leaders and does not tell us about the social dynamics within each program. Thus, to complement these data, we need to observe informal physics events as they occur. In this talk, we will discuss our strategy for visits to program sites to observe social interactions between program participants as well as programmatic details in action. We report on an initial site visit to a physics open house outreach event, where we took field-notes and conducted formal and informal interviews with participating personnel members. Here, we discuss findings from applying an organizational framework to the site visit data and challenges we encountered in the data collection process.

An Ethnographic Approach to Understanding Informal Physics

Although we have found that students in our Introductory Physics for Life Science (IPLS) course describe physics as more relevant to their primary interests than do their counterparts in a traditional introductory physics environment, we do not yet know whether IPLS courses better prepare life science students to use physical reasoning in contexts that extend beyond those explicitly encountered in IPLS. To answer this question of whether IPLS better prepared our students for future learning, we designed and administered a task related to fluid dynamics at the conclusion of both traditional and IPLS introductory physics courses. We describe the construction of the task and the ways in which IPLS students approached the task differently than did students in the traditional course. We interpret the results in light of the goal of the IPLS course, supporting transfer within the preparation for future learning paradigm around which our course is designed.

Assessing the impact of IPLS on physical reasoning

Previous studies have shown evidence of a gender gap in performance in introductory physics courses for conceptual assessments, course grades, exams, and homework. This study explores the relation between a student’s gender and their performance, and perception of that performance, in both calculus-based and algebra-based introductory physics sequences at Texas A&M University. Investigation of student performance based on gender is done using analysis of variance on student midterm exam grades and final letter grades from multiple instructors’ courses from 2008-2018. Student perception of their performance and inclusion in the course was measured through a survey administered in fall 2019. We present results from both parts of this study to try to better understand differences in gendered performance in introductory physics sequences.

Instructor and Student Gender's Effect on Performance in Introductory Courses

Within the last 15 years, computational thinking (CT) has emerged as a focal point of K-12 education. Numerous frameworks have outlined the practices involved when students and teachers engage in CT. These frameworks discuss how CT practices should be understood, implemented, and assessed in the classroom. While curricular expectations around this topic are becoming clearer, teachers (especially high school STEM instructors) still express a significant need for assessment strategies in the classroom. In light of this, we have begun to develop an assessment that will measure high school and early college instructors’ perspectives on CT practices. Currently, we are constructing an open-ended pilot survey that will help us understand which practices are relevant to teachers and how familiar teachers are with these practices. Once we have identified the CT practices that teachers are interested in evaluating, we will move to the next step of the assessment development: creating an open-ended free-response questionnaire to administer to students.

Navigating Computational Thinking Frameworks for Introductory Physics Courses

The Attitudes and Approaches to Problem Solving (AAPS) Survey has been used to analyze students in introductory calculus-based physics, introductory math-based astronomy, upper division/graduate-level physics, as well as Turkish students in high school and introductory university physics. In this study, post-test survey results from introductory algebra-based physics course sections in 2019 and 2020 will be checked for statistical relationship with the following variables: 1) students’ self-reported learning goals in introductory algebra-based physics courses; 2) students’ choice of major (in particular, life science majors dominate the student population). From prior studies regarding these two variables, it is anticipated that students that express a mastery learning goal (as opposed to a performance learning goal or otherwise) may have more expert-like views than other students; discussion will focus on the accuracy of this statement and potential explanations for results.

Attitudes and Approaches to Physics Problem Solving Survey: Life Science Majors

The Big Astronomy Project is a multi-institutional effort that aims to share the stories of the people and places that make big astronomy possible in Chile. We have developed a model, the Dome+ model, to identify best practices for extending engagement beyond planetarium shows. The model’s main component is a planetarium show, where planetarium visitors are shown NSF ground-based observatories in Chile. To support STEM identity, interest, and agency for planetarium visitors, we are creating additional resources such as weekly virtual sessions with STEM professionals, a web portal with additional content, and a suite of hands-on activities. The project strives to 1) promote awareness of the investments in astronomy being made by the US in Chile, 2) encourage interest in diverse STEM career opportunities at large observatories, 3) share knowledge of the science enabled by big astronomy, 4) increase perceptions by Latinx youth and adults about careers at observatories.

Extending Engagement Beyond the Planetarium: Big Astronomy in Chile

Diagraming physical scenarios is ubiquitous in physics problem solving and a key focus of physics education. In this talk, we discuss and compare unprompted and prompted diagrams generated by 19 physics majors (ranging from lower-division to graduate students) during one-on-one problem-solving interviews. These interviews included multiple-choice introductory-level physics problems (that neither contained nor asked for diagrams) followed by explicit diagraming tasks for similar scenarios. We discuss patterns and surprises in student-generated diagrams as well as implications for instruction and assessment.

Premium content

Downloads

Next from AAPT SM 20

Changes to equipotential diagrams to improve student ranking of electric potential

Similar lecture

Diagnosing Middle-School Students’ Cognition in Argumentation Practices Using Machine Learning-based Assessments