United States

The application of various machine learning algorithms and techniques to the prediction of student performance in an introductory mechanics class is discussed (N = 2061 students). The dataset used for prediction includes institutional variables (GPA, credit hours) and in-class variables gathered from the first two weeks of class (homework and clicker scores). These variables are used to predict which students are at risk of failing the course (receiving a D or F). The effectiveness and implementations of random forest, logistic regression, and voting ensemble classification models are discussed. Various model optimization methods are explored including decision threshold tuning, model regularization, synthetic upsampling, and variable selection techniques. The best performing models achieved a balanced accuracy of around 80%.

AAPT SM 22

Machine Learning Techniques for Classifying Physics Performance

AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques. Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. This meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.



poster

This work is part of a multi-institution collaboration developing a fair, valid, and reliable research-based conceptual fluids assessment, the Fluids Conceptual Evaluation (FCE). The FCE will utilize two-tier multiple-choice items covering both fluids statics and fluids dynamics. Rasch analysis will be used to create different equivalent versions of the FCE, allowing instructors to give just the statics questions, just the dynamics questions, or a combination of both. In this poster we will describe the development of student-based distractors for the FCE. IPLA instructors interested in serving as a pilot test site should contact Project Lead Dawn Meredith.

Development of questions for the Fluids Conceptual Evaluation (FCE)

This work is part of a multi-institution collaboration developing a fair, valid, and reliable research-based conceptual fluids assessment, the Fluids Conceptual Evaluation (FCE). The FCE will utilize two-tier multiple-choice items covering both fluid statics and dynamics. Initially we developed 101 tier-1 questions covering 8 statics constructs and 9 dynamics constructs, with the intent of including 3 questions per construct in the final assessment. In the spring of 2021, we conducted 15 clinical interviews asking student volunteers selections from those questions. Analysis of interviews, and feedback from the interviewers, led to the revision, removal, and addition of several questions, which were used in the second round of interviews during the 2021-2022 academic year. In this poster we describe selected takeaways from this process, including some “dos and don’ts” for constructing robust questions. IPLS instructors interested in serving as a pilot test site for the assessment should contact Project Lead Dawn Meredith.

Refining Assessment Questions Based on Clinical Interviews

Creating a more equitable scientific community is an important goal in physics. To this end, one approach is addressing students’ views about the socially and culturally embedded nature of science (NOS) and increasing their awareness about the influence of subjectivity in physics. Identifying ways personal experience and values affect the culture of physics, may promote student self-efficacy and empower them to work towards greater social justice in physics. Prior studies reveal that without explicit instruction about the NOS, students tend to describe physics without reference to human influence. Adding to this, we explored undergraduate students’ ideas about the presence of subjectivity in physics in an introductory calculus-based physics sequence and with prospective elementary teachers in a physics course. We collected written responses on whether students viewed physics as objective or subjective. In this talk, we discuss how student responses describe subjectivity in physics and argue for explicit NOS instruction.

Addressing Subjectivity in Physics for Equity in Physics Education

The PhysPort Data Explorer (www.PhysPort.org/DataExplorer) is an online tool for physics instructors to analyze their research-based assessment data (including FCI, FMCE, BEMA, CSEM, CLASS, MPEX, and many more). Instructors upload their students' responses using our secure interface. The Data Explorer matches their pre/post data, scores it, compares it to national data, and graphs it in an interactive and intuitive manner. Instructors can look at results over time, breakdown by question or cluster, and comparisons between courses. Instructors can also upload data related to student demographics, academic record, or background, and look at their assessment results based on this information. We present the Data Explorer and discuss equity issues around analysis and interpretation of research-based assessment results, in order to support faculty in using data to reveal and address inequities, to more effectively support minoritized groups of students, and to minimize and eliminate harm.

Using the PhysPort Data Explorer to analyze research-based assessment results

Force symbols in University Modeling Instruction explicitly represent forces as detailed descriptions of interactions to facilitate (i) coordinating force with the system schema and (ii) learning force. For example, represents the gravitational force by Earth on ball, where “g” represents gravitational (i.e. the type of interaction), “E” represents Earth, represents “by” and “on”, and “B” represents ball. Although students are taught to say as “gravitational force”, audio data from student-led whole-class discussions shows that more than 40% percent of the time was referred to as “force gravity” instead. Analogous results were obtained for contact force symbols as well. Because language plays such a crucial role in learning physics, several years ago, as an experiment, the notation was changed from to to make it more closely match how it is to be read. Student use of “force gravity” and “force contact” dropped to less than 5% with this notation switch. Energy.

Changing Notation That Represents Force Changes How Students Say It

With the growing ubiquity of computation in STEM fields, understanding how to teach computational thinking (CT) practices has become an active research area in the last two decades, with particular emphasis on developing CT frameworks. In this paper, we apply one of these CT frameworks and correlate the results with a task analysis to examine how CT practices relate to specific design features of an in-class problem. We have analyzed video data from two separate groups working on one computational class period, which utilizes a minimally working program to model magnetic field vectors. While still in the initial stages of the study, our preliminary results indicate that what is left out of the minimally working program will impact the CT practices students use, particularly around building computational models. Ultimately, we hope this work will help instructors to design activities that can target & build specific CT practices

Relating Computational Thinking Practices and Problem Design Features

We’re interested in ways that students collaborate to solve conceptual physics problems in the context of spherical unit-vectors in upper-division E&M, especially ones that are difficult for individuals to solve, but which small groups are more successful at. Using a think-out-loud protocol, we interviewed students in small groups, asking them to solve together problems from this context on a large whiteboard. Using the resources framework and an emergent coding analysis of interview video, we observed one common mechanism in all three group-interviews whereby students collaborated effectively: first one student activated a conceptual resource and expressed it, then another student took up that idea, then finally the whole group together used that idea to move forward with the problem. In this poster, we report our method of analysis and results from one group interview to illustrate a resource activated and used in this way, which we call a shared resource.

Shared Resources in Student Problem-Solving of Spherical Unit Vectors

This poster analyzes two examples of whiteboard meetings from a college calculus-based introductory physics course taught using University Modeling Instruction. In UMI, student small groups create a solution to the same problem on 2? × 3? whiteboards. They then sit in a large circle with whiteboards held facing in and conduct a student-led whole-class discussion to reach consensus. In the first example students overcame sharp disagreements to reach consensus, but in the second example they don’t. We examine how social positioning contributed to students either successfully examining and resolving different ideas or failing to do so. Results from these two examples support the idea that meetings where “experts” soften their position by “hedging” more frequently are better able to overcome sharp initial disagreements to reach consensus on their own. Our analysis suggests that the way students position themselves in discussions may open or close the collaborative space to productive sense-making.

Two contentious board meetings; social positioningCorrelates with consensus building

In California and other western states, discussions about climate change and water access have become increasingly urgent. Connecting science learning inside the classroom to real issues that affect our students and their families is key to help students see science as a tool to work for change in their communities. Place-based education, which is rooted in historic and Indigenous educational practices, strongly supports these goals. In our Physical Science course for ninth and tenth graders, we added an outdoor experiential component to our “Science of Water” unit thereby placing our learning about water into our local context of rural Northern California. From the chemical properties of water, power generation via our local hydroelectric dam, decisions about water usage and the effects on ecosystems, and the implications of climate change, students can see the connections between all of these topics rather than learning each in isolation in separate courses.

Place-based education in high school physical science

Due to the low number of physics majors choosing to go into high school teaching positions, there has been an increasing number of physics classrooms led by teachers with little to no background knowledge. These limitations reduce the effectiveness of the class, and tend to go unaddressed by public school systems. The Mitchell Institute Physics Enhancement Program was created to address these issues through assisting out-of-field teachers in gaining necessary background knowledge. Over a two week period, participants work with physics faculty as well as two master high school physics teachers, training in both their subject knowledge and implementation. Participant pre- and post-program knowledge and confidence were measured using an assessment compiled of questions from mechanics, electricity and magnetism, optics, and modern physics. New results from the 2021 and 2022 cohorts will be presented. Analysis of our findings will also be discussed to encourage other institutions to create similar programs.

Premium content

Machine Learning Techniques for Classifying Physics Performance

Next from AAPT SM 22

Development of questions for the Fluids Conceptual Evaluation (FCE)

Similar lecture

Causal mapping analysis of Universal Design for Learning-aligned instructional changes