United States

Recently, machine learning algorithms, primarily random forests, have been used to identify students at risk of failing introductory physics. While these algorithms produced excellent accuracy when identifying students who would receive an “A” or “B” in a physics class, the performance was much worse when identifying students who would receive a “D” or “F”. This work investigates multiple pathways to optimize the algorithms to improve performance including using alternate algorithms such as support vector machines, optimization algorithms such as gradient boosting and regularization, and ensemble algorithms which allow multiple different algorithms to vote on the results. This work seeks to identify the limit of accurate prediction with only institutional variables.

AAPT SM 20

Optimizing Machine Learning to Identify At-Risk Physics Students

force concept inventory

misconceptions

network analysis

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. The program featured plenary speakers, sessions, workshops and many special activities.

For more information please visit the 2020 AAPT Virtual Summer Meeting page at: https://www.aapt.org/Conferences/SM2020/index.cfm.


The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. 

technical paper

Earning a bachelor's degree is expensive and time-consuming. Many undergraduate students pursue Advanced Placement (AP) courses in high school or transfer coursework from degree-granting institutions. However, the effect of those transfer courses on the time that it takes students to graduate (time-to-degree) is currently not well understood. We have investigated how incoming transfer courses impact students’ time-to-degree by defining three independent groups of transfer students: (1) those entering with college level transfer courses, (2) those with only AP level transfer courses, and (3) those without any transfer credit. The time-to-degree has been shown to be statistically different for each of these groups, with students who enter with college level transfer courses graduating the earliest. We have explored differences in time-to-degree for various demographic information (e.g. gender, race/ethnicity). In this presentation, we will discuss these results in addition to the results from regression models that predict a student’s time-to-degree.

Analyzing Time-to-Degree for Transfer Students at Michigan State University

Physics education research on student understanding of kinematics has largely focused on misconceptions and difficulties. Our project reports student resources for understanding kinematics -- ideas that we consider to be the “beginnings” of sophisticated understandings. Our preliminary analysis highlighted four common resources that students are using to solve kinematics problems. In this talk, we will elaborate on the resources used most often by students and give examples from our preliminary research.

Identifying student resources for understanding kinematics

This study examines the predictive power of different parameters on FMCE (Force and Motion Conceptual Evaluation) pretest score. The sample consists of students attending a large eastern land-grant university in the US enrolled in the introductory calculus-based mechanics class (N=2457). A variety of parameters were used to predict FMCE pretest score including gender, underrepresented minority status, first-generation status, high school GPA, college GPA, ACT/SAT score, college math entry level, AP classes taken, transfer credits, class standing, prerequisite course grade, and college credits completed. Correlation analysis and linear regression were used to understand relations between the variables. Several parameters were statistically significant explaining the variation of FMCE pretest score including AP classes taken, class standing, and calculus-readiness.

What does the Force and Motion Conceptual Evaluation pretest measure?

Thermal physics encompasses thermodynamics and statistical mechanics content and is a core upper-division course requirement for most physics degrees. Thus, in order to improve student outcomes in these courses, it is important for both educators and researchers to have a validated method of evaluating student understanding of upper-division thermal physics content. For an assessment like this to be useful, it is crucial that the assessment tool can be both easily implemented and easily scored. In this presentation, we discuss a multiple response upper-division thermal physics assessment that was distributed in an online format in the Spring of 2020. The development of multiple choice and coupled, multiple response assessment items and initial implementation of these items, including validation, will be discussed.

Initial Implementation of an Upper-Division Thermal Physics Assessment

Flat-earthism is one kind of science denial, and it asserts that scientists conspire with governments to keep secret the "fact" that the Earth is flat. Many scientists' attempts to disprove a flat Earth unfortunately cite evidence that is not easily accessible to a non-scientist. I have made a navigable 3D computational model of a popular U.S. flat-earthers' model of a flat Earth, which makes it possible to identify a number of naked-eye observations that strongly disagree with the predictions of the flat-earthers' own model. This is potentially useful because flat-earthers privilege naked-eye observations as the only valid kind of evidence. I will demonstrate the computational model, which is available at tinyurl.com/FEmodel.

A Flat Earth?

Periscope is a set of lessons that connects big questions of physics teaching and learning to authentic video episodes from best-practices physics classrooms. Periscope lessons are useful if you supervise learning assistants or teaching assistants, lead faculty development, seek to improve teaching in your department, or want to improve your own teaching. Periscope’s primary aim is to help STEM instructors see authentic teaching events the way an expert educator does – to develop their “professional vision” (C. Goodwin, American Anthropologist 96(3), 1994). This development of professional vision is particularly critical for educators in transformed STEM courses, who are expected to respond to students’ ideas and interactions as they unfold moment to moment. By watching and discussing authentic teaching events, instructors enrich their experience with noticing and interpreting student behavior; practice applying lessons learned about teaching to actual teaching situations; train to listen to and watch students in their own classrooms by having them practice on video episodes of students in other classrooms; observe, discuss, and reflect on teaching situations similar to their own; develop pedagogical content knowledge; get a view of other institutions’ transformed courses; and expand their vision of their own instructional improvement. Periscope is free to educators at physport.org/periscope.

Periscope: Looking into learning in best-practices physics classrooms

Wave optics is a difficult teaching topic that was up to now usually investigated on university students. In Croatia, this topic is compulsory for a significant fraction of high school students, aged 18-19 years. In 2018. we started a four-year research project that includes investigation of high school students' difficulties with wave optics, construction of a new diagnostic instrument (Conceptual Test on Wave Optics), and development and testing of an experimental teaching sequence on wave optics. The final goal of the project is to investigate the effect of the new teaching approach, which includes several students' investigative experiments, on students' conceptual understanding of wave optics and on their scientific reasoning. Some preliminary results of the project will be presented, including findings from the 27 semi-structured interviews and from the process of the CTWO development. The most frequent students' conceptual difficulties regarding interference, diffraction and polarization of light will be discussed.

Investigating high school students' understanding of wave optics

The Applying Scientific Knowledge (ASK) program recruits science and math majors in their second year, who take a shared research methods class and then conduct research with faculty mentors for two or more semesters. Assessment throughout the program studies students' experiences of research, their STEM community development, and their skill development and academic progress. The question guiding this portion of the analysis was: How do students define or describe the concept of research, and how does that change as they advance through the ASK program? Students were interviewed near the end of the methods class (prior to joining labs), then after one or more semesters of research experience. We will present a preliminary analysis of interview themes from the first two cohorts, focusing on the physics majors and their similarities or differences from other students. We find that students have a mix of career-oriented and personal motivations for pursuing research.

Diving into a void: Student views of research

Get the Facts Out (GFO) is a complex and ambitious social change initiative. It involves a huge number of people at national scale, across different levels of the system. As external evaluator, it is my (rather difficult) job to evaluate the success of the project, and help it to improve. I will describe my use of “developmental evaluation,” where results are intended to provide timely feedback to a rapidly evolving complex project. The evaluation has also been guided by the project’s theory of change, and thus addresses questions such as “how well prepared are champions to undertake the work,” and “are campaigns faithful to the principles of GFO”? I will discuss the decentralized, embedded evaluation measures we are using (including a Fidelity of Implementation rubric, activity tracking forms, chat-bots, and surveys). This talk will be of interest to people with an interest in program design and/or project evaluation and assessment.

My biggest project evaluation challenge: Get the Facts Out

Self-efficacy, or confidence in one's ability to perform some task, is often used as a predictor of academic persistence, particularly in science fields with low retention and in the difficult process of transferring from a two-year college (TYC) to a four-year college (FYC). Self-efficacy has traditionally been measured quantitatively through surveys explicitly asking for confidence rankings. We describe the process of developing a codebook to qualitatively code statements of self-efficacy made in open-ended settings like interviews. To develop this codebook, we used interview and written data from two populations: (1) FYC students pursuing a science degree; and (2) TYC students planning to transfer to a university. The discussion prompts were designed to elicit discussion of these students’ self-efficacy to complete their degrees in natural science majors, to navigate academic requirements, and, when applicable, to transfer to a FYC. We will discuss the mechanics of using this codebook and intended applications.

Premium content

Optimizing Machine Learning to Identify At-Risk Physics Students

Downloads

Next from AAPT SM 20

Analyzing Time-to-Degree for Transfer Students at Michigan State University

Similar lecture

Extending Modified Module Analysis to Include Correct Conceptual Physics Responses