United States

Research has shown that students’ achievement in science courses as well as their persistence within STEM fields is correlated to a student’s sense of self-efficacy -- the confidence in one’s own ability to perform a task. Traditionally, quantitative changes in a student’s self-efficacy over an academic semester have been measured using pre- and post-test surveys. However, with the complexity of the construct of self-efficacy, a pre-post design may warrant limitations in deeply understanding the impacts curricular and co-curricular activities have on students’ sense of self-efficacy. To investigate the dynamics of students’ self-efficacy in finer detail, we employed the Experience Sampling Method (ESM) -- self-reported, in-the-moment measurements of student experiences over time. Using the results from our pilot study, we will present how we employed the ESM to measure students’ self-efficacy while performing various tasks throughout the week, the validity of our measurements, and the possible versatility of the ESM within STEM education.

AAPT SM 21

The Experience Sampling Method: Measuring the Dynamics of Students’ Self-Efficacy

experienced sampling method

self-efficacy

validation

Join AAPT and Physics Educators worldwide in developing and strengthening your professional connections by attending the 2021 AAPT Virtual Summer Meeting. AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques.

Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. Attending the virtual summer meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.

Please log in with your AAPT account or register for the event if you haven't already.

If you are registered please note that you need to be logged in with your AAPT email / credentials in order to get access.

technical paper

Practice exams with immediate feedback and video solutions are among the highest rated learning resources by students taking introductory physics at the University of Illinois. These ratings align with a research showing the positive impact of formative assessment and the testing effect. Despite this alignment, data on our student use of practice exams suggests a majority of students are missing the most important benefits. In particular, the large fraction of students choosing to complete their first practice exam less than 24 hours before the exam, suggests they are primarily being used for cramming. During the Spring 2021 semester we introduced a “mock” proctored exam one week prior to the actual exam. We look at how performance on the mock exam impacted student predictions, preparation and performance on the actual midterm exam. Our results show that despite the valuable formative assessment available from the mock exam, students were resilient in their dependence on cramming. 


Practice exams impact on student exam preparation and performance

We present a qualitative analysis of twenty-six interviews about computation in introductory mechanics courses. These interviews are part of a longer term project to develop an assessment protocol for computation in introductory physics. Interviewees were drawn from either: introductory physics, research on computation within physics, or an industrial physics career that involves computation. The interviews were coded into twenty-seven different nodes and sub-codes within each node to concentrate and differentiate between node details. We find that an assessment should not be language specific, even though Python is a favorite. The computational level at which interviewees expect students to have when they exit the course varies immensely. Some interviewees have a goal of getting students thinking about or feeling better about computational practices without performing computational practices themselves. Other interviewees have a goal of students solving many problems computationally, the capability to use multiple programming environments, and writing programs from scratch.

Computational Assessment in Introductory Physics: Codes From Qualitative Interview Analysis

As part of a project to develop a written instrument for assessing learning assistants (LAs) pedagogical content knowledge (PCK) in the context of questioning (PCK-Q), we are experimenting with alternate question formats. Previously developed and validated free-response questions serve as the basis for a new question format. This new format consists of Likert-style answer choices that require the LAs to assess a scenario where a student has a misunderstanding, evaluate the situation, and score potential LA responses. The instrument will examine an LAs ability to identify appropriate responses that provide evidence of PCK-Q in the classroom. 


Using Ranking Question to Assess Students Assistants' PCK-Q

We report on the development of a written assessment designed to analyze pedagogical content knowledge in the context of questioning (PCK-Q). We discuss the process of analyzing and coding classroom videos and writing and testing a written assessment. The classroom observations have first been coded using a coding scheme that analyzes levels of questioning in an inquiry-based classroom. A written instrument was then developed based on the classroom videos, administered to student teaching assistants (TAs and LAs) and compared with the video results to validate the written instrument. We report on the coding scheme and the validation of results as part of this ongoing project.

Development of an Instrument for Analysis of Student Assistants’ PCK-Q

The Force Concept Inventory (FCI) and the Brief Electricity and Magnetism Assessment (BEMA) were used in our sophomore-level courses on classical mechanics and electricity and magnetism, respectively, in 2020-21. Both courses used pedagogical methods to encourage interactive, engaged learning. The assessment results and students’ course grades are compared. The result shows that the assessment tools help to diagnose weak topics in student’s command of concepts. The effect of the pandemic of COVID-19 on students’ learning is discussed.

Using research-based assessment tools in intermediate physics courses

Postponing work impacts virtually everyone at some point, especially students. We found academic procrastination is easily observed among students by studying the student’s online submission times. We characterized physics student assignment completion times (ie. procrastintation) behaviors for online homework assignments into 4 completion time classes that clearly distinguish between week-to-week patterns, mean course grades, assignment completion rates, and proportion of women. Consistent with many studies, we find that procrastinating students tend to have lower course grades. Closer examination of exam and non-exam grade components reveals that exam and non-exam components are separately predicted by the two “orthogonal” measures, ACT score and completion time, and we propose that these are measuring so-called cognitive and non-cognitive factors, respectively. Identification of procrastinating students (using completion time) could be important in measuring non-cognitive factors that lead to early identification of at-risk students in physics courses.

Procrastination patterns impact on assignment submission times and grade components

What are the conditions for students to experience real joy in physics learning? Work from authors like Valerie Kaur & Brené Brown suggests that one of the conditions is the opportunity for students to make the continuous choice of being in the emotional discomfort of learning. In this research presentation we will explore frameworks for connecting discomfort to joy in physics learning . To ground our thinking, we will use video data from one group of students in an Introductory Physics for Life Science majors course, who are collaborating on a computational activity to understand physics concepts. We present the analysis of one student’s learning experience, working on a computational activity of containing particles in a box and modeling diffusion, using a lens of joy and discomfort.

Can learning physics be both joyous and uncomfortable?

Informal physics programs facilitate learning spaces that allow for physicists, physics students, and public audiences to interact and engage with each other in a variety of formats, such as public lectures, summer camps and open houses. We want to understand how programs can be sustained over time, and thus, we are interested in documenting the critical organizational dimensions of informal programming, like personnel involved, audience satisfaction, and program content. To identify main challenges across programs, we conducted surveys, interviews with lead program facilitators, and site visits to multiple programs of varying formats. From emergent themes and challenges in our data analysis, we are developing and validating a model for the key organizational components of informal physics programs. In this talk, we share our current key components model of informal programs and discuss future work to develop tools for program facilitators to assess their programs.

Identifying the key organizational components of informal physics programming

In this case study, a small group of secondary students learning geometrical optics with support of a computer simulation was followed. The aim of the study was to observe the how the students make meaning in an inquiry-based learning situation. We used a social semiotic framework to study how the students transfer information using different modalities.

Developing fluency using multiple representations in geometrical optics.

In previous work and in the literature, we have identified multiple reasons why physics problem solvers may generate diagrams as part of the problem-solving process: to orient to the problem; to serve as a tool that is manipulated to help reach an answer; to serve as a means of communication; and/or because it is a habitual part of problem solving. In this talk, we discuss a follow-up study in which we first observe students and instructors solving problems to see what role diagrams play in that process, then we explicitly ask them about their use and beliefs about diagrams generated as part of problem solving. We compare student and instructor use of and beliefs around diagrams as part of an effort to inform pedagogical approaches to teaching diagramming in physics courses.

Premium content

The Experience Sampling Method: Measuring the Dynamics of Students’ Self-Efficacy

Downloads

Next from AAPT SM 21

Practice exams impact on student exam preparation and performance

Similar lecture

Identifying Threats and Supports to Self-Efficacy Using Mixed Methods Research