United States

We designed a rubric to assess free-response exam problems in order to compare thinking skills evidenced in exams in classes taught by different pedagogies. The rubric was designed based on Bloom’s taxonomy (revised version) [1-3]. It was then used to code exam problems. We are now using this rubric to analyze the thinking skills of students taught online during covid vs. students taught in class after covid. We analyzed exams from different sections of the algebra-based physics course taught online during covid and in class after covid. We discuss the instrument, present results and future research plans.

AAPT SM 22

Assessing Thinking Skills in Free-response Exam Problems: Covid vs. Non-covid

AAPT meetings are a great way to reflect on your teaching, recommit to best practices and re-energize your career with new research, products, and techniques. Listen to internationally known plenary speakers, attend sessions, workshops, and special activities. This meeting is an excellent opportunity to share your expertise and learn from and network with leaders in the field.



poster

Open-ended student responses to assessments probing their understanding often contain information about student reasoning that closed-ended responses fail to capture. The Physics Measurement Questionnaire (PMQ) is an open-ended assessment tool designed to measure student understanding of measurement uncertainty. We collected student responses to the PMQ from those enrolled in the two-semester Design, Analysis, Tools, and Apprenticeship Lab course at Michigan State University. The responses were hand-coded by researchers into one of three paradigms for student reasoning about measurement uncertainty: point-like, set-like, or undefined. Here, I will discuss how we used these data to train and test a multi-class logistic regression model. The accuracy and the highest weighted features (words) will be presented and then compared with an equivalent model developed using student data at the University of Colorado, Boulder. The comparison supports a conclusion that models trained on open-ended student responses to the PMQ are robust under small population differences.

Exploring the Reliability of Natural Language Processing Models Across Populations

This action-research project is the culmination of a semester-long course called Modern Physics for Educators. In this study, I examined how active learning using PhET simulations and Pivot Interactives affect student learning of physics as well as their attitudes and beliefs about physics, and what impact,if any, reflective writing had on student learning outcomes. This poster aims to address the following research questions: 1) Does an active learning approach using simulations correlate with student learning of physics content? 2) How do students’ incoming beliefs about physics impact their learning of physics? 3) How does reflective writing impact student learning outcomes and attitudes toward physics?

Student Learning of Photoelectric Effect Using Simulations and Reflective Writing

In this study, we examined two current physics instructors’ Bruce and Thomas views of objectivity in physics and their views of culturally relevant pedagogy. Their statements were analyzed using the conceptions of Culturally Relevant Pedagogy (CRP) listed in Ladson-Billings’ (1995) framework: Conceptions of Self and Others, Conceptions of Knowledge, and Social Relations. Interviews were transcribed and analyzed using thematic analysis. We shared illustrations of their Conceptions of Knowledge that supported CRP that focused on the knowledge of physics as being constructed and reconstructed - now and into the future. Findings from our study indicate that recognition of subjectivity in physics can support instructors in effectively enacting the third conception, Conceptions of Knowledge for optimal use of CRP.

How Instructor's Conceptions of Knowledge Bolster their Culturally Relevant Teaching

Recently, many efforts to improve undergraduate STEM instruction encourage team formation. While team-based approaches are promising for enacting high quality and sustainable outcomes, they are also “risky.” Our research group is currently developing a survey instrument for teams. This research builds on our prior work to develop an instructional change teams model. The model includes five team processes – strategic leadership, egalitarian power dynamics, team member commitment, effective communication, clear decision-making processes, and three emergent states – shared vision, psychological safety, team cohesion. Through the survey development, we aim to explore the interrelationships between all the major pieces in our model and provide formative feedback about team member collaboration. In this poster, I will describe the key parts of our emerging survey instrument and share preliminary findings from the psychometric evaluation of the pilot data. Finally, I will describe opportunities for teams to engage with our work.

A survey for assessing instructional change teams in undergraduate STEM

We report on a study comparing the impact on student attitudes about science and scientific reasoning skills of two instructional interventions applied in a single semester general education physics course. The first intervention had students watch videos on physics topics and complete worksheets testing their comprehension of the concepts and the scientific reasoning shown. The second intervention had students manipulate PhET simulations and complete worksheets based on the simulations. The results of these instructional interventions are presented for two control, three video treatment, and four PhET simulation semesters. The CLASS and Lawson’s CTSR were used for pre- and posttests to assess changes in students’ attitudes about science and students’ scientific reasoning skills. Pre-post shifts in student attitudes were 3% toward more expert-like attitudes with the video interventions and 6.3% with the PhET simulations, and improvements of scientific reasoning skills of 4.2% with the videos and 7% with the PhET simulations.

Student attitude changes and curricular benefits from two instructional interventions

In introductory physics, we often encourage students to see themselves as part of the physics community. To do so, we might highlight inclusive aspects of our classroom culture and the broader physics community. In this poster, we offer a lesson plan that we designed to critique certain ways of knowing in physics. In this lesson plan, we critique gendered science practices, norms, and assumptions as existing in defined boxes. The lesson is structured to support students in noticing, naming, and rejecting these boxes as they have experienced them in life and in their STEM classes. We then offer an alternative view in which STEM classes and culture do not conform to gendered practices and expectations, but instead open avenues for broader inclusion. We aim with this lesson and reflection on science to support students in developing agency to see their ways of being as an integral part of our community.

Rejecting the Gendered Boxes of Science

Current ethics training shows little improvement in ethical decision making. Recent studies have shown that values play a much bigger role in science than what is normally accepted. We believe that by having discussions about the values found in decisions in science, we can improve scientists’ ethical decision making. We are measuring better ethical decision making by seeing how scientists’ awareness of ethical values and level of moral reasoning changed. We formed a fellowship of fifteen science faculty where they discussed the values embedded in various scientific norms to bring awareness to the values inherent in research practices. In order to see how the fellowship impacted the scientists' ethical reasoning, we conducted pre/post interviews where we asked “what would you do” in different ethical vignettes. We determined that scientists’ identified more values and we are still looking at their ethical reasoning.

The Effect of Value-Focused Discussions on Scientists' Ethical Decision Making

To boost student autonomy, traditional lectures have been replaced by a more active format integrating different student programs at the University of Cologne: Learning by Teaching (LbyT). The LbyT concept was researched and developed by using Action Research with the *teacher as reflective practitioner* model (1). The university faculty were simultaneously in the role of researchers as well as teachers exploring their own practices. Initial focus was on their role as teachers and the scope and sequence of student activities. In further development of the courses, the agency of the roles of mentors and tutors unfolded. We analyzed and present the evolution these developments by considering the individual roles. (2) (3) Beyond, we present experiences for tutoring concepts in the typical first semester courses in experimental physics for more than 10 years, where physics students can gain initial experience teaching small learning groups starting their studies.

The unfolding roles of mentors and tutors in Learning-by-Teaching model

The Departmental Action Leadership Institute (DALI) is an initiative that works with physics programs to implement institutional change within their departments. Each participating program sends two faculty members who attend ongoing DALI meetings for one year to learn and reflect on effective change practices. During that year, these change leaders facilitate local teams based on the Department Action Team (DAT) model. We present outcomes associated with individuals’ professional growth, based on survey data, comparing current states of change leaders’ abilities and their retrospective accounts prior to the DALI. These outcomes span areas such as change leaders’ self-efficacy in facilitating DATs, partnering with students, and joint data sense-making. These data also highlight aspects of DALI participants attribute their growth to, as well as what could be improved. We hope this talk serves as inspiration and lessons learned for others partnering with faculty to develop capacity for change in physics programs.

Growing as a change agent: Slowing down and facilitating teams

The practice of partnering with students on departmental projects and initiatives has grown within STEM and Physics programs. These collaborations between faculty and students span multiple contexts, from curriculum and classroom partnerships to institutional change efforts. While these partnerships produce more student-centered results and are able to engage a wider set of departmental stakeholders, they place students in complex roles in which they must navigate complicated systems. This presentation will attempt to summarize the literature student partnership initiatives and situate two ongoing research projects within this literature. The first concerns students who are members of Departmental Action Leadership Institute (DALI) affiliated Departmental Action Teams (DAT), working on institutional change within their programs. The second study is focused on undergraduates who take up the role of Network Fellows within the Access Network, an inter-institutional coalition that works towards a vision of a diverse, equitable, inclusive, and accessible STEM community.

Premium content

Assessing Thinking Skills in Free-response Exam Problems: Covid vs. Non-covid

Next from AAPT SM 22

Exploring the Reliability of Natural Language Processing Models Across Populations

Similar lecture

Team-based approaches to programmatic resources: Who the Guide really guides