United States

Get the Facts Out (GFO) is a complex and ambitious social change initiative. It involves a huge number of people at national scale, across different levels of the system. As external evaluator, it is my (rather difficult) job to evaluate the success of the project, and help it to improve. I will describe my use of “developmental evaluation,” where results are intended to provide timely feedback to a rapidly evolving complex project. The evaluation has also been guided by the project’s theory of change, and thus addresses questions such as “how well prepared are champions to undertake the work,” and “are campaigns faithful to the principles of GFO”? I will discuss the decentralized, embedded evaluation measures we are using (including a Fidelity of Implementation rubric, activity tracking forms, chat-bots, and surveys). This talk will be of interest to people with an interest in program design and/or project evaluation and assessment.

AAPT SM 20

My biggest project evaluation challenge: Get the Facts Out

theory of change

teacher preparation

evaluation

AAPT® is a strong professional physics science society dedicated to the pursuit of excellence in physical science education.

Winter and Summer National Meetings:
These National Meetings, held each winter and summer, are opportunities for members, colleagues, and future physicists from around the world to:

- participate in physics workshops
- meet and greet other physics educators
- form networks nationally and locally
- engage exhibitors and learn about the latest physics resources
- discuss innovations in teaching methods
- share the results of research about teaching and learning.

If you're not able to log in, you're most likely using **incorrect login credentials**. Double-check that you are using the same email and password that you used to register for the conference, submit your abstract or access your AAPT membership. There may also be **an issue with your AAPT meeting registration**. Please email aapt2021@underline.io and programs@aapt.org and we will work quickly to resolve this issue.

AAPT WM 21

technical paper

The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. The program featured plenary speakers, sessions, workshops and many special activities.

For more information please visit the 2020 AAPT Virtual Summer Meeting page at: https://www.aapt.org/Conferences/SM2020/index.cfm.


The 2020 AAPT Virtual Summer Meeting brought together physics educators from around the globe to learn about the latest in physics theory, research methodology and education from leaders in their field. 

Self-efficacy, or confidence in one's ability to perform some task, is often used as a predictor of academic persistence, particularly in science fields with low retention and in the difficult process of transferring from a two-year college (TYC) to a four-year college (FYC). Self-efficacy has traditionally been measured quantitatively through surveys explicitly asking for confidence rankings. We describe the process of developing a codebook to qualitatively code statements of self-efficacy made in open-ended settings like interviews. To develop this codebook, we used interview and written data from two populations: (1) FYC students pursuing a science degree; and (2) TYC students planning to transfer to a university. The discussion prompts were designed to elicit discussion of these students’ self-efficacy to complete their degrees in natural science majors, to navigate academic requirements, and, when applicable, to transfer to a FYC. We will discuss the mechanics of using this codebook and intended applications.

Coding Students' Statements of Science Degree and Transfer Self-Efficacy

Disability is an important dimension of human diversity. Previous research indicated that physics instructors receive little training about supporting people with disabilities, physics curricular materials are not designed to support students with disabilities, and STEM professionals hold more negative views about people with disabilities than their peers in other academic disciplines. The purpose of this study was to investigate the knowledge possessed by practicing physicists about people with disabilities and different types of impairments. We collected data at 10 American Physical Society meetings via a home-grown survey, the Disability and Physics Career Survey. Specifically, 208 participants were given a list of disability diagnoses (e.g., attention deficit-hyperactivity disorder, blindness, amputated limb) and were tasked with selecting the category of impairment (e.g., cognitive, visual, mobility) that the diagnoses belonged in. If practicing physicists are not knowledgeable about diagnoses and/or categories of impairment, they will be less likely appropriately mentor students with disabilities.

Practicing Physicists’ Knowledge about Disability

Most research focusing on student ideas about electric circuits frames these ideas in terms of misconceptions, difficulties, and misunderstandings. Our project reports student resources for understanding electric circuits, ideas that we consider to be the “beginnings” of foundational understanding. In this talk, we will present four conceptual resources for understanding electric circuits based on our analysis of students’ written responses to conceptual circuits questions. Our aim is to add to instructors’ existing knowledge of students’ ideas, supporting them in identifying and building on student resources for understanding electric circuits.

Identifying Student Conceptual Resources   for Understanding Electric Circuits

Today’s high school students need to develop abilities and skills that are applicable across many fields. Recommendations from the Next Generation Science Standards (NGSS) call for integrating science practices into learning of normative concepts in science classrooms. In my classroom, students learn physics through the Investigative Science Learning Environment (ISLE) approach. Based on previous studies, we know that ISLE students are capable of developing science-process abilities. However, how do we know if this approach to curriculum design and learning prepares students for success in the workplace? During Fall 2019, I administered a survey to alumni who learned physics through the ISLE approach. In this talk, I report on the findings from this study with regard to what students remember learning, how their mindset changed during their time in the course, and what elements of their experience had an effect on and/or were transferrable to their future courses and careers.

Studying the Long Term Effects of Learning Physics Through ISLE

In 2015, seven institutions joined forces to apply a common model of change, creating the Transforming Education, Stimulating Teaching and Learning Excellence (TRESTLE; http://trestlenetwork.org) network. In this talk I will discuss how we are using “principles” to evaluate the network and its impacts (see M.Q. Patton, 2017). Principles are statements which identify the core values, philosophy, or operating assumptions of an initiative, allowing a project to externalize its core values and create accountability for enacting those values. For example, one core principle of TRESTLE is “focus on the department as the main unit of change.” Identifying the core principles of TRESTLE allows us to ask questions such as “Has the network adhered to its principles?”, “How were the principles adapted?”, “Are these principles effective?”, and “How do these principles appear differently at different levels of the system?”

“Guiding star” principles to organize change efforts: The TRESTLE Network

While we might assume that courses are intelligently designed to reflect core concepts and how people learn, all too often course materials reflect disengaged and encyclopedic surveys. Beginning with a fortuitous discussion on the energetics of chemical bonding, Melanie Cooper (Michigan State University) and I have worked to reframe the disciplinary purpose, driving narrative, and conceptual focus of introductory courses in general (CLUE) and organic (OCLUE) chemistry and biology (biofundamentals™). These efforts align with the NAS Framework for Science Education and are based on OER books supporting interactive courses that employ the beSocratic™ web-based system free-form formative assessment system. The result is an emphasis on helping students identify and apply the underlying processes that govern the behavior of the systems under study, enabling them develop well reasoned and plausible models for complex phenomena. Studies on learning outcomes reveal significant improvements in understanding of core ideas.

Building effective and engaging courses supported by evidence and theory

Researchers and educators have developed numerous tools to help students simulate and visualize aspects of quantum mechanics taught in the classroom. However, these tools are typically limited in their scope in that they focus on a narrow range of phenomena. In order to provide a more versatile tool, the ScienceAtHome group at Aarhus University has developed the Quantum Composer, a flow-based program capable of simulating 1D quantum evolution, including potentials that are not typically analytically tractable. Composer also has a suite of other tools that allow students to explore real research problems, including the simulation of Bose-Einstein condensation and control of quantum dynamics via a built-in optimizer. This presentation will introduce Quantum Composer and describe its applications in the classroom from the advanced high school level through to graduate physics education.

Quantum Composer: A flow-based tool for visualization and simulation of quantum mechanics

In traditional labs, instructors provide explicit directions for students to complete experiments and achieve predetermined results that coincide with concepts learned in lecture. However, in inquiry-based labs where students generate their own research questions and procedures, the results of an experiment are often unknown or unexpected. Students in this setting typically find outcomes that do not align with their intuitive predictions or models presented to them in lecture. This perplexing moment in the lab may lead them to take various actions, from collecting more data or asking an instructor, to trying to sweep it under the rug. We call this recognition of a gap in understanding and the subsequent activities that tend to the issue problematizing. I will discuss a study in which we analyzed video of students performing a free fall experiment to determine how they problematize after finding an object’s acceleration to be quite different from g = 9.81 m/s^2.

Problematizing in inquiry-based labs: How students respond to unexpected results

This talk will describe the innovations that we made in our introductory physics courses at Rutgers, Newark (algebra-based and calculus-based) using the ISLE approach. In the spring of 2019 we ran two pilot sections of ISLE-based labs and in the fall of 2019 we implemented whole course reforms in two of our introductory courses with over 400 students. The reforms included revisions in student activities in all three parts of the courses - lectures (large room meetings), recitations (small room meetings) and laboratories. We provided professional development for the 12 instructors and 20 Learning Assistants teaching these courses in a pre-semester workshop and weekly preparation meetings. We will share the details of the changes that we made and the data that we collected using the E-CLASS survey and student evaluations. Rutgers, Newark is an urban public university. Our experience will be useful for those teaching in similar institutions.

ISLE-based reforms in an urban public university

In this talk we present data collected in two introductory physics courses (algebra-based and calculus-based) using the ISLE approach at Rutgers, Newark. The data come from students’ solutions of traditional physics problems evaluated using a rubric that assesses student reasoning abilities, such as ability to communicate, to use different representations consistently, and to evaluate their answer. We track students through one semester and analyze their progress using weekly assessments and course exams.

Premium content

My biggest project evaluation challenge: Get the Facts Out

Downloads

Next from AAPT SM 20

Coding Students' Statements of Science Degree and Transfer Self-Efficacy

Similar lecture

Teacher Recruitment: Focus Group Insights from Six Universities