Skip to main content

Publications

Methods for trustworthy application of Large Language Models in PER

Cornell Affiliated Author(s)
Author
Rebeckah Fussell
Megan Flynn
Anil Damle
Natasha Holmes
Abstract

Within physics education research (PER), a growing body of literature investigates using natural language processing machine learning algorithms to apply coding schemes to student writing. The aspiration is that this form of measurement may be more efficient and consistent than similar measurements made with human analysis, allowing larger and broader data sets to be analyzed. In our work, we are harnessing recent innovations in Large Language Models (LLMs) such as BERT and LLaMA to learn complex coding scheme rules.

Conference Name
APS April Meeting 2024
Date Published

What topics of peer interactions correlate with student performance in physics courses?

Cornell Affiliated Author(s)
Author
L Simpfendoerfer
Meagan Sundstrom
Matthew Dew
N Holmes
Abstract
Research suggests that interacting with more peers about physics course material is correlated with higher student performance. Some studies, however, have demonstrated that different topics of peer interactions may correlate with their performance in different ways, or possibly not at all. In this study, we probe both the peers with whom students interact about their physics course and the particular aspects of the course material about which they interacted in six different introductory physics courses: four lecture courses and two lab courses.
Journal
European Journal of Physics
Date Published

Who and what gets recognized in peer recognition

Cornell Affiliated Author(s)
Author
Meagan Sundstrom
L. Simpfendoerfer
Annie Tan
Ashley Heim
N. Holmes
Abstract
Previous work has identified that recognition from others is an important predictor of students’ participation, persistence, and career intentions in physics. However, research has also found a gender bias in peer recognition in which student nominations of strong peers in their physics course disproportionately favor men over women.
Journal
Physical Review Physics Education Research
Date Published

Method to assess the trustworthiness of machine coding at scale

Author
Rebeckah Fussell
Emily Stump
N. Holmes
Abstract
Physics education researchers are interested in using the tools of machine learning and natural language processing to make quantitative claims from natural language and text data, such as open-ended responses to survey questions. The aspiration is that this form of machine coding may be more efficient and consistent than human coding, allowing much larger and broader datasets to be analyzed than is practical with human coders.
Journal
Physical Review Physics Education Research
Date Published

Context affects student thinking about sources of uncertainty in classical and quantum mechanics

Cornell Affiliated Author(s)
Author
E.M. Stump
M. Dew
G. Passante
N.G. Holmes
Abstract

Measurement uncertainty is an important topic in the undergraduate laboratory curriculum. Previous research on student thinking about experimental measurement uncertainty has focused primarily on introductory-level students' procedural reasoning about data collection and interpretation. In this paper, we extended this prior work to study upper-level students' thinking about sources of measurement uncertainty across experimental contexts, with a particular focus on classical and quantum mechanics contexts.

Journal
Physical Review Physics Education Research
Date Published

Comparing introductory and beyond-introductory students' reasoning about uncertainty

Cornell Affiliated Author(s)
Author
E.M. Stump
M. Hughes
G. Passante
N.G. Holmes
Abstract

This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Uncertainty is an important concept in physics laboratory instruction. However, little work has examined how students reason about uncertainty beyond the introductory (intro) level. In this work we aimed to compare intro and beyond-intro students' ideas about uncertainty.

Journal
Physical Review Physics Education Research
Date Published

Instructing nontraditional physics labs: Toward responsiveness to student epistemic framing

Author
M. Sundstrom
R.K. Fussell
A.M. Phillips
M. Akubo
S.E. Allen
D. Hammer
R.E. Scherr
N.G. Holmes
Abstract

[This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Research on nontraditional laboratory (lab) activities in physics shows that students often expect to verify predetermined results, as takes place in traditional lab activities. This understanding of what is taking place, or epistemic framing, may impact students' behaviors in the lab, either productively or unproductively.

Journal
Physical Review Physics Education Research
Date Published

Comparing introductory and beyond-introductory students reasoning about uncertainty

Cornell Affiliated Author(s)
Author
Emily Stump
Mark Hughes
Gina Passante
N. Holmes
Abstract

Uncertainty is an important concept in physics laboratory instruction. However, little work has examined how students reason about uncertainty beyond the introductory (intro) level. In this work we aimed to compare intro and beyond-intro students’ ideas about uncertainty. We administered a survey to students at 10 different universities with questions probing procedural reasoning about measurement, student-identified sources of uncertainty, and predictive reasoning about data distributions.

Journal
Physical Review Physics Education Research
Date Published

Instructing nontraditional physics labs: Toward responsiveness to student epistemic framing

Author
Meagan Sundstrom
Rebeckah Fussell
Anna Phillips
Mark Akubo
Scott Allen
David Hammer
Rachel Scherr
Natasha Holmes
Abstract

This paper is part of the Focused Collection on Instructional labs: Improving traditions and new directions.] Research on nontraditional laboratory (lab) activities in physics shows that students often expect to verify predetermined results, as takes place in traditional lab activities. This understanding of what is taking place, or epistemic framing, may impact students’ behaviors in the lab, either productively or unproductively.

Journal
Physical Review Physics Education Research
Date Published

New perspectives on student reasoning about measurement uncertainty: More or better data

Cornell Affiliated Author(s)
Author
Andy Schang
Matthew Dew
Emily Stump
N. Holmes
Gina Passante
Abstract

Uncertainty is an important and fundamental concept in physics education. Students are often first exposed to uncertainty in introductory labs, expand their knowledge across lab courses, and then are introduced to quantum mechanical uncertainty in upper-division courses. This study is part of a larger project evaluating student thinking about uncertainty across these contexts.

Journal
Physical Review Physics Education Research
Date Published