Netherlands

Humans extract useful abstractions of the world from noisy sensory data. Serial reproduction allows us to study how people construe the world through a paradigm similar to the game of telephone, where one person observes a stimulus and reproduces it for the next to form a chain of reproductions. Past serial reproduction experiments typically employ a single sensory modality, but humans often communicate abstractions of the world to each other through language. To investigate the effect language on the formation of abstractions, we implement a novel multimodal serial reproduction framework by asking people who receive a visual stimulus to reproduce it in a linguistic format, and vice versa. We ran unimodal and multimodal chains with both humans and GPT-4 and find that adding language as a modality has a larger effect on human reproductions than GPT-4&#39;s. This suggests human visual and linguistic representations are more dissociable than those of GPT-4.

**Authors:**

Sreejan Kumar: Princeton University; Raja Marjieh: Princeton University; Byron Zhang: Princeton University; Declan Campbell: Princeton University; MICHAEL Y HU: New York University; Umang Bhatt: New York University; Brenden Lake: NYU; Tom Griffiths: Princeton University 

CogSci 2024

Comparing Abstraction in Humans and Machines Using Multimodal Serial Reproduction

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Humans extract useful abstractions of the world from noisy sensory data. Serial reproduction allows us to study how people construe the world through a paradigm similar to the game of telephone, where one person observes a stimulus and reproduces it for the next to form a chain of reproductions. Past serial reproduction experiments typically employ a single sensory modality, but humans often communicate abstractions of the world to each other through language. To investigate the effect language on the formation of abstractions, we implement a novel multimodal serial reproduction framework by asking people who receive a visual stimulus to reproduce it in a linguistic format, and vice versa. We ran unimodal and multimodal chains with both humans and GPT-4 and find that adding language as a modality has a larger effect on human reproductions than GPT-4's. This suggests human visual and linguistic representations are more dissociable than those of GPT-4.

**Authors:**

Sreejan Kumar: Princeton University; Raja Marjieh: Princeton University; Byron Zhang: Princeton University; Declan Campbell: Princeton University; MICHAEL Y HU: New York University; Umang Bhatt: New York University; Brenden Lake: NYU; Tom Griffiths: Princeton University 

technical paper

We investigate whether more accurate representation of syntactic
information in Transformer-based language models is
associated with better alignment to brain activity. We use fMRI
recordings from a large dataset (MOUS) of a Dutch sentence
reading task, and perform Representational Similarity Analysis
to measure alignment with 2 mono- and 3 multilingual
language models. We focus on activity in a region known
for syntactic processing (the Left posterior Medial Temporal
Gyrus). We correlate model-brain similarity scores with the
accuracy of dependency structures extracted from model internal
states using a labelled structural probe. We report three key
findings: 1) Accuracy of syntactic dependency representations
correlates with brain similarity, 2) The link between brain similarity
and dependency accuracy persists regardless of sentence
complexity, although 3) Sentence complexity decreases dependency
accuracy while increasing brain similarity. These results
highlight how interpretable, linguistic features such as syntactic
dependencies can mediate the similarity between language
models and brains

**Authors:**

Abraham Jacob Fresen: University of Amsterdam; Rochelle Choenni: ILLC; Micha Heilbron: University of Amsterdam; Willem Zuidema: University of Amsterdam; Marianne de Heer Kloots: Institute for Logic, Language and Computation

Language Models That Accurately Represent Syntactic Structure Exhibit Higher Representational Similarity To Brain Activity

Large scale neural models show impressive performance across a wide array of linguistic tasks. Despite this they remain, largely, black-boxes - learning vector-representations of their input that prove difficult to interpret. This limits our ability to understand what they learn, and when the learn it, or characterise why they often fail to generalise systematically. To address this we introduce a novel approach to interpretability that looks at the mapping a model learns from sentences to representations as a kind of language in its own right. In doing so we introduce a set of information-theoretic measures that quantify how structured a model's representations are with respect to its input, and when during training that structure arises. Our measures are fast to compute, grounded in linguistic theory, and can predict which models will generalise best based on their representations. We use these measures to describe two distinct phases of training a transformer: an initial phase of in-distribution learning which reduces task loss, then a second stage where representations becoming robust to noise. Generalisation performance begins to increase during this second phase, drawing a link between generalisation and robustness to noise. Finally we look at how model size affects the structure of the representational space, showing that larger models ultimately compress their representations more than their smaller counterparts.

**Authors:**

Henry Conklin: University of Edinburgh; Kenny Smith: University of Edinburgh

Deep Learning as Language Emergence: Noise Drives Generalisation

Cognitive control is thought to regulate the conflict between stability---maintaining the current task in the face of distraction---and flexibility---switching to a new task of greater priority. However, evidence conflicts regarding when and to what extent stability and flexibility trade-off. A normative theory of flexibility and stability may help clarify when and why we should expect such trade-offs to occur. Towards such a theory, we model task-switching as a problem of decision-making under uncertainty, in which the decision-maker must simultaneously infer both the identity of a stimulus and the task governing the correct response to that stimulus. We find that optimal behavior is either extremely stable or extremely flexible, but not both, indicating a normative basis for a trade-off between the two. However, we also show that a sub-optimal but more realistic decision-maker exhibits behavior between these two extremes, and more closely resembles experimental data.

**Authors:**

Seth Madlon-Kay: Duke University; John Pearson: Duke University; Tobias Egner: Duke University

Optimal decision-making under task uncertainty: a computational basis for cognitive stability versus flexibility

Our choices often require us to prioritize some features of our rich sensory experience over others. Past work suggests that humans solve this problem by focusing on relevant information while discarding that which is irrelevant. Yet learning which features to prioritize requires extensive experience. Moreover, features that are irrelevant now may become relevant in the future. One way to address these issues is by sampling individual richly encoded experiences from episodic memory. Here we hypothesize that episodic memory is used to guide decisions based on multiple features of past events. We test this hypothesis using an experiment in which participants made choices about the value of features that were present in multiple past experiences. We find evidence suggesting that participants used episodic memories to flexibly access features of past events during decision making. Overall, these results suggest that episodic memory promotes adaptive decisions when knowledge of multiple features is necessary.

**Authors:**

Jonathan Nicholas: New York University; Marcelo G Mattar: New York University

Humans use episodic memory to access features of past experience for flexible decision making

This paper tests the similarity in neural responses across repeated words and morphosyntactic features both within and between two languages. Prior work using priming has revealed robust cross-linguistic lexical effects and effects for shared grammatical form, such as argument structure; these methods have been less successful when applied to morphosyntactic features. Combining machine-learning based neural decoding with EEG data collected from Korean-English bilinguals we, first, replicate prior work showing successful classification of lexical items from EEG signals. We then extend this to demonstrate successful classification of morphosyntactic features of number and tense. Finally, we find that EEG decoding in one language does not successfully generalize to another, even when temporal differences are considered. Taken together, these results point to stable EEG representations for lexical items and morphosyntactic features, but suggest that these representations are different between the two languages investigated here.

**Authors:**

Jeonghwa Cho: University of Michigan; Jonathan R. Brennan: University of Michigan

Neural decoding of words and morphosyntactic features within and across languages

The ability to exert cognitive control is central to human brain function, facilitating goal-directed task performance. However, humans exhibit limitations in the duration over which they can exert cognitive control---a phenomenon referred to as cognitive fatigue. This study explores a computational rationale for cognitive fatigue in continual learning scenarios: cognitive fatigue serves to limit the extended performance of one task to avoid the forgetting of previously learned tasks. Our study employs a meta-learning framework, wherein cognitive control is optimally allocated to balance immediate task performance with forgetting of other tasks. We demonstrate that this model replicates common patterns of cognitive fatigue, such as performance degradation over time and sensitivity to reward. Furthermore, we discuss novel predictions, including variations in cognitive fatigue based on task representation overlap. This approach offers a novel perspective on the computational role of cognitive fatigue in neural systems.

**Authors:**

Yujun Li: Peking University; Rodrigo Carrasco-Davis: UCL; Younes Strittmatter: Brown University; Stefano Sarao Mannelli: UCL; Sebastian Musslick: Brown University

A meta-learning framework for rationalizing cognitive fatigue in neural systems

Social media is often used as a platform where individuals engage in debate regarding topics that are important to them. Not all arguments are equally convincing, and whilst a given argument may be persuasive to some people, it is often seen as inadequate by others. We are interested in both the individual and argument level differences that make ‘everyday’ arguments such as those on social media persuasive. In a replication of our Everyday Belief Bias Task (Deans-Browne & Singmann, 2024), we investigate this question using a paradigm that consists of two parts. In the first part, we measure participant’s individual beliefs about eight claims each referring to a political topic (e.g., Abortion should be legal). In the second part, participants rated an argument for each of these claims that was deemed as either good, inconsistent (containing internal inconsistencies), or authority-based (being centered around appeals to authority). We replicated the belief consistency effect – participants preferred arguments that were also in line with their beliefs. We also found that authority-based arguments were rated as worse than inconsistent arguments, and that both types of arguments were rated as worse than good arguments. The implications are first that people do not evaluate arguments independently of the background beliefs held about them. Secondly, people are willing to ignore inconsistencies in arguments more than they are willing to accept the endorsement of authority figures as adequate evidence for arguments.

**Authors:**

Calvin Deans-Browne: University College London; Alexandra Băitanu: University College London; Yuliya Dubinska Ms: UCL; Henrik Singmann: UCL

Inconsistent Arguments are Perceived as Better Than Appeals to Authority: An Extension of the Everyday Belief Bias

The complexity of everyday choices make them difficult to formally study. We address this challenge by constructing a dataset of over 100K real-life decision problems based on a combination of social media and large-scale survey data. Using large language models (LLMs) for automated coding, we are able to extract hundreds of choice attributes at play in these problems and map them onto a common representational space. This representation allows us to quantify both the content (e.g. broader themes) and the structure (e.g. specific tradeoffs) inherent in everyday choices. We also present subsets of these decision problems to human participants, and find consistency in choice patterns, allowing us to predict naturalistic choices with established decision models. Overall, our research provides new insights into the attributes and tradeoffs that underpin important life choices. In doing so, our work shows how LLM-based structure extraction can be used to study real-world cognition and behavior.

**Authors:**

Sudeep Bhatia: University of Pennsylvania; Simon T. van Baal: University of Vienna; Lukasz Walasek: University of Warwick

The Structure of Everyday Choice: Insights from 100K Real-life Decision Problems

Content-curating algorithms provide a crucial service for social media users by surfacing relevant content, but they can also bring about harms when their objectives are misaligned with user values and welfare. Yet, potential behavioral consequences of this alignment problem remain understudied in controlled experiments. In a preregistered, two-wave, collaborative filtering experiment, we demonstrate that small changes to the metrics used for sampling and ranking posts affect the beliefs people form. Our results show observable differences in two types of outcomes within statisticized groups: belief accuracy and consensus. We find partial support for hypotheses that the recently proposed approaches of "bridging-based ranking" and "intelligence-based ranking" promote consensus and belief accuracy, respectively. We also find that while personalized, engagement-based ranking promotes posts that participants perceive favorably, it simultaneously leads those participants to form more polarized and less accurate beliefs than any of the other algorithms considered.

**Authors:**

Jason W. Burton: Copenhagen Business School; Stefan M Herzog: Max Planck Institute for Human Development; Philipp Lorenz-Spreen: Max Planck Institute for Human Development

Simple changes to content curation algorithms affect the beliefs people form in a collaborative filtering experiment

Category fluency is a widely studied cognitive task. Two major competing accounts have been proposed as the underlying retrieval mechanism: an optimal foraging process deliberately searching through memory (Hills et al., 2012) and a random walk sampling from a semantic network (Abbott et al., 2015). Evidence for both accounts has centered around predicting human patch switches, where both existing models of category fluency produce paradoxically identical results. We begin by peeling back the assumptions made by existing models, namely that each named exemplar only depends on the previous exemplar, by (i) adding an additional bias to model the category transition probability directly and (ii) relying on a large language model to predict based on the full prior exemplar sequence. Then, we present evidence towards resolving the disagreement between different models of foraging by reformulating them as sequence generators. For evaluation, we compare generated category fluency runs to a bank of humanwritten sequences by utilizing a metric based on n-gram overlap. We find that category switch predictors do not necessarily produce human-like sequences; rather, the additional biases used by the Hills et al. (2012) model are required to improve generation quality, which is further improved by our category modification. Even generating exclusively with an LLM requires an additional global cue to trigger the patch switching behavior during production. Further tests on only the search process on top of the semantic network highlight the importance of deterministic search in replicating human behavior.

**Authors:**

David Heineman: Georgia Institute of Technology; Reba Koenen: Georgia Institute of Technology; Sashank Varma: Georgia Tech

Premium content

Downloads

Next from CogSci 2024

Language Models That Accurately Represent Syntactic Structure Exhibit Higher Representational Similarity To Brain Activity

Similar lecture

Human-Like Geometric Abstraction in Large Pre-trained Neural Networks