Netherlands

Humans learn and generate languages, music, games, and seemingly limitless varieties of other structures across domains. Unlike many AI systems, we often do so from little data. How do we learn such large varieties of richly structured representations so efficiently? One possibility is that people ``learn by programming,&#39;&#39;  synthesizing data-generating algorithms to explain what we observe. We examine the nature and origins of this learning mechanism in adults, children, and nonhuman primates (macaque monkeys), using a highly unconstrained sequence prediction task. Although adults and children quickly learn many richly structured sequences, monkeys learn only the simplest sequences (e.g. lines). We test multiple learning models, finding that adults are best explained by a ``Language of Thought&#39;&#39;-like program-learning model and monkeys by a simpler extrapolation strategy. Children exhibit varied learning strategies but are best fit in aggregate by an intermediately expressive model. Paper available at https://sites.google.com/view/patternlearning.

**Authors:**

Tracey Mills: MIT; Nicole Coates: Massachusetts Institute of Technology; Alessandra A. Silva: University of Wisconsin-Madison; Stephen Ferrigno: University of Wisconsin-Madison; Laura Schulz: Massachusetts Institute of Technology; Josh Tenenbaum: MIT; Samuel J. Cheyette: MIT

CogSci 2024

Connecting the dots: a comparative and developmental analysis of spatiotemporal pattern learning

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Humans learn and generate languages, music, games, and seemingly limitless varieties of other structures across domains. Unlike many AI systems, we often do so from little data. How do we learn such large varieties of richly structured representations so efficiently? One possibility is that people ``learn by programming,''  synthesizing data-generating algorithms to explain what we observe. We examine the nature and origins of this learning mechanism in adults, children, and nonhuman primates (macaque monkeys), using a highly unconstrained sequence prediction task. Although adults and children quickly learn many richly structured sequences, monkeys learn only the simplest sequences (e.g. lines). We test multiple learning models, finding that adults are best explained by a ``Language of Thought''-like program-learning model and monkeys by a simpler extrapolation strategy. Children exhibit varied learning strategies but are best fit in aggregate by an intermediately expressive model. Paper available at https://sites.google.com/view/patternlearning.

**Authors:**

Tracey Mills: MIT; Nicole Coates: Massachusetts Institute of Technology; Alessandra A. Silva: University of Wisconsin-Madison; Stephen Ferrigno: University of Wisconsin-Madison; Laura Schulz: Massachusetts Institute of Technology; Josh Tenenbaum: MIT; Samuel J. Cheyette: MIT

technical paper

Studies of meaning in human and primate communication face, in principle, similar methodological problems. In both cases, meaning is not observable directly, but must be inferred from more indirect sources, such as directly observable behavior. Recent work in probabilistic cognitive modeling of language use has therefore developed methods of inferring latent se- mantic meaning through the lens of a probabilistic model of language use. In this paper, we explore how to adapt such an approach for insightful investigations of primate communication. Towards this end, we develop a suitable probabilistic model of processes that generate communicative behavior by making use of functionally specified latent meaning representations. As a proof of concept, we apply this model to a rich, annotated data set of orangutan communicative dyadic interaction and conclude that explicit probabilistic modeling can provide additional insights for the study of animal communication pertaining to the context-dependent nature of signals and the gradual evolution of human communication systems.

**Authors:**

Michael Franke: University of Tübingen; Manuel Bohn: Max Planck Institute for Evolutionary Anthropology; Marlen Fröhlich: University of Tübingen

Latent meaning representations in great-ape gestural communication

Languages tend to describe who is doing what to whom by placing subjects before objects. This bias for agents is reflected in event cognition: agents capture more attention than patients in human adults and infants. We investigated whether this agent preference is unique to humans. We presented Guinea baboons (Papio papio, N = 13) with a change detection paradigm with chasing animations. The baboons had to respond to a colour change which was applied to either the chaser/agent or the chasee/patient. They were faster to detect a change to the chaser than to the chasee, which cannot be explained by low-level features in our stimuli. Our study suggests that baboons show an agent preference similar to human infants and adults. This may be an evolutionarily old mechanism that is shared between humans and other primates, which could have become externalised in language as a tendency to place the subject first.

**Authors:**

Floor Meewis: CNRS / Aix-Marseille University; Joel Fagot: CNRS / Aix-Marseille University; Nicolas Claidiere: CNRS / Aix-Marseille University; Isabelle Dautriche: CNRS / Aix-Marseille University

Guinea baboons (Papio papio) show an agent preference in chasing interactions

Video recording is a widely used tool for studying animal behavior, especially in fields such as primatology. Primatologists rely on video data to analyze and research topics such as social grooming to uncover subtle mechanisms behind complex social behavior and structures. Insights into these social behaviors may provide us with a better understanding of our closest living relatives, but also new theories and insights into our own behavior. However, analyzing this type of data using manual annotation is currently a time-consuming task. Here we present an end-to-end pipeline to track chimpanzee (Pan troglodytes) poses using DeepLabCut (DLC) which then serves as input to a support vector machine. This classifier was trained to detect role transitions within grooming interactions. We replicate a recent study showing that DLC has usability value for chimpanzee data collected in natural environments. Our combined method of tracking and classification is remarkably successful in detecting the presence of grooming, indicating the directionality and a change in turn with an accuracy above 86% on unseen videos. We can identify particular pose features used in the classification of grooming, which will contribute to the exploration of turn-taking dynamics on a scale that would otherwise be difficult to attain with traditional methods. Finally, our pipeline can in principle be applied to recognize a variety of other socially interactive behaviors that are largely recognizable by (joint) postural states.

**Authors:**

Yana van de Sande MSc.: Radboud University; Wim Pouw: Radboud University Nijmegen; Lara M Southern: Osnabrück University

Automated Recognition of Grooming Behavior in Wild Chimpanzees

An important aspect of language comprehension is learning and generalizing complex lexical relations. For instance, having learned that the phrase preserve cucumbers predicts vinegar and that preserve berries predicts dehydrator, one should be able to infer that the novel phrase preserve peppers is more compatible with vinegar, because pepper is more similar to cucumber. We studied the ability to perform such (compositional) generalization in distributional models trained on an artificial corpus with strict semantic regularities. We found that word-encoding models failed to learn the multi-way lexical dependencies. Recurrent neural networks learned those dependencies but struggled to generalize to novel combinations. Only mini GPT-2, a minified version of the Transformer GPT-2, succeeded in both learning and generalization. Because successful generalization in our tasks requires capturing the relationship between a phrase and a word, we argue that mini GPT-2 acquired hierarchical representations that approximate phrase structure. Our results show that, compared to older models, Transformers are architecturally advantaged to perform compositional generalization.

**Authors:**

Shufan Mao: University of Illinois at Urbana Champaign; Philip A Huebner: Pattern; Jon Willits: University of Illinois at Urbana-Champaign

Compositional Generalization in Distributional Models of Semantics: Transformer-based Language Models are Architecturally Advantaged

Prediction error, both at the level of sentence meaning and at the level of the next presented word, has been shown to successfully account for N400 amplitudes. Here we investigate whether people differ in the representational level at which they implicitly predict upcoming language. We computed a measure of prediction error at the level of sentence meaning (termed semantic update) and a measure of prediction error at the level of the next word (surprisal). Both measures significantly accounted for N400 amplitudes even when the other measure was controlled for. Most important for current purposes, both effects were significantly negatively correlated. Moreover, random-effects model comparison showed that individuals differ in whether their N400 amplitudes are driven by semantic update only, by surprisal only, or by both, and that the most common model in the population was either semantic update or the combined model but clearly not the pure surprisal model.

**Authors:**

Milena Rabovsky: University of Potsdam; Alessandro Lopopolo Lopopolo: University of Potsdam; Daniel J. Schad: HMU Health and Medical University Potsdam

Interindividual differences in predicting words versus sentence meaning: Explaining N400 amplitudes using large-scale neural network models

Recent studies link surprisal —a measure of conditional probability of words in context—to the N400 component size in event-related potentials (ERP), supporting a role for predictive coding in language comprehension. An alternative account argues that N400 variations are better explained by a retrieval mechanism sensitive to the semantic similarity between a word and its preceding context. Because jokes often rely on the presence of unexpected words that relate to the prior context multiple ways, they afford observation of the relative importance of contextual predictability and contextual similarity. We employed state-of-the-art machine learning to assess the surprisal and contextual semantic similarity of critical words in jokes and control stimuli. Using regression models to predict ERP, we found contextual similarity best explains N400 and P600 responses, supporting the semantic similarity account. Additionally, jokes elicit enhanced N400 and P600 responses that go beyond that attributable to their surprisal and contextual semantic similarity.

**Authors:**

Haoyin Xu: University of California, San Diego; Masaki Nakanishi: 1986; Seana Coulson: UC San Diego

Revisiting Joke Comprehension with Surprisal and Contextual Similarity: Implication from N400 and P600 Components

Surprisal values from large language models (LLMs) have been used to model the amplitude of the N400.
This ERP component is sensitive not only to contextual word expectancy but also to semantic association, such that unexpected but associated words do not always induce an N400 increase.
While LLMs are also sensitive to association, it remains unclear how they behave in these cases.
Moreover, another ERP component, the P600, has shown graded sensitivity to plausibility-driven expectancy, while remaining insensitive to association; however, its relationship to LLM surprisal is not well researched yet.
In an rERP analysis, we evaluate surprisal values of two unidirectional transformers on their ability to model N400 and P600 effects observed in three German ERP studies isolating the effects of association, plausibility, and expectancy.
We find that surprisal predicts an N400 increase for associated but implausible words, even when no such increase was observed in humans.
Furthermore, LLM surprisal accounts for P600 effects elicited by violations of selectional restrictions, but captures neither P600 effects from more subtle script knowledge violations nor graded P600 modulations.
The results of our investigation call into question the extent to which LLM surprisal offers an accurate characterisation of the functional generators of either the N400 or P600.

**Authors:**

Benedict Krieger: Saarland University; Harm Brouwer: Tilburg University; Christoph Aurnhammer: Saarland University; Matthew W Crocker: Saarland University

On the limits of LLM surprisal as functional Explanation of ERPs

Speech perception involves storing and integrating sequentially presented items. Recent work in cognitive neuroscience has identified temporal and contextual characteristics in humans’ neural encoding of speech that may facilitate this temporal processing. In this study, we simulated similar analyses with representations extracted from a computational model that was trained on unlabelled speech with the learning objective of predicting upcoming acoustics. Our simulations revealed temporal dynamics similar to those in brain signals, implying that these properties can arise without linguistic knowledge. Another property shared between brains and the model is that the encoding patterns of phonemes support some degree of cross-context generalization. However, we found evidence that the effectiveness of these generalizations depends on the specific contexts, which suggests that this analysis alone is insufficient to support the presence of context-invariant encoding.

**Authors:**

Oli Danyi Liu: School of Informatics; Hao Tang: School of Informatics; Naomi H Feldman: University of Maryland; Sharon Goldwater: University of Edinburgh

A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech

It has long been assumed that infants' ability to discriminate between languages stems from their sensitivity to speech rhythm, i.e., organized temporal structure of vowels and consonants in a language. However, the relationship between speech rhythm and language discrimination has not been directly demonstrated. Here, we use computational modeling and train models of speech perception with and without access to information about rhythm. We test these models on language discrimination, and find that access to rhythm does not affect the success of the model in replicating infant language discrimination results. Our findings challenge the relationship between rhythm and language discrimination, and have implications for theories of language acquisition.

**Authors:**

Ruolan Leslie Famularo: University of Maryland; Ali Aboelata: University of Maryland; Thomas Schatz: Aix-Marseille Université; Naomi H Feldman: University of Maryland

Language Discrimination May Not Rely on Rhythm: A Computational Study

This study delves into how various musical factors influence the experience of auditory illusions, building on Diana Deutsch's scale illusion experiments and subsequent studies. Exploring the interaction between scale mode and timbre, this study assesses their influence on auditory misperceptions, while also considering the impact of an individual's musical training and ability to discern absolute pitch. Participants were divided into non-musicians, musicians with absolute pitch, and musicians with relative pitch, and were exposed to stimuli modified across three scale modes (tonal, dissonant, atonal) and two timbres (same, different). The findings suggest that scale illusions occur less frequently with different timbres and vary with scale mode. Crucially, the absolute pitch ability appears to have a more significant impact on the perception of illusions than the duration of musical training. This research contributes to understanding the complex interplay between various factors in auditory perception and the mechanisms behind the experience of auditory illusions.

**Authors:**

Ahyeon Choi: Seoul National University; Younyoung Bang: Seoul National University; Jeong Mi Park: Seoul National University; Kyogu Lee: Seoul National University

Premium content

Downloads

Next from CogSci 2024

Latent meaning representations in great-ape gestural communication

Similar lecture

Knowing What Counts for Counting