Netherlands

Emotions are altered not only by external events, but also by the emotions of the people in front of us. While these emotional linkages are considered an important aspect of empathy, it is unclear how they relate to cognitive empathy, that is, how we perceive others’ emotions. We investigated the relationship between emotional linkage and emotional cognition using a joint task in which two participants simultaneously estimate their partner’s emotion while measuring their heart rate. We also collected data from the two observers because, in reality, in addition to understanding the emotions of the interacting partner, it is sometimes necessary to understand the emotions of a non-interacting person from a third-person perspective. The results suggest that, for players, their own heart rate is related to perceived partner emotion, and for observers, the degree of heart rate synchrony between observers is related to perceived player emotion. Capturing emotional cognition in a joint task requires consideration of both individual emotion and interpersonal emotional linkages.

**Authors:**

Aiko Murata: Nippon Telegraph and Telephone Corporation; Shiro Kumano: Nippon Telegraph and Telephone Corporation

CogSci 2024

Relationship between emotional linkages and perceived emotion during a joint task

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

poster

According to previous research, gestural input with semantic relation to the spoken part of an utterance contributes to addressees’ understanding. In this paper on explanatory dialogues, we reverse the focus onto the use of different forms of deictic gestures by different explainers in relation to their monitoring of explainees’ understanding during board game explanations in the absence of a visible explanandum. Our analysis of 24 dyadic interactions reveals that monitoring explainees’ complete understanding is not related to a decrease in explainers’ frequency of pointing on the shared referential space. In addition, we provide insights into intra-individual differences within each explainer interacting with three different explainees.

**Authors:**

Stefan Lazarov: Paderborn University; Angela Grimminger: Paderborn University

Variations in explainers’ gesture deixis in explanations related to the monitoring of explainees’ understanding

Transcription with Incremental Presentation of the Stimulus (TIPS) is a novel approach relying on micro-behaviours proposed by Colarusso and colleagues (2023) to study users’ cognition with data visualizations. The study in this paper has two primary objectives: (a) investigate whether TIPS can measure an individual’s competence with data visualizations; and (b) explore the potential enhancement of TIPS measures by normalizing them with the individual’s performance on tests of visuo-spatial abilities and memory capacity., We test 30 participants with different expertise and cognitive skills. Results reveal that TIPS provides some promise for individual competence assessment, but only when normalized with the individual’s performance on a test of rigid transformation of mental images. Other tests measuring visuospatial abilities or memory capacity did not produce effective normalizations.

**Authors:**

Fiorenzo Colarusso: University of Sussex; Peter Cheng: University of Sussex; Ronald Grau: University of Sussex; Grecia Garcia Garcia: University of Sussex; Daniel Raggi: University of Cambridge; Mateja Jamnik: University of Cambridge

Decoding Expertise: Exploring Cognitive Micro-Behavioural Measurements for Visualization Competence

Modeling the user state of knowledge in information seeking is crucial for the
design of user-adaptive search systems. Gaze data provides a window to the
cognitive processes involved with reading in information seeking. However, the
representation of the user state of knowledge in information retrieval systems
like search engines, has not been well-informed by cognitive perspectives. To
that end, we reanalyse an eye-tracking dataset and construct mixed-effects user
models to specify which measurements of gaze activities (i.e. gaze metrics
captured by eye trackers) are reflective of user knowledge in task-based
reading. Specifically, the user model examines the relationships among
user-perceived prior knowledge, search task difficulty, gaze metrics and search
performance. Research findings suggest that the variability in saccadic eye
movements, measured by the standard deviation of absolute saccadic directions,
answers. Implications for the design of search systems adaptable to the user's
state of knowledge are discussed.

**Authors:**

Ying-Hsang Liu: Uppsala University; Andreas Nürnberger: Otto von Guericke University; Jenny Rettstatt: Chemnitz University of Technology; Marco Ragni: TU Chemnitz

Saccadic Eye Movements and Search Task Difficulty as Basis of Modelling User Knowledge in Information Seeking

Pragmatic implicature derivation presupposes that the cooperative principle is observed and critically depends on interlocutors expecting each other to behave cooperatively. It is much less clear, however, whether people extend this assumption to communication with artificial agents. People might therefore not draw the same pragmatic inferences when interacting with an artificial agent as they would with other conversationally competent humans, even if the agent is in principle believed to be similarly competent. In our study, we ask participants to interpret messages in a pragmatic reference game which they are told were generated by ChatGPT. Additionally, participants report whether they believe ChatGPT to be capable of the reasoning needed to select the optimal message. We observe a noteworthy discrepancy: in the reference game, participants interpret ChatGPT's messages less pragmatically than those of another adult human, but in the post-test questionnaire, they overwhelmingly rate ChatGPT's pragmatic ability very highly.

**Authors:**

Alexandra Mayn: Saarland University; Jia Loy: Saarland University; Vera Demberg: Saarland University

Capable but not cooperative? Perceptions of ChatGPT as a pragmatic speaker

Humans can flexibly bind familiar functional roles to novel entities in their environment. For example, children who have the concept of ``goal posts'' can bind this abstract role to two hats placed on the street. In doing so, they can port over existing expectations of ``goal posts" for the duration of the game. In this paper, we seek to explore artificial agents' ability to perform flexible role binding and rebinding. To this end, we designed a Gridworld navigation game and tested a popular CNN-based agent which has had success in other tasks involving visual and spatial state spaces (e.g. Atari or Minigrid). To our surprise, we found that while this architecture was capable of over-fitting to the training set, it was not able to learn flexible role binding without intervention. We ultimately show that with carefully engineered data augmentation techniques, our artificial agent is able to learn the task. This suggests that the diversity of the training dataset was a limiting factor.

**Authors:**

Brian Pennisi: New York University; Todd M Gureckis: New York University; Rheza Budiono: New York University; Mark K Ho: Stevens Institute of Technology

Investigating Flexible Role Binding in AI Agents

This study investigated visual similarity modeling of Chinese characters across proficiency levels. Similarity models of Chinese characters from previous research were evaluated against human judgments. Extended tree analysis highlighted the importance of characters’ configurations in expert similarity judgments, with character components as support. In contrast, Chinese learners’ and novices’ criteria for similarity judgments were less clear as indicated by the low proportion of variance that their extended tree models accounted for. Although cophenetic correlation coefficients revealed only weak correlations between similarity models and human judgments, their scores on configurations and components pointed to the nonlinear development of visual expertise. The computational challenges in replicating human perception and judgments were discussed and future directions, including machine learning with larger datasets for improved model performance, were proposed.

**Authors:**

Zhuqian Zhou: Teachers College, Columbia University; James E. Corter: Columbia University

Visual Similarity Modeling of Chinese Characters Across Natives, Second-Language Learners, and Novices

Gonering & Corina (2023) argued that abstractions over visual scenes (i.e. schemas or situation models) provide a semantic scaffold for acquiring verb argument structures. This work intends to support their argument through systematic meta-analysis of 158 fMRI studies of verb processing (from NeuroSynth) and 208 fMRI studies of visual event processing (from NeuroQuery). Activation Likelihood Estimation using the Neuroimaging Meta-Analysis Research Environment package (Salo et al., 2022) will be used to assess separate and overlapping regions of significant neural activity (cluster-level family-wise error corrected using Monte Carlo method). Meta-analyses on additional visual scene and verb processing studies from NeuroSynth and NeuroQuery are also underway. We further intend to show that a hierarchical Bayesian model can learn verb argument structures from input statistics, even when they deviate from strong prior event semantic knowledge.

**Authors:**

Brennan T Gonering: University of California, Davis; Zoey Liu: University of Florida

Real world event schemas offer modality-independent conceptual bases for verb argument structures

This paper explores how speakers choose between two utterance alternatives with similar syntactic properties and distinct yet related meanings. In doing so, we consider the interaction of two speaker pressures: accessibility of lexical items, and pressure to be informative at early stages in the utterance; the latter of which is explicitly predicted by an incremental Rational Speech Act (IRSA) model. A significant effect of accessibility on speaker choice is established based on the results of an online spoken production task, which asked speakers to describe the relationship between two objects using a provided verb. We find that foregrounding objects leads speakers to mention them earlier, even when the resulting utterance has a different meaning. This task is then extended to an interactive context, where speakers describe images while paired with a partner who must click on the corresponding visual scene based on the description. In this goal-oriented context, IRSA predicts more informative words to come earlier in the sentence. The interactive task does not show a significant effect of foregrounding or the predicted informativity effect.

**Authors:**

Emily Goodwin: Stanford University; Judith Degen: Stanford University

Production of Syntactic Alternations Displays Accessibility But Not Informativity Effects

Infants learn actively in their environments, shaping their own learning curricula. They selectively pay attention, orient themselves, and later move to particular stimuli and entities. As a result, infants master progressively more complex tasks, and exhibit physical and agentive knowledge. Essentially, they learn about their environments' affordances, that is, how their behavior can affect the environment under local circumstances. Here we model this type of behavior by means of a deep learning architecture. The architecture mediates between global cognitive map exploration and local affordance learning. Inference processes actively move the simulated agent towards regions with high uncertainty, where they expect knowledge gain about local affordances. We contrast three measures of uncertainty: predicted uncertainty of a model, standard deviation between the means of several models (SD), and the Jensen-Shannon Divergence (JSD) between several models. We show that the first measure gets fooled by aleatoric uncertainty inherent in the environment, while the two other measures focus learning on epistemic uncertainty. However, SD yields behavior that is attracted by extreme behavior regimes, while JSD shows the most balanced epistemic exploration strategy. While our brain may have other means to implement the proposed processes, on computational levels our model suggests three key ingredients for coordinating the active generation of learning curricula: (1) Navigation behavior needs to be coordinated with local motor behavior for enabling the active search for affordances in an environment. (2) Affordances need to be encoded locally for acquiring generalized knowledge about affordances. (3) Effective active affordance learning mechanism may use density comparison techniques for estimating expected knowledge gain. Future work may seek collaborations with developmental psychology to model active play in children in more realistic scenarios.

**Authors:**

Fedor Scholz: University of Tübingen; Erik Ayari: University of Tübingen; Johannes Bertram: University of Tübingen; Martin V. Butz: University of Tübingen

Quick and Accurate Affordance Learning

German has two demonstrative series, the der (die, das) series and the dieser (diese, dieses) series. Both have been claimed to be topic shifters, taking up a non-topical antecedent and promoting it to topichood. However, der can form topical referential chains, while dieser cannot. We operationalize discourse topichood via questions and provide evidence from a corpus study and an acceptability study that while dieser is indeed sensitive to topichood and avoids topical antecedents, der is compatible with topical antecedents. We hypothesize that only dieser is a discourse topic shifter, while der marks a sentence topic.

**Authors:**

Timo Buchholz: Universität zu Köln; Klaus von Heusinger: University of Cologne

Premium content

Downloads

Next from CogSci 2024

Variations in explainers’ gesture deixis in explanations related to the monitoring of explainees’ understanding

Similar lecture

A high-dimensional semantic space of emotion representations support circumplex structure