Netherlands

Emerging folklore in the cognitive sciences suggests that interpretability techniques to reverse-engineer artificial neural networks (ANNs) could speed up discovery and theory-building. For many researchers in psychology, linguistics, neuroscience, and artificial intelligence (AI), the full observability and perturbability of ANNs trained on complex tasks affords a shortcut to domain insights, cognitive theories, neurocognitive models, application improvement, and user safety. Folklore intuitions, however, are typically disconnected from other relevant knowledge. Here we examine these intuitions formally by drawing relevant connections to computational complexity theory. We model interpretability queries computationally and analyze their resource demands for biological/artificial high-level cognition. We prove mathematically that, contrary to folklore, basic circuit-finding queries in classic ANNs are already infeasibly demanding to answer even approximately. We discuss how interdisciplinary integration can mitigate this disconnect and situate the broader implications for the cognitive sciences, the philosophy of AI-fueled discovery, and AI ethics.

**Authors:**

Federico Adolfi: Ernst-Strüngmann Institute for Neuroscience; Martina G. Vilas: Goethe University Frankfurt; Todd Wareham: Memorial University of Newfoundland

CogSci 2024

Complexity-Theoretic Limits on the Promises of Artificial Neural Network Reverse-Engineering

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

CogSci 2023

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

poster

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE**<br>
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25**<br>
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment)<br>
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk)<br>
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture)<br>
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation)<br>

**Friday, July 26**<br>
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address)<br>
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration)<br>
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture)<br>
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture)<br>
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language)<br>

**Saturday, July 27**<br>
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind)<br>
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture)<br>
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture)<br>


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou**  
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT**<br>
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY**<br>
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.
<br><br>

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

We introduce an approach and method that helps explain how humans compare images. We produce Alignment Importance Score (AIS) heatmaps from deep-vision models, focusing on feature maps in the deepest convolutional layer. The AIS reflects a feature-map's unique contribution to the alignment of Deep Neural Network's (DNN) representational geometry and that of humans. We first validate the AIS by showing that prediction of out-of-sample human similarity judgments is improved when constructing representations using only higher-AIS feature maps identified by a training set. We then compute image-specific heatmaps that visually indicate the areas that correspond to feature-maps with higher AIS scores. These maps provide an intuitive explanation of which image areas are more important when it is compared to other images in a cohort. We find that these heatmaps have good correspondence with saliency maps produced by models trained to predict gaze location. However, in some exceptions, meaningful differences emerge, as the relevant dimensions for comparison are not necessarily the most visually salient. To conclude, by using AIS it is possible to improve prediction of human similarity judgments from DNN embeddings, and to depict the relevant information in image space.

**Authors:**

Nhut Truong: University of Trento; Dario Pesenti: University of Trento; Uri Hasson: University of Trento

Explaining Human Comparisons using Alignment-Importance Heatmaps

Prognostic assessment of patients with disorders of consciousness (DoC) remains one of the most challenging problems in contemporary medicine. The long treatment cycle and high costs of treatment are heavy burdens to our society. In this paper, we use deep network to investigate potential indicators of consciousness within brain signals of DoC patients. In the experiments, we study P300 and resting-state Electroencephalogram (rs-EEG) signals of 22 DoC patients to investigate neural correlation between brain signals and the improvement of consciousness. Synergistic integration of P300 and rs-EEG signals demonstrated superior predictive proficiency for cross-subject and cross-paradigm prognosis in DoC, achieving an accuracy of 81.1%. Our investigation is the first known to the literature to combine P300 and rs-EEG signals for analyzing DoC. This novel approach leverages advanced neural network models to elucidate the complex neural patterns associated with DoC, setting a precedent for future research in the field.

**Authors:**

Jingcong Li: South China Normal University; Biao Huang: South China Normal University; Jiahui Pan: South China Normal University; Fei Wang: South China Normal University

An Investigation on EEG-based Prognosis Prediction of Patients with Disorders of Consciousness

Emotion recognition is crucial for enhancing human-computer interaction. Due to considerable individual differences in emotion manifestation, traditional models do not adapt well to new individuals. Moreover, existing algorithms typically focus on identifying a single emotion, overlooking intrinsic connections among multiple emotions. Therefore, we propose a multi-task adversarial domain adaption (MADA) model for EEG-based emotion recognition. First, domain matching is employed to identify the most similar individual from the dataset as the source domain, alleviating individual differences and reducing training time. Subsequently, multi-task learning is utilized to simultaneously classify multiple emotions, capturing their intrinsic connections. Finally, adversarial domain adaption is applied to learn the individual differences between the source and target domains. Cross-subject experiments on the DEAP dataset indicate that our model achieves accuracies of 78.08%, 68.36%, and 69.64% on the valence, arousal, and dominance, respectively, surpassing state-of-the-art methods. This indicates the effectiveness of our model in recognizing multi-dimensional emotions.

**Authors:**

Lina Qiu: South China Normal University; Zuorui Ying: South China Normal University; Weisen Feng: South China Normal University; Jiahui Pan: South China Normal University

Cross-subject EEG Emotion Recognition based on Multitask Adversarial Domain Adaption

Previous studies suggested differences in the temporal unfolding of face processing in the brain between real and virtual faces, starting from 400 ms onwards. However, few studies explicitly compared the early and the late processing stages in real and virtual faces in the same paradigm. Here we conducted an EEG study utilizing real human faces and high-quality realistic virtual agent faces, examining two event-related potentials; the early N170 and the Late Positive Potential (LPP). Our results showed that the N170 response was identical for both types of faces. Regarding the LPP response, the results revealed a proclivity for real human faces to elicit a slightly larger LPP compared to virtual agent faces. These results suggest that although high-quality virtual agent faces can approach the level of higher-order evaluation typically associated with real human faces, human faces remain the most engaging.

**Authors:**

Julija Vaitonytė: Tilburg University; Maryam Alimardani: Tilburg University; Max Louwerse: Tilburg University

Face Processing in Real and Virtual Faces: An EEG Study

The complexity of visual stimuli plays an important role in many cognitive phenomena, including attention, engagement, memorability, time perception and aesthetic evaluation. Despite its importance, complexity is poorly understood and ironically, previous models of image complexity have been quite \textit{complex}. There have been many attempts to find handcrafted features that explain complexity, but these features are usually dataset specific, and hence fail to generalise. On the other hand, more recent work has employed deep neural networks to predict complexity, but these models remain difficult to interpret, and do not guide a theoretical understanding of the problem. Here we propose to model complexity using segment-based representations of images. We use state-of-the-art segmentation models, SAM and FC-CLIP, to quantify the number of segments at multiple granularities, and the number of classes in an image respectively. We find that complexity is well-explained by a simple linear model with these two features across six diverse image-sets of naturalistic scene and art images. This suggests that the complexity of images can be surprisingly simple.

**Authors:**

Tingke Shen: Max Planck Institute for Biological Cybernetics; Surabhi S Nath: Max Planck Institute for Biological Cybernetics; Aenne Brielmann: University of Tubingen; Peter Dayan: Max Planck Institute for Biological Cybernetics

Simplicity in Complexity

Video game playing is an extremely structured domain where algorithmic decision-making can be tested without adverse real-world consequences. While prevailing methods rely on image inputs to avoid the problem of hand-crafting state space representations, this approach systematically diverges from the way humans actually learn to play games. In this paper, we design object-based input representations that generalize well across a number of video games. Using these representations, we evaluate an agent's ability to learn games similar to an infant - with limited world experience, employing simple inductive biases derived from intuitive representations of physics from the real world. Using such biases, we construct an object category representation to be used by a Q-learning algorithm and assess how well it learns to play multiple games based on observed object affordances. Our results suggest that a human-like object interaction setup capably learns to play several video games, and demonstrates superior generalizability, particularly for unfamiliar objects. Further exploring such methods will allow machines to learn in a human-centric way, thus incorporating more human-like learning benefits.

**Authors:**

Abhishek Jaiswal: Indian Institute of Technology Kanpur; Nisheeth Srivastava: Indian Institute of Technology

Learning to Play Video Games with Intuitive Physics Priors

Program induction is an appealing model for human concept learning, but faces scaling challenges in searching the massive space of programs. We propose a computational model capturing two key aspects of human concept learning – our ability to judge how promising a vague, partial hypothesis is, and our ability to gradually refine these vague explanations of observations to precise ones. We represent hypotheses as probabilistic programs with randomness in place of unresolved programmatic structure. To model the evaluation of partial hypotheses, we implement a novel algorithm for efficiently computing the likelihood that a probabilistic program produces the observations. With this, we guide a search process whereby high-entropy, coarse programs are iteratively refined to introduce deterministic structure. Preliminary results on list tasks show orders of magnitude improvement in sample efficiency when augmenting a sampling-based search with likelihood guidance, and intermediate hypotheses were similar to those considered by humans verbalizing their thought processes.

**Authors:**

Maddy L Bowers: MIT; Alexander Lew: MIT; Wenhao Qi: University of California, San Diego; Joshua S Rule: UC Berkeley; Vikash Mansinghka: MIT; Josh Tenenbaum: MIT; Armando Solar-Lezama: Massachusetts Institute of Technology

Concept Learning as Coarse-to-Fine Probabilistic Program Induction

Language models have great potential as cognitive models for studying human language acquisition, but current models are far less data-efficient than human learners. Children acquire language from 100 million words or less, but large language models are trained on trillions of words. We discuss the prospects for improving language models’ developmental plausibility through a meta-analysis of results from the 2023 BabyLM Challenge. BabyLM was a competition that invited participants to train a language model on a 100 million-word corpus including transcribed speech and child-appropriate texts. Results from over 30 submissions showed that new machine learning techniques and increased training iterations yielded models that outperformed leading large language models in grammar, language understanding, and linguistic generalization, while cognitively plausible approaches such as curriculum learning were less effective. We discuss the implications of these and other findings for computational cognitive modeling and explore ideas to ensure future competitions’ contributions to cognitive science.

**Authors:**

Alex Warstadt: ETH Zurich; Aaron Mueller: Northeastern University; Leshem Choshen: IBM; Ethan Gotlieb Wilcox: ETH Zurich; Chengxu Zhuang: MIT; Adina Williams: Meta Platforms Inc.; Ryan Cotterell: Institute for Machine Learning; Tal Linzen: New York University

Insights from the first BabyLM Challenge: Training sample-efficient language models on a developmentally plausible corpus

Sleep staging serves as the foundation for sleep assessment and disease diagnosis, constituting a crucial aspect of sleep research. The related work on automatic sleep staging has achieved numerous satisfactory outcomes. However, current research predominantly focuses on using sleep information as classification features, employing time-domain or frequency-domain measures as local features, using comprehensive brain network information across channels as global features, while overlooking the spontaneous regularities in brain activity. Simultaneously, brain microstates are considered closely linked to brain activity and can be used to investigate the regular variations in the overall brain potential. To explore the regular changes in the microstates of brain function during sleep stages based on electroencephalogram (EEG), especially the regular changes in sleep structure, we initially conduct microstate clustering on the EEG data during sleep, followed by characterizing the sleep structure of the participants based on these microstates. Subsequently, we integrate the sleep structure with traditional sleep information features and perform automatic sleep staging.Our experiments make the following contributions: (1) Being the first to introduce the use of sleep structure for automatic sleep staging. (2) When there are 7 or more than 7 microstate classes, the model performs well. (3) Proposing a sleep automatic staging model that integrates sleep structure and sleep information.

**Authors:**

Ruixiang Liao: Hangzhou Dianzi University; Li Zhu: Hangzhou Dianzi University; Wanzeng Kong: Hangzhou Dianzi University; Zhengyi Wang: Hangzhou Dianzi University

An Automated Sleep Staging Method with EEG-based Sleep Structure Computation

Theory of mind is an essential ability for complex social interaction and collaboration. Researchers in cognitive science and psychology have previously sought to integrate theory of mind capabilities into artificial intelligence (AI) agents to improve collaborative abilities (Cuzzolin, Morelli, Cirstea, & Sahakian, 2020). These approaches, however, are hampered by the need for labor-intensive hand-labeling of datasets, which prevents them from scaling up to large, real-world datasets. To address this challenge, we introduce the Recurrent Conditional Variational Autoencoder (RCVAE), a novel model designed to predict intent from human behavioral trajectories without the prerequisite of hand-labeled data. We show that in the Overcooked-AI environment, the RCVAE outperforms baseline Long Short-Term Memory (LSTM) models in predicting intent, achieving higher prediction accuracy and greater predictive stability. The implications of these results are significant; the RCVAE's proficiency in learning the relationship between basic actions and resulting contextual behaviors, without needing hand-labeled data, will be crucial for scaling from simple to complex, real-world environments.

**Authors:**

Willa Mannering: Johns Hopkins Applied Physics Laboratory; Noah Ford: Johns Hopkins University Applied Physics Lab; Justin J Harsono: Johns Hopkins University Applied Physics Laboratory; John Winder: Johns Hopkins University Applied Physics Laboratory

Premium content

Downloads

Next from CogSci 2024

Explaining Human Comparisons using Alignment-Importance Heatmaps

Similar lecture

Predictive and Interpretable: Combining Artificial Neural Networks and Classic Cognitive Models to Understand Human Learning and Decision Making