Netherlands

We compare storytelling in GPT-3.5, a recent large language model, with human storytelling. Although GPT models are capable of solving novel and challenging tasks and matching human-level performance, it is not well understood if GPT processes information similarly as humans. We hypothesized that GPT differs from humans in the kind of memories it possesses, and thus could perform differently on tasks influenced by memory, such as storytelling. Storytelling is an important task for comparison as GPT becomes an increasingly popular writing and narrative tool. We used an existing dataset of human stories, either recalled or imagined (Sap et al., 2022), and generated GPT stories with prompts designed to align with human instructions. We found that GPT&#39;s stories followed a common narrative flow of the story prompt (analogous to semantic memory in humans) more than details occurring in the specific context of the event (analogous to episodic memory in humans). Furthermore, despite lacking episodic details, GPT-generated stories exhibited language with greater word affect (valence, arousal, and dominance). When provided with examples of human stories (through few-shot prompting), GPT was not able to align its stories&#39; narrative flow with human recalled stories, nor did it match its affective aspects with either human imagined or recalled stories. We discuss these results in relation to GPT&#39;s training data as well as the way it was trained.

**Authors:**

Charlotte Cornell: Rutgers University--New Brunswick; Shuning Jin: Rutgers University; Qiong Zhang: Rutgers University - New Brunswick

CogSci 2024

The Role of Episodic Memory in Storytelling: Comparing Large Language Models with Humans

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

We compare storytelling in GPT-3.5, a recent large language model, with human storytelling. Although GPT models are capable of solving novel and challenging tasks and matching human-level performance, it is not well understood if GPT processes information similarly as humans. We hypothesized that GPT differs from humans in the kind of memories it possesses, and thus could perform differently on tasks influenced by memory, such as storytelling. Storytelling is an important task for comparison as GPT becomes an increasingly popular writing and narrative tool. We used an existing dataset of human stories, either recalled or imagined (Sap et al., 2022), and generated GPT stories with prompts designed to align with human instructions. We found that GPT's stories followed a common narrative flow of the story prompt (analogous to semantic memory in humans) more than details occurring in the specific context of the event (analogous to episodic memory in humans). Furthermore, despite lacking episodic details, GPT-generated stories exhibited language with greater word affect (valence, arousal, and dominance). When provided with examples of human stories (through few-shot prompting), GPT was not able to align its stories' narrative flow with human recalled stories, nor did it match its affective aspects with either human imagined or recalled stories. We discuss these results in relation to GPT's training data as well as the way it was trained.

**Authors:**

Charlotte Cornell: Rutgers University--New Brunswick; Shuning Jin: Rutgers University; Qiong Zhang: Rutgers University - New Brunswick

poster

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

People have strong intuitions about which colors do and do not match particular pieces of music --- a phenomenon often conceptualized through semantic mediation. We further explored the specific strategies people employ to navigate these color-music associations which offers crucial insights to identifying the cognitive mechanisms that enable such cross-domain associations. We show that while some people rely more on intuition, other people actively seek justification of association by consulting common linguistic descriptors, emotional contents, and environmental cues. We found that more spontaneous strategies lessen the role of semantic mediation, while more evaluative strategies, especially those involving the use of language, amplify it. Notably, the use of evaluative strategies introduced an asymmetrical effect for matching and mismatching colors. Additionally, individuals employing similar strategies associated a given music excerpt with more similar colors, suggesting that strategy alignment enhances the consistency of color-music associations. Interestingly, this pattern of convergence was not observed among individuals who predominantly relied on guessing.

**Authors:**

Qiawen Liu: University of Wisconsin-Madison; Huang Ham: Department of Psychology; Kesong Cao: University of Wisconsin-Madison; Gary Lupyan: University of Wisconsin - Madison

Why is Bach blue instead of red? Different strategies moderate people’s color-music associations

Index systems are central to our everyday and intellectual lives. Their ubiquity and diversity make them an im-portant class of cognitive artifacts, the study of which has implications for our understanding of representational systems in general. This paper builds schema-theoretic network models of the nature of the memory structures, that underpin the interpretation of indexing systems. We identify four common classes of index systems. Using Representation Interpretation Structure Theory, we explain how the four basic classes can be responsible for the sub-stantial diversity among index systems.

**Authors:**

Peter Cheng: University of Sussex; Grecia Garcia Garcia: University of Sussex; Daniel Raggi: University of Cambridge; Mateja Jamnik: University of Cambridge

Index Systems: Enumerating Their Forms and Explaining Their Diversity With Representational Interpretive Structure Theory

The ability to flexibly interpret signals in context is at the core of human communication, as even the most conventionalized linguistic signals are necessarily ambiguous and subject to inter-individual variability. We introduce a novel experimental semiotic game (the Pizzini game) requiring pairs of participants to exchange linguistic signals that can only be successfully interpreted when using contextual information freshly generated by each pair. By allowing this common ground between once-strangers to be developed interactively in the lab, we are able to characterize the pair-specific contextual information available to participants when inferring intended meanings. We present preliminary data testing the predictions that (1) interactants align on an abstracted conceptual representation of a set of stimuli during the context-building portion of the task and (2) that the characteristics of this pair-specific conceptual representation predict the dynamics of how participants later resolve context-specific references to the same stimuli.

**Authors:**

Miriam Greidanus Romaneli: Radboud University Nijmegen; Jesper Sinneker: Radboud University Nijmegen; Jana Bašnáková: Radboud University Nijmegen; Bruno Galantucci: Yeshiva University; Ivan Toni: Radboud University

Investigating contextual effects in referential communication: an experimental semiotic game

Various methods for social learning have been proposed within the reinforcement learning framework. These methods involve the social transmission of information within specific representational formats like policies, value, or world models. However, transmission of higher-level, model-based representations typically require costly inference (i.e., mentalizing) to ``unpack'' observable actions into putative mental states (e.g., with inverse reinforcement learning). Here, we investigate cheaper, non-mentalizing alternatives to social transmission of model-based representations that bias the statistics of experience to ``hijack'' asocial mechanisms for learning of environments. We simulate a spatial foraging task where a naïve learner learns alone or through observing a pre-trained expert. We test model-free vs. model-based learning together with simple non-mentalizing social learning strategies. Through analysis of generalization when the expert can no longer be observed and through correspondence between expert and learner representations, we show how simple social learning mechanisms can give rise to complex forms of cultural transmission.

**Authors:**

Miriam Bautista: University of Tübingen; Ryutaro Uchiyama: Tübingen AI Center; Claudio Tennie: University of Tübingen; Charley M Wu: University of Tübingen

Emergent social transmission of model-based representations without inference

Alongside the advancement of large language models (LLMs), attention towards their limitations and potential risks has also increased. One common issue is hallucination, which occurs when LLMs generate inaccurate or irrelevant answers, especially for complex sentences. To address this issue, we propose a novel question preprocessing method inspired by how young children comprehend complex sentences. Our method consists of two modules: (1) hierarchical clause annotation (HCA)-based sentence decomposition, which breaks down complex sentences into one-verb-centered clauses, and (2) abstract meaning representation (AMR)-based clause rewriting, which reformulates the clauses based on AMR into the child-comprehensible subject-verb-object (SVO) structure. We evaluate our method on the question-answering dataset, TruthfulQA, and show that it can improve the truthfulness and informativeness of widely-used LLMs, LLaMA-7B, and LLaMA-2-7B-chat, preventing from generating hallucinated answers. Moreover, our method is highly efficient, as it does not require any pre-training, fine-tuning, or invoking larger-scale models.

**Authors:**

Yunlong Fan: Southeast University; Bin Li: Southeast University; Zhiqiang Gao: Southeast University

Mitigating Hallucinations in Large Language Models by Preprocessing Questions into Child-Comprehensible

This study investigates whether fixation behaviour during scene viewing can offer insights into sequentialisation in verbal scene description production. We explored the correlation between visual and linguistic attention on naturalistic scenes using scene descriptions and eye movement measures. Results demonstrate an overlap in object prioritization during scene viewing and describing. Our additional analysis of scene descriptions reveals a tendency towards selecting and prioritizing category-specific objects.

**Authors:**

Pelin Celikkol: University of Potsdam; David Schlangen: University of Potsdam; Jochen Laubrock: University of Potsdam

Effects of Eye Movement Patterns and Scene-Object Relations on Description Production

We investigate the relations between conceptual and syntactic structure by focusing on the phenomenon of circumstantial metonymy. We hypothesize that the construal of a metonymic interpretation is facilitated when the metonymized argument occurs earlier in time relative to the metonymizing predicate, since this gives the processor more time to access the conceptual structure associated with the argument. This hypothesis predicts a greater cost of metonymy composition when the argument is in object position (after the predicate) relative to subject position (before the predicate). Study 1 (acceptability task) shows a main effect of metonymy for both syntactic positions. Study 2 (self-paced reading task) demonstrates a cost for metonymy in object but not subject position. This indicates that the cost of metonymy composition is rooted in the requirement that the conceptual structure for the metonymic argument be fully retrieved, a process constrained by the order of lexical retrieval provided by syntactic structure.

**Authors:**

Maria Teresa Borneo: Yale University; Maria M Piñango: Yale University

Linear Word Order Modulates the Cost of Metonymy Comprehension: Dynamics of Conceptual Composition

The study aims to determine if the positive ERP effect associated with self-relevance extends from first to second person pronouns and whether it is independent of the pronoun's referent. Two EEG experiments were conducted with 72 participants listening to two distinct audiobooks, "Tschick" and "Auferstehung der Toten" (AdT). The chosen novels differ in narrative structure, allowing for a comparison of the ERP response of 2sg pronouns that potentially refer to the listener with personal pronouns that do not. The narrative design of "Tschick" directed all 2nd person pronouns to characters in the story, while in "AdT" the listener was the most likely referent. The results reveal a significant positive ERP effect for second person pronouns in "AdT" compared to "Tschick," supporting the hypothesis that the self-relevance effect generalizes to second person pronouns. The findings suggest that this positivity in ERP reflects attentional processes enhancing the cortex's sensitivity to self-other distinctions.

**Authors:**

Magdalena Repp: University of Cologne; Petra B. Schumacher: University of Cologne; Ingmar Brilmayer: University of Cologne

ERP insights into self-relevance with second-person pronouns during auditory story processing

In this work, we discuss the complexity surrounding the origins of human language, focusing on the debate between gesture-first and vocalization-first theories. While some evidence supports the idea that gestures played a primary role in early communication, others argue that vocalizations are equally expressive. We think that methodological differences and biases in the choice of concepts may contribute to the challenge of comparing these modalities directly. 
For example, to what extent does selecting a certain concept from a semantic category matter to reproduce an effect? This and similar questions are explored in a data-driven way by simulating experiments. In these simulations the number of participants, number of concepts, and semantic categories are varied, drawing on Bayesian posterior predictive distributions of collected data on expressibility ratings of 207 concepts. Our results show that gestures are more expressive on average, but vocalization are expressive too. The differences between the two are limited in terms of effect size (medium sizes by common standards) so that one may question whether this difference is meaningful for bigger claims about early language evolution. This study further provides valuable information for further research on how to select stimuli and how to setup one's design in a balanced way.

**Authors:**

Šárka Kadavá: Leibniz Center General Linguistics; Aleksandra Cwiek: Leibniz-Centre General Linguistics; Susanne Fuchs: Leibniz-Zentrum Allgemeine Sprachwissenschaft; Wim Pouw: Radboud University

What do we mean when we say gestures are more expressive than vocalizations? An experimental and simulation study

Homophony (i.e, multiple meanings expressed by the same
form) is ubiquitous across the world’s languages. Despite
its pervasiveness, not all instances of homophony are equally
likely, which suggests that homophony is unlikely to be ac-
cidental. There is a growing body of literature which aims
to thoroughly examine cross-linguistic regularities in patterns
of homophony and explain these from constraints in language
learning and use, both at the lexical and morphosyntactic
levels. Here, we examine a specific case of homophony in
pronominal paradigms, that is, the lack of a number distinc-
tion (singular vs plural) for a given person value (first, second
and third), a phenomenon coined as horizontal homophony.
Cysouw (2003) suggested that a lack of number distinction is
more likely to be found in third person (i.e., 3SG=3PL) than in
second (i.e., 2SG=2PL), and it is least frequently found in first
person (i.e., 1SG=1PL). We refer to this generalisation as the
Horizontal Homophony Hierarchy: 3 > 2 > 1 (where > repre-
sents frequency inequality). This generalisation was neverthe-
less only made via qualitative description and by raw counts,
and merely described without motivated explanation. In this
study we take a step back and present additional evidence sup-
porting the Horizontal Homophony Hierarchy. First, we as-
certain the robustness of this typological tendency through a
statistical analysis using the largest cross-linguistic database
of pronominal paradigms to date (926 languages from 229
different families). Next, we explore whether the Horizon-
tal Homophony Hierarchy has a corresponding learning cor-
relate, which would indicate that this asymmetry is at least
partly rooted in a cognitive bias. Specifically, we examine
asymmetries in how easily adult humans learn different types
of horizontal homophony in an artificial language learning ex-
periment. The results from our typological analysis corrobo-
rate a hierarchy of horizontal homophony 3 > 2 > 1 in the
world’s languages. However, our experimental results provide
evidence against a learning bias underlying the hierarchy, thus
suggesting that motivated explanations of the typology (if any)
are more likely to be found in alternative pressures such as
communicative need and efficiency.

**Authors:**

Carmen Saldana: Universitat Pompeu Fabra; Mora Maldonado: CNRS/Nantes Université

Premium content

Downloads

Next from CogSci 2024

Why is Bach blue instead of red? Different strategies moderate people’s color-music associations

Similar lecture

The GPT-WritingPrompts Dataset: A Comparative Analysis of Character Portrayal in Short Stories