United States

The N400 brain potential has been used as a neural correlate of meaning-related processing in the brain, but its underlying computational mechanism is still not well understood. Although efforts to model the N400 as an update of a probabilistic representation of meaning have been promising, the limited scope of earlier models has restricted experiments to highly simplified sentences. Here, we expand modelling of the N400 to naturalistic sentences using a large-scale, state-of-the-art deep learning language model. We investigate the correspondence between updates in the internal state of the model and the N400 in one quantitative experiment and four qualitative experiments. Our findings suggest that activation updates in the model correspond to several N400 effects, but cannot account for all of them.

CogSci 2021

Meaning in brains and machines: Internal activation update in large-scale language model partially reflects the N400 brain potential

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

poster

This preregistered study examined whether extraneous illustration details promote attentional competition and hinder reading comprehension in beginning readers. Reading comprehension was highest in the Streamlined Condition (text + relevant illustrations) compared to a Standard Condition (text + relevant illustrations + extraneous illustrations) and Text Only Condition (no illustrations). Gaze shifts away from the text were highest in the Standard Condition, indicating increased distractibility while reading text with extraneous illustration details. Gaze shifts away from the text were associated with performance on an independent measure of attention, validating eye gaze patterns as an assessment of attentional allocation while reading. Lower comprehension in the Standard Condition was associated with higher gaze shifts away from text and lower scores on the independent measure of selective attention. This study suggests that illustrations can support reading comprehension, but only when they are optimally designed. Importantly, the removal of extraneous details did not decrease book enjoyment.

The Optimal Amount of Visuals Promotes Children’s Comprehension and Attention: An Eye Tracking Study

Predictive Processing (PP) has been suggested to account for early cognitive development (Köster et al., 2020). In this paper, we extend its application to early social coordination in parent-infant interactions. Interpersonal neural synchrony in parent-child interactions is hypothesized to be a function of the coupling of internal generative models for mutual prediction. By aligning these internal models and reducing prediction errors, social uncertainty is reduced, and interpersonal neural synchrony is enhanced. Support for this hypothesis is provided by assessing neural synchrony during mother-infant interaction.

Infants’ Social Communication from a Predictive Processing Perspective

In order to communicate, humans flatten complex ideas and their attributes into a sequence of words. Humans can use this ability to express and understand complex hierarchical and relational concepts, such as kinship relations and logical deduction chains. We simulate communication of relational and hierarchical concepts using artificial agents. We propose a new set of graph communication games, which show that agents parametrized by graph neural networks develop a more compositional language compared to bag-of-words and sequence models. Graph-based agents are also more successful at systematic generalization to new combinations of familiar features. We release the implementation to probe research on emergent communication over complex data.

Structural Inductive Biases in Emergent Communication

Humans have a remarkable ability to go beyond the observable. From seeing the current state of our shared kitchen, we can infer what happened and who did it. Prior work has shown how the physical state of the world licenses inferences about the causal history of events, and the agents that participated in these events. Here, we investigate a previously unstudied source of evidence about what happened: social evaluations. In our experiment, we present situations in which a group failed to optimally coordinate their actions. Participants learn how much each agent was blamed for the outcome, and their task is to make inferences about the situation, the agents' actions, as well as the agents' capabilities. We develop a computational model that accurately captures participants' inferences. The model assumes that people blame others by considering what they should have done, and what causal role their action played. By inverting this generative model of blame, people can figure out what happened.

Who went fishing? Inferences from social evaluations

The present paper addresses the study of cross-linguistic phonosemantic correspondences within a deep learning framework. An LSTM-based Recurrent Neural Network is trained to associate the phonetic representation of a word, encoded as a sequence of feature vectors, to its corresponding semantic representation in a multilingual and cross-family vector space. The processing network is then tested, without further training, in a language that does not appear in the training set and belongs to a different language family. The performance of the model is evaluated through a comparison with a monolingual and mono-family upper bound and a randomized baseline. After the assessment of the network's performance, the distribution of phonosemantic properties in the lexicon is inspected in relation to different (psycho)linguistic variables, showing a link between lexical non-arbitrariness and semantic, syntactic, pragmatic, and developmental factors.

A Layered Bridge from Sound to Meaning: Investigating Cross-linguistic Phonosemantic Correspondences

The question of how algorithms in general and cognitive skills in particular are implemented by our nervous system is at the core of cognitive science. The notions of what it means for a physical system (such as our nervous system) to implement an algorithm, however, are surprisingly vague. We argue that a rigorous theory is needed to formulate and evaluate precise hypotheses about the brain's cognitive functions and propose a definition of the term algorithm as a chain of functions. Subsequently, we define the term implementation via a sequence of projections from a dynamical system, represented by a Markov process, to the algorithm. We furthermore show the practical applicability of this approach in a simulated example. We believe that the theory proposed here contributes to bridging the gap between the algorithmic and the implementational level by rendering the task at hand theoretically precise.

A theory of algorithms and implementations and their relevance to cognitive science

Despite their apparent importance for the acquisition of full-fledged human intelligence, mechanisms of intrinsically motivated autonomous learning are poorly understood. How do humans identify useful sources of knowledge and decide which learning situations to approach in the absence of external rewards? While the recognition of this important problem has grown in psychological sciences over the recent years, an intriguing proposition for the possible mechanism comes from artificial intelligence, where efficient autonomous learning is achieved by programming agents to follow the heuristic of maximizing learning progress (LP) during exploration. In this study, we set out to examine the empirical evidence for this idea. Using computational modeling, we demonstrate that humans show signs of following LP while they freely explore and practice a set of multiple learning activities of varying difficulty, including an activity that is impossible to learn. Different approaches to operationalizing the notion of LP and their plausibility in light of empirical data are also discussed. We also show that models combining several types of intrinsic rewards fit better human exploration data than single component models considered so far in theoretical accounts.

Intrinsic Rewards in Human Curiosity-Driven Exploration: An Empirical Study

We propose a conceptual framework of multiversional narrative processing. Multiversional narrative processing is the consideration of multiple possible event sequences for an incomplete narrative during reception. It occurs naturally and is experienced in a wide range of cases, such as suspense, surprise, counterfactuals, and detective stories. Receiving a narrative, we propose, is characterized by the spontaneous creation of competing interpretive versions of the narrative that are then used to create predictions and projections for the narrative’s future. These predictions serve as a mechanism for integrating incoming information and updating the narrative model through prediction error, without completely eliminating past versions. We define this process as having three aspects: (1) constrained expectations, (2) preference projection, and (3) causal extrapolation. Constrained expectations and preference projections respectively create the bounds and subjective desires for a narrative’s progress, while causal extrapolation builds, reworks, and maintains the potential models for understanding the narrative.

Multiversionality: Considering multiple possibilities in the processing of narrative

Recent advances in neural architectures have revived the problem of morphological rule learning. We evaluate the Transformer as a model of morphological rule learning and compare it with Recurrent Neural Networks (RNN) on English, German, and Russian. We bring to the fore a hitherto overlooked problem, the morphological gaps, where the expected inflection of a word is missing. For example, 63 Russian verbs lack a first-person-singular present form such that one cannot comfortably say "*oščušču" ("I feel"). Even English has gaps, such as the past participle of "stride": the function of morphological inflection can be partial. Both neural architectures produce inflections that ought to be missing. Analyses reveal that Transformers recapitulate the statistical distribution of inflections in the training data, similar to RNNs. Models' success on English and German is driven by the fact that rules in these languages can be identified with the majority forms, which is not universal.

Downloads

Next from CogSci 2021

The Optimal Amount of Visuals Promotes Children’s Comprehension and Attention: An Eye Tracking Study

Similar lecture

The N400 event-related potential component reflects a learning signal