United States

Language is multimodal: non-linguistic cues, such as prosody, gestures and mouth movements, are always present in face-to-face communication and interact to support processing. In this paper, we ask whether and how multimodal cues affect L2 processing by recording EEG for highly proficient bilinguals when watching naturalistic materials. For each word, we quantified surprisal and the informativeness of prosody, gestures, and mouth movements. We found that each cue modulates the N400: prosodic accentuation, meaningful gestures, and informative mouth movements all reduce N400. Further, effects of meaningful gestures but not mouth informativeness are enhanced by prosodic accentuation, whereas effects of mouth are enhanced by meaningful gestures but reduced by beat gestures. Compared with L1, L2 participants benefit less from cues and their interactions, except for meaningful gestures and mouth movements. Thus, in real-world language comprehension, L2 comprehenders use multimodal cues just as L1 speakers albeit to a lesser extent.

CogSci 2021

Electrophysiological signatures of multimodal comprehension in second language

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

poster

A crucial step in children's language development is the mastery of how to use language in context. This involves the ability to recognize and use major categories of speech acts (e.g., learning that a "question" is different from a "request"). The current work provides a quantitative account of speech acts' emergence in the wild. Using a longitudinal corpus of child-caregiver conversations annotated for speech acts (Snow et al.,1996), we introduced two complementary measures of learning based on both children's production and comprehension. We also tested two predictors of learning based on input frequency and the speech acts' quality of linguistic cues. We found that children's developmental trajectory differed largely between production and comprehension. In addition, development in both of these dimensions was not explained with the same predictors (e.g., frequency in child-directed speech was predictive of production, but not of comprehension). The broader impact of this work is to provide a computational framework for the study of communicative development where both measures and predictors of children's pragmatic development can be tested and compared.

Modeling speech act development in early childhood: the role of frequency and linguistic cues

Similarities of either artificial or natural groups are extractable by our visual system in terms of ensemble perception.
On the other hand cognitive linguistics postulates that changing mental image schema for describing a scene from global to local and vice versa is obvious in our daily conversations. 
Here we hypothesized that global or local priming of participants would modulate their threshold in mean discrimination task (MDT).
Thereby, 10 right-handed , 14 years old participants went through an ABBA within-subject counterbalance design to compare 4 pairs of collective and singular nouns priming effect by successive ensemble size perception task(MDT) including a display of random circles sizes and then a mean discrimination task to decide which one of the test circles match the previous display mean circle size. 
Our preliminary results demonstrate Persian collective nouns in comparison to singular nouns could decrease threshold to distinguish mean circle sizes in last display.

Persian collective nouns enhance ensemble size perception

The ability to rapidly recognize words and link them to referents in context is central to children's early language development. This ability, often called word recognition in the developmental literature, is typically studied in the looking-while-listening paradigm, which measures infants' fixation on a target object (vs. a distractor) after hearing a target label. We present a large-scale, open database of infant and toddler eye-tracking data from looking-while-listening tasks. The goal of this effort is to address theoretical and methodological challenges in measuring vocabulary development. We present two analyses of the current database (N=1,320): (1) capturing age-related changes in infants' word recognition while generalizing across item-level variability and (2) assessing how a central methodological decision -- selecting the time window of analysis -- impacts the reliability of measurement. Future efforts will expand the scope of the current database to advance our understanding of participant-level and item-level variation in children's vocabulary development.

Peekbank: Exploring children's word recognition through an open, large-scale repository for developmental eye-tracking data

Top-down predictive processes and bottom-up auditory processes interact in speech comprehension. In background noise, the acoustic signal is degraded. This study investigated the interaction of these processes in a word recognition paradigm using high and low predictability sentences in two types of background noise and using phonetically controlled contrasts. Previous studies have reported false hearing, but have not provided insight into what phonetic features are most prone to false hearing. We here systematically explore this issue and find that plosives lead to increased false hearing compared to vowels. Furthermore, this study on German for the first time replicates the overall false hearing effect in young adults for a language other than English.

Recognition of Minimal Pairs in (un)predictive Sentence Contexts in two Types of Noise

Recent research suggests that an individual's disgust sensitivity affects language comprehension and correlates with political attitudes. Importantly, disgust sensitivity is not a stable measure and can be manipulated dynamically. We investigated the effect of the ongoing pandemic on language processing in a word rating and lexical decision study. Each participant was first exposed to either headlines portraying CoViD-19 as a serious disease or those downplaying it. The results showed an interaction between person-based factors, inherent word characteristics, and the participants’ responses. After reading headlines emphasizing the threat of CoViD-19, easily disgusted participants considered the least disgusting words more disgusting. Further, political views played a role. More liberal participants rated the words lower for disgust in the downplayed condition but higher in the severe condition than their more conservative peers. The results of this study shed new light on how the media's stance on the pandemic may affect the public’s response.

CoViDisgust: Language Comprehension at the Intersection of a Global Pandemic and Individual Disgust Sensitivity

Is “cow” more closely related to “grass” or “chicken”? Speakers of different languages judge similarity in this context differently, but why? One possibility is that cultures co-varying with these languages induce variation in conceptualizations of similarity. Specifically, East Asian cultures may promote reasoning about thematic similarity, by which cow and grass are more related whereas Western cultures may bias similarity judgements toward taxonomic relations, like cow-chicken. This difference in notions of similarity is the consensus interpretation for cross-cultural variation in this paradigm. We consider, and provide evidence for, an alternative possibility, by which notions of similarity are equivalent across contexts, but the statistics of the environment vary. On this account, similarity judgements are guided by co-occurrence in experience, and observing or hearing about cows and grass or cows and chickens more often could induce preferences for the relevant grouping, and account for apparent differences in notions of similarity across contexts.

Re-examining cross-cultural similarity judgements using lexical co-occurrence

Knowledge built culturally across generations rests on language, but the power and mechanisms of language as a means of cultural learning are not well understood. We take a first step towards reverse-engineering cultural learning through language. We developed a suite of complex, high-stakes video games, which we deployed in an iterated learning paradigm. Game participants were limited to only two attempts (lives) per game, after which they wrote a message to a future participant who read the message before playing. Knowledge accumulated gradually across generations, with later generations advancing further in the games while performing more efficient actions, in a manner strikingly similar trajectory to the learning trajectories of isolated, immortal individuals. These results suggest that language is sufficient to accumulate diverse repertoires of knowledge acquired in these tasks. The video game paradigm is thus a rich test-bed for theories of cultural transmission and learning from language.

Growing knowledge culturally across generations to solve novel, complex task

We asked whether categories expressed through lists of salient exemplars (e.g., car, truck, boat, etc.) convey the same meaning as categories expressed through conventional superordinate nouns (e.g., vehicles). We asked English speakers to list category members, with one group given superordinate labels like vehicles and the other group given only a list of salient exemplars. We found that the responses of the group given labels were more related, more typical, and less diverse than the responses of the group given exemplars. This result suggests that when people do not see a superordinate label, the categories that they infer are less well aligned across participants. In addition, categories inferred based on exemplars may be broader in general than categories given by superordinate labels.

Lions, tigers and bears: Conveying a superordinate category without a superordinate label

Asking and answering questions is a staple of human communication. To answer a question effectively, a hearer must interpret the speaker's intention given the specific question asked. ‘Wh’-questions like ‘Where can I get coffee?’ are underspecified for (non-)exhaustivity, i.e., how many answers must be provided to resolve the speaker's goal. Intuitions from the semantics literature report that questions are generally exhaustive, and non-exhaustive only in the context of specific linguistic factors (e.g., the modal ‘can’, certain ‘wh’-words). To test these assumptions, we collected question paraphrase ratings for naturally occurring root questions in variable linguistic contexts. In contrast to previous claims, we find that questions are not biased for exhaustivity. However, other prior observations are supported by the data. We argue that a full account of the observed distribution of meanings must integrate discourse factors like the hearer's estimate of the speaker's goal, alongside (or subsuming the effect of) linguistic cues.

Who thinks wh-questions are exhaustive?

How does abstract linguistic negation develop in early child language? Previous research has suggested that abstract negation develops in stages and from more concrete communicative functions such as rejection, prohibition, or non-existence.
The evidence for the emergence of these functions in stages is mixed, however, leaving the possibility that negation is an abstract concept from the beginning that can serve multiple specific functions depending on early communicative environment. Leveraging automatic annotations of large-scale child speech corpora in English, we examine the production trajectories of seven negative constructions that tend to convey communicative functions previously discussed in the literature. The results demonstrate the emergence and gradual increase of these constructions in child speech within the age range of 18-36 months. Production mostly remains stable, regular, and close to parents’ levels after this age range. These findings are consistent with two hypotheses: first, that negation starts as an abstract concept that can serve multiple functions from the beginning; and second, that negation develops in stages from specific communicative functions but this development is early and quick, leaving our corpus methods incapable of detecting them from the available corpus data.

Electrophysiological signatures of multimodal comprehension in second language

Downloads

Next from CogSci 2021

Modeling speech act development in early childhood: the role of frequency and linguistic cues

Similar lecture

Internet-based Assessment of an Inhibitory Control Advantage in Bilingual Chinese High School Students