United States

Deep reinforcement learning (DRL) is capable of training agents that exceed human-levels of performance in multi-agent tasks. However, the behaviors exhibited by these agents are not guaranteed to be human-like or human-compatible. This poses a problem if the goal is to design agents capable of collaborating with humans or augmenting human actions in cooperative or team-based tasks. Indeed, recommender systems designed to augment human decision making need to not only recommend actions that align with the task goal, but which also maintain coordinative behaviors between agents. The current study simultaneously explored skill learning performance of human learners when working alongside different artificial agents (AAs) during a collaborative problem-solving task, as well as evaluated the effectiveness of the same AAs as action decision recommender systems to aid learning. The action decisions of the AAs were either modelled by a heuristic model based on human performance or by a deep neural network trained by reinforcement learning in self-play. In addition to evaluating skill learning performance, the current study also tested the congruence between the decisions of the AAs with actual decisions made by humans. Results demonstrate that the performance of humans was significantly worse when working alongside the DRL AA compared to the heuristic AA. Additionally, the action decisions participants made also showed less allignment with the recommendations made by the DRL AA.

CogSci 2023

Action decision congruence between human and deep reinforcement learning agents during a coordinated action task

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

This study attempts to gain novel insights into the mechanisms underlying the bilingual advantage in cognitive control and investigates the effect of code-switching experience in daily life, rather than considering only language proficiency, on these underlying mechanisms. We compared the performances of early and late Japanese-English bilinguals on the AX-Continuous Performance Task. We found early and late bilinguals show different cognitive control mechanisms to cope with interference: with the late bilinguals, the more frequently they switched languages, the better they performed in the proactive control tasks. Conversely, with the early bilinguals who are generally more proficient, the frequency of code-switching did not modulate their dual mechanism of cognitive control any better, showing little correlation with the ability of proactive control. These findings verify the role of switching frequency in the development of cognitive control and suggest a better understanding of the nature of mechanisms regulating early and late bilinguals’ brains.

The effect of code-switching experience on the dual mechanism of cognitive control: A comparison between early and late Japanese-English bilinguals

When assessing children in laboratory experiments, the measured responses also contain task-irrelevant participant-level variability (“noise”) and measurement noise. Since experimental data are used to make inferences of development of cognitive capabilities with age, it is important to know if reliability of the used measurements depends on child age. Any systematic age-dependent changes in reliability could result in misleading developmental trajectories, as lower reliability will necessarily result in smaller effect sizes. This paper examines age-dependency of task-independent measurement variability in early childhood (3–40 months) by analyzing two large-scale datasets of participant-level experimental responses: the ManyBabies infant-directed speech preference (MB-IDS) dataset and a saccadic reaction time (SRT) dataset collected from rural South Africa. Analysis of participant- and study-level data reveals that MB-IDS shows comparable reliability across the included age range. In contrast, SRTs reflect systematically increasing measurement consistency with increasing age. Potential reasons and implications of this divergence are briefly discussed.

Is Reliability of Cognitive Measures in Children Dependent on Participant Age? A Case Study with Two Large-Scale Datasets

Previous research suggests that a verb’s meaning is learned partly through the aggregated profile of syntactic frames associated with it. For example, “turn” occurs with transitive and intransitive frames in causative alternation (“He turned the car”/“The car turned”), indicating it is a causal verb. Some evidence demonstrates that young children combine multiple frames to map verbs to appropriate events. However, previous work always presented these frames together, in a single dialogue. What remains unknown is how verb learning occurs when the frames are separated, uttered in different referential contexts, as is likely in children’s everyday life. In a series of cross-situational word-learning experiments, we show that both adults and three-year-olds generalize verb meanings across different syntactic frames in a cross-situational learning task. These results shed light on the cross-situational mechanisms of syntactic bootstrapping.

Evidence for Cross-situational Syntactic Bootstrapping: Three-year olds Generalize Verb Meaning across Different Syntactic Frames

Children are able to associate a word with its referent even after a single exposure (i.e., fast-mapping). This ability plays a central role in vocabulary development. Often, words are learned in context and not in isolation. However, most word-learning studies have not considered visual context cues. In the present eye-tracking study we investigated the effects of contextual information (constant versus variable context) on word-referent association (exposure phase) and learning (test phase) during fast-mapping in school-age children. Also, we examined how vocabulary and attention skills modulate this process. Our results show that constant visual context facilitated novel word-reference association (exposure phase) in both age groups while word learning (test phase) was modulated by the visual context only in the younger group. Similarly, visual attention skills ease word learning in the most demanding condition (variable condition) in this group only. Finally, vocabulary skills predicted word learning in the older group.

Visual Context and Word Learning in School-Age Children: An Eye-tracking Study

The existing literature on incidental word learning has not studied the time course of this process in adolescents, even though this type of learning is the first source of acquisition of new words, particularly in secondary education. The present study fills this gap, and also examines the role of preexisting vocabulary in this process, and the relationship between the percentage of words learned and reading times. A sample of fourteen adolescents completed self-paced reading study with three-phase (exposure, implicit assessment, explicit assessment) plus a vocabulary test. The results show that those participants who learned more words spent more reading time on the new words in the exposure phase, and furthermore, evidenced a facilitation effect after only four exposures. Tentatively, this ability could be related in part to the adolescents’ pre-existing vocabulary.

Understanding the Time Course of Context-Based Incidental Word Learning

Past studies have revealed that language experience impacts children’s vocabulary development. For example, bilingual children tend to have smaller vocabularies than monolingual children in each of their languages. However, it is unclear whether routine exposure to multiple accents also affects children’s vocabulary growth. Here, using standardized vocabulary assessments, we compared the reported vocabulary sizes of 11- to 34-month-old monolingual and bilingual children (N = 2881) who had various degrees of accent exposure. Our results show that routine exposure to multiple accents, regardless of accent type, does not negatively impact vocabulary development. Our findings suggest that children are well-equipped to handle language variation in their input.

Effects of Accent Exposure on Monolingual and Bilingual Children’s Early Vocabulary Development

The input for early language learning is often viewed as a landscape of ambiguity with the occasional high-quality naming event providing resources to resolve uncertainty. Word learning from ambiguous naming events is often studied using screen-based cross-situational learning tasks. Little is known, however, on how ambiguity impacts real-time word learning in free-flowing interactions. To explore this question, we asked parent-infant dyads to play in a home-like environment with unfamiliar objects while wearing head-mounted eye trackers. After the play session, we tested whether infants learned any of the object-label mappings and categorized individual words as learned or not learned. Dyadic behaviors and the visual information available to infants during the naming moments of learned and not learned words were analyzed. The results show that infants’ embodied attention during ambiguous naming moments was the key to predicting learning outcomes. Specifically, infants held and looked at the target object longer in ambiguous instances that led to learning. Our results emphasize the importance of studying word learning in naturalistic environments to better understand the cues infants use to resolve ambiguity in everyday learning contexts.

Embodied attention resolves visual ambiguity to support infants’ real-time word learning

Adults are more accurate at identifying own- than other-accented talkers (e.g., Stevenage, 2012). However, no study to date has examined when the Other Accent Effect (OAE) first emerges. Here, we use a voice line-up task to test 6- to 12-year-olds (N=59) on their ability to recognize own- (Canadian English) and other-accented (Japanese- or Mandarin-accented English) talkers. Own-accented talkers (M=0.77) were recognized better than other-accented talkers (M=0.61), p<.005. No effect of age was observed. This indicates the OAE is well-entrenched by 6 years of age. Interestingly, all children tended to evaluate other-accented talkers negatively, but these biases did not predict the strength of the OAE, suggesting that a factor other than social biases drives the effect (Yu et al., 2021). To further understand the underlying mechanisms of the OAE, we are currently testing children in the same line-up task on their ability to recognize Southern American and Australian English talkers.

The Emergence of the Other Accent Effect in Talker Recognition

Learning verbs is critical to learning one’s native language. Two new studies with comparison trials and additional test trials reveal better verb learning by both Japanese- and English-speaking children than prior studies (Imai et al., 2005; 2008). Specifically, 2½-, 3½- and 4½-year-old Japanese-speaking children (total n=170; n= 43 2.5s; n= 48 3.5s, n= 48 4.5s) in Japan and English-speaking children in the US (total n=107 to date; n= 28 2.5s; n= 36 3.5s; n= 43 4.5s) were shown 3 events linked to each of 4 novel verbs, either in a Progressive Alignment (similar before varied events) or All Varied condition; children pointed to new events at test. Multivariate ANOVAs in both languages show children benefit from seeing high than low similarity events during verb learning, particularly at younger ages. Links to key theories in verb learning, including structure mapping and statistical learning, will be discussed.

High Similarity Comparison Experience Helps Japanese- and English-speaking Children Learn New Verbs

Word associations are among the most common paradigms for studying the human mental lexicon. While their structure and underlying relation types have been widely studied, surprisingly little attention has been given to the question of why participants produce the observed associations. Answering this question would not only advance understanding of human cognition, but could also aid machines in learning and representing basic commonsense knowledge. Here, we introduce WAX: a large, crowd-sourced data set of 19K English word associations paired with human-generated free-text explanations and relation type labels. We present an efficient framework for eliciting associations together with explanations, and a comprehensive analysis of the emerging types of explanations and relations. We test language models in their ability to predict associations and generate explanations, demonstrating that models struggle to capture the diversity of human-produced associations, and suggesting WAX as a fertile resource for future research.

Premium content

Action decision congruence between human and deep reinforcement learning agents during a coordinated action task

Next from CogSci 2023

The effect of code-switching experience on the dual mechanism of cognitive control: A comparison between early and late Japanese-English bilinguals