Canada

In discourse, entities that are discontinuous with the current storyline are seen as cues for an event boundary, as they are too irregular to be mapped to the existing scene. However, some instances of discontinuity can maintain coherency, as exemplified in Calvin &amp; Hobbes comics, where visual discontinuities can be resolved by the understanding that they depict Calvin’s imagination rather than actual events. This requires the reconciliation of the original storyworld domain with a private, mental domain (the alternative). In our first experiment, we examined whether switching between domains and/or the nature of the presented domain(s) incurred processing costs. Our second experiment examined whether physical cues such as the contours between the discontinuous entities facilitated processing. The results indicate that switching domains is indeed more costly, despite still being understood as congruous sequences. Moreover, strong similarity in contours aided readers with greater comics proficiency. Overall, our results show that the processing of visual narratives extends beyond mere event understanding and is not universally transparent.

CogSci 2022

The Costs and Composition of Discontinuity in Visual Narratives

The purpose of the AAAI conference is to promote research in artificial intelligence (AI) and scientific exchange among AI researchers, practitioners, scientists, and engineers in affiliated disciplines. 

AAAI 2021

In discourse, entities that are discontinuous with the current storyline are seen as cues for an event boundary, as they are too irregular to be mapped to the existing scene. However, some instances of discontinuity can maintain coherency, as exemplified in Calvin & Hobbes comics, where visual discontinuities can be resolved by the understanding that they depict Calvin’s imagination rather than actual events. This requires the reconciliation of the original storyworld domain with a private, mental domain (the alternative). In our first experiment, we examined whether switching between domains and/or the nature of the presented domain(s) incurred processing costs. Our second experiment examined whether physical cues such as the contours between the discontinuous entities facilitated processing. The results indicate that switching domains is indeed more costly, despite still being understood as congruous sequences. Moreover, strong similarity in contours aided readers with greater comics proficiency. Overall, our results show that the processing of visual narratives extends beyond mere event understanding and is not universally transparent.

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





In a preregistered pupillometry study we tested whether a perspective cue can both increase and decrease 7.5-month-olds' surprise.
In the congruent condition infants saw an agent watching a ball roll behind an occluder. After the agent left, infants saw the ball rolling outside the stage (an informational asymmetry). In the baseline the agent watched the final event as well. At outcome, the occluder was lowered to reveal empty space: congruent with reality, as the ball is out. In the incongruent condition the outcome is identical, but the ball should have been there. Informational asymmetry was manipulated again.
We found overall larger pupil dilation in the incongruent condition, indicating infants’ remembering the ball’s existence. As hypothesised, we could increase their surprise in the congruent condition by having the agent last see the ball inside, but found mixed results when trying to use the perspective cue to also decrease infants’ surprise.

Altercententric interference vs. bias in 7.5 month-old infants: a pupillometry study

Infants’ ability to anticipate the actions they observe is thought to be a critical skill in early social-cognitive development. A considerable body of research has demonstrated that infants can anticipate observed actions from a variety of cues in different kinds of visual contexts. However, most of this work has relied on screen-based eye-tracking paradigm using highly controlled computer stimuli. Less is known is known about whether and how infants anticipate live actions in natural settings. One prior study demonstrated that infants did anticipate their parents’ actions during live, free-flowing parent-infant play, though they did so infrequently. Using head-mounted eye-tracking, the current study aimed to build upon this work by exploring whether infants make gaze predictions while observing their parents perform an everyday activity in a home-like environment—assembling several peanut-butter and jelly sandwiches. Preliminary findings reveal that infants anticipate their parents’ actions at relatively high rates, indicating a close coupling between parent actions and infant gaze. Rates of prediction did not change over the course of the observation, suggesting that infants already possess prior knowledge of the action sequence structure that they utilize to successfully predict their parents’ ongoing actions.

Infant Action Prediction of Everyday Food Preparation

The fact that all natural language quantifiers are conservative raises the question of whether people could hypothetically learn the conservative quantifiers more easily than the non-conservative ones. Some developmental studies attempted to answer this question, yet they did not reach any consistent results. This study offers an insight into this debate by investigating the learning of four unnatural language quantifiers with an eye-tracking experiment. This experiment employs the occluded referent paradigm, allowing us to identify which referents, hypothetical parts of the sets that the quantifiers relate to, the participants address when discovering the meanings of the quantifiers. The results show that when figuring out the quantifiers’ meanings, people refer to the lexically-related referents instead of limiting their hypothesis space based on conservativity. This implies that difficulty of learning is associated with the number and lexical-relatedness of referents that the quantifier is related to, rather than conservativity.

Learning Unnatural Language Quantifiers

Sign languages are visual-spatial languages, articulated not only by hands but also with facial expressions and other body parts. Facial expressions, in particular, have important grammatical roles, such as defining whether a sentence is a question or not and differentiating the meaning of a sign when assuming the role of a quantifier. Despite the importance of facial expressions, technologies to recognize and generate sign language mainly focus on manual signs, ignoring the information from facial expressions. In this work, we focus on collecting and annotating data to evaluate the role of facial expressions in communicating adjectives. We study the role of facial muscle activity to express the intensities of manual signs and show how modeling facial action units change the overall meaning of a sentence. We also explore how facial expressions allow a deeper understanding of sign language when included in the design of machine learning models.

Look For Adjectives In the Face: How Facial Expressions Contribute To Meaning In Signed Languages

A non-trivial number of the world’s languages have grammatical gender linked to biological sex. It is known that grammatical gender classification is not arbitrary, often following a semantic “core”. We investigate whether grammatical gender systems exhibit gender bias in different semantic domains, particularly terminologies of technology (e.g., microcomputer), a domain that has been considered male-dominant, and disease terminologies (e.g., hepatitis), a domain susceptible to negative connotations. In an initial
analysis with five languages, we found a baseline bias towards masculine classification of nouns compared to feminine. The technology domain showed no significant bias in grammatical gender with respect to the baseline, but two of the studied languages, French and German, showed statistically significant biases towards feminine classification in disease terminologies, while Arabic showed a significant bias towards masculine classification. These initial findings suggest that languages may be subject to domain general and specific gender biases through overt grammatical features.

Gender bias in grammatical gender systems across languages

Developing accurate models of word meaning requires good empirical evidence about what words mean. I investigate how English verbs encode relationships between event participants, focusing on how verbs encode instruments (e.g., does slice specify that a tool must be used for slicing?). I compared two commonly used indices of verb meaning: linguistic judgments and sentence completions. Although these two indices were moderately correlated for a small sample of verbs, they were only weakly correlated when a larger sample of verbs was tested. These results indicate that the particular context of a task can strongly influence how meaning affects behavior. Dominant models of verb meaning fail to fully account for the results (either a logical entailment model or a cue-based model).

Measuring Meaning: Alignment and Misalignment Across Indices of Verb Semantics

The paper investigates whether English native speakers (NSs)’ and non-native speakers (NNSs)’ online processing of prepositional phrases were primed by schematic diagrams and whether the priming effects were modulated by different prepositions and senses. 125 adult NS and NNS participants judged on over and in phrases that encode spatial and extended senses in a semantic priming task. Results showed an inhibitory effect of diagrams on NNSs’ processing but a marginal facilitatory effect on NSs’ processing. In addition, the inhibitory effect of diagrams only applied to the processing of extended senses, but not spatial senses. The current findings indicate that NNSs’ processing of English PPs could be negatively influenced by schematic diagrams particularly when the target phrases are more complex (Boers, 2011), presumably due to NNSs’ more limited cognitive capacity of processing complex second language expressions under time pressure than NSs.

Schematic Diagrams and Semantic Processing of English Prepositional Polysemy

The paper explores the impact of social role assignment and the corresponding payoff distribution on cooperation in the Prisoner’s dilemma following the types of relations according to Fiske’s relational models’ theory: communal sharing, authority ranking, equality matching, and market pricing. The corresponding roles and payoff distribution are teammates (each player receives the sum of the payoffs), partners (each player receives half of the sum of the payoffs), boss and subordinate (the boss receives 2/3 and the subordinate 1/3 of the sum of the payoffs), and opponents (each player receives the standard payoff). The results show that in the teammates’ and partners’ conditions cooperation was significantly higher than in the other conditions. Surprisingly, the results for the boss and subordinate condition although sharing a similar payoff distribution rule to the teammates’ and partners’ conditions were more similar to the opponents’ condition with significantly lower cooperation rate.

Can Social Relations Influence Cooperation in Prisoner’s Dilemma?

The Shapley value is one of the most important normative division schemes in cooperative game theory, satisfying basic axioms. However, some allocation according to the Shapley value may seem unfair to humans.
In this paper, we develop an automatic method that generates intuitive explanations for a Shapley-based payoff allocation, which utilizes the basic axioms.
Given any coalitional game, our method decomposes it to sub-games, for which it is easy to generate verbal explanations, and shows that the given game is composed of the sub-games. 
Since the payoff allocation for each sub-game is perceived as fair, the Shapley-based payoff allocation for the given game should seem fair as well.
We run an experiment with 630 human participants and show that when applying our method, humans perceive the Shapley-based payoff allocation as more fair than the Shapley-based payoff allocation without any explanation or with explanations generated by other methods.

Improving the Perception of Fairness in Shapley-Based Allocations

Linguistic similarities increase cooperative behavior between people, while linguistic differences can reduce cooperativeness, and even promote aversion towards outsiders. This may not only be true for sharing the same native language, but also for more subtle similarities such as using similar syntactic constructions. In our experiment, participants communicated with two interaction partners (which were in fact standardized chat bots), one of which syntactically aligned their language with the participants and one of which didn’t. Afterwards, participants decided with which of the two partners they would like to play a cooperation game that determined the financial bonus that they would receive for participation. Against our predictions, linguistically aligned communication partners were not preferred as cooperation partners over non-aligned communication partners. This suggests that formal linguistic similarity may be less relevant for the outcome of negotiations than previously expected, and has real-life implications for conflict resolution, especially in intercultural communication settings.

Premium content

Downloads

Next from CogSci 2022

Altercententric interference vs. bias in 7.5 month-old infants: a pupillometry study

Similar lecture

Rethinking Boundaries: End-To-End Recognition of Discontinuous Mentions with Pointer Networks