United States

Words express various sensory information to various degrees. Norming studies have collected native speakers’ subjective perceptual strength ratings for numerous words in several languages. This paper presents perceptual strength norms for 510 Japanese words, including iconic lexemes called ideophones. The newly collected norms replicated some previously reported correlations, such as the one between certain sensory modalities and iconicity. A systematic comparison between the Japanese and English perceptual strength norms further revealed that Japanese onomatopoeic ideophones tend to be more multisensory than their English equivalents and that Japanese words in general tend to encode more interoceptive information than English words. These findings suggest the usefulness of norming data in linguistic typology.

CogSci 2023

Perceptual Strength Norms for 510 Japanese Words, Including Ideophones: A Comparative Study with English

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

How similar are people's meanings of common words? Do differences in word meanings lead to miscommunication? We examined divergence between word meanings using similarity ratings such as "Is a penguin more similar to a chicken or a dolphin". We found that given the identical instructions some people prioritized taxonomic relationships more than other people. Moreover, this taxonomic bias generalized from animals to artifacts suggesting a more general difference in semantic organization. When people with different biases are paired in a matcher-director type task, they were less likely to achieve communicative success.

Divergence in Word Meanings and its Consequence for Communication

English and Mandarin Chinese are two distinct languages in many aspects, such as orthography and morphology. Previous network analyses show strong clustering coefficients ($C$) on English semantic networks, revealing the interconnectedness of semantic representations between words. However, it is not clear whether such semantic representation properties are language specific or general, and whether the linguistic-feature difference (e.g., subword components such as orthography and morphology) may affect the lexico-semantic structure. Here, we compared $C$s of words in English and Mandarin semantic networks based on a) feature norms empirically derived from human subjects and b) distributed semantic information of text retrieved by word embedding models. We consistently observed higher $C$s of Mandarin words than English words, especially when the semantic network considers subword features. Linear regressions suggested that the subword components' semantic properties in Mandarin, but not in English, could significantly and positively predict the $C$ of words in semantic networks. The results indicate an important role of language-specific properties in lexico-semantic structures and imply the diversity of human language processing.

Large-scale Network Analyses Reveal Cross-Language Differences in Semantic Structures: A Comparative Study

In two-person asymmetric coordination dilemmas, both people are better off if they coordinate, but one person benefits more than the other. When these interactions repeatedly recur, people can form expectations to avoid failing to coordinate, and to balance who is better off over time. What does it mean when asymmetric social interactions recur, and what can we learn from how people solve these dilemmas? We hypothesize that people expect social interactions to recur when two people are in a social relationship, and that knowing about the symmetry of the social relationship influences the stable solution to asymmetric coordination dilemmas over time. We report two experiments where participants read stories and answered questions about social interactions between two people. In Experiment 1, participants infer that two people are in a social relationship when there is a sequence of altruistic interactions between them, and specifically infer an asymmetric relationship when one person always performs the altruistic action, and a symmetric relationship when the two people alternate performing the altruistic action. In Experiment 2, participants equally expect alternating and repeating altruistic actions when the relationship is symmetric, but expect repeating actions (following a precedent) when the relationship is asymmetric. Our results suggest that people are able to use knowledge of relationships to generate shared expectations for coordinating on recurring altruistic social interactions, and vice versa.

People have systematic expectations linking social relationships to patterns of reciprocal altruism

Collaboration in shared environments requires human agents to coordinate their behaviour according to the machines’ actions. In this study, we compared the performance and behaviour of Human-Machine (HM) and Human-Human (HH) teams. While HH teaming behaviour is sensitive to Collaborative contexts, little is known about HM teaming behaviour. Furthermore, teaming behaviour may impact the team’s Joint Capacity – the team’s ability to handle teamwork processes and task demands. To assess teaming behaviour at every moment of a trial we used three distinct spatiotemporal measures (Momentary Distance, Highly Correlated Segments, and Running Correlation). To assess the team’s joint performance, we adopted the Capacity Coefficient (Townsend & Nozawa, 1995). For both HH and HM teams, behavioural measures predicted Joint Capacity. HH teams demonstrated greater performance and less synchronous behaviour than HM teams. The reduced synchrony of HH teams likely improved their performance as they could complement each other’s behaviour rather than duplicate inefficiencies.

The Relationship Between Teaming Behaviours and Joint Capacity of Hybrid Human-Machine Teams

Intention prediction often plays a crucial role in successful social interaction. Previous studies have attempted to understand this skill by focusing on the role of movement kinematics in isolation. However, this approach is limited as the same kinematics typically map to multiple action possibilities (affordances) and as a result, individual's also employ contextual information to predict others' intentions. In this study we present preliminary findings from a qualitative study aimed at investigating intention prediction in naturalistic contexts. Participants viewed an individual reaching for a cup with one of two object-directed intentions: to drink OR to clear the table. A third non-object-directed intention was also included where the observed individual placed their hand on the table next to the cup. For each intention the contextual information was varied by changing the environmental scene between (1) cups full of juice, (2) almost empty cups, and (3) half-empty cups. The findings reveal that participants perceived the cup's functional (most salient) affordance - drink - for the intention so far as the movement kinematics specified an object-directed intention (drink or clear) with a context that clearly afforded it (full and half-full cups). However, participants were also sensitive to the kinematic differences between the object-directed intentions when the context made the functional affordance seem improbable (almost empty cups). The methodological and theoretical implications of the results with regards to studying intention prediction are discussed.

Predicting intentions: How do we predict other's action intentions?

Collaboration and teaming are critical for solving complex problems. However, little is known about how group dynamics affect teaming behaviours and, ultimately, problem-solving effectiveness. The present study aimed to validate a novel measure of the dynamics of team communication – here termed turn-taking entropy – and to investigate what aspects of those dynamics affect collaborative-problem-solving performance. Thirty-two teams of 4 were asked to complete a simulated crisis-response task in which they had to rank 15 items in order of their importance to their team’s survival (first individually and then as a team). Group responses were better than the aggregated individual responses of team members (suggesting teaming benefits), and were better when team members had task-relevant skills and knowledge. However, response quality was not significantly related to task completion time. Additionally, the proposed entropy measure appeared to capture group communication dynamics, and appeared to differentiate stable an unstable patterns of communication. Implications and directions for future research are discussed.

The Entropy of Communication Turn Taking during a Collaborative Problem Solving Task

Understanding the extent to which the perceptual world can be recovered from language is a fundamental problem in cognitive science. We reformulate this problem as that of distilling psychophysical information from text and show how this can be done by combining large language models (LLMs) with a classic psychophysical method based on similarity judgments. Specifically, we use the prompt auto-completion functionality of GPT3, a state-of-the-art LLM, to elicit similarity scores between stimuli and then apply multidimensional scaling to uncover their underlying psychological space. We test our approach on six perceptual domains and show that the elicited judgments strongly correlate with human data and successfully recover well-known psychophysical structures such as the color wheel and pitch spiral. We also explore meaningful divergences between LLM and human representations. Our work showcases how combining state-of-the-art machine models with well-known cognitive paradigms can shed new light on fundamental questions in perception and language research.

What Language Reveals about Perception: Distilling Psychophysical Knowledge from Large Language Models

Spurious correlations were found to be an important factor explaining model פerformance in various NLP tasks (e.g., gender or racial artifacts), often considered to be ''shortcuts'' to the actual task.
However, humans tend to similarly make quick (and sometimes wrong) predictions based on societal and cognitive presuppositions.
In this work we address the question: 
can we quantify the extent to which model biases reflect human behavior?
Answering this question will help shed light on model performance and provide meaningful comparisons against humans. 
We approach this question through the lens of the dual-process theory for human decision-making. This theory differentiates between an automatic unconscious (and sometimes biased) ''fast system'' and a ''slow system'', which when triggered may revisit earlier automatic reactions. 
We make several observations from two crowdsourcing experiments of gender bias in coreference resolution, using self-paced reading to study the ''fast'' system, and question answering to study the ''slow'' system under a constrained time setting.
We find that humans and models show comparable biases. 
On real-world data humans make ~3% more gender-biased decisions compared to models, while on synthetic data models are ~12% more biased.

Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution

In this work, we use question answering as a general framework for studying how eye movements in reading reflect the reader's goals, how they are pursued, and the extent to which they are achieved. We leverage fine-grained annotations of task-critical textual information to perform a detailed comparison of eye movements in information-seeking and ordinary reading regimes. We further examine how eye movements during information seeking relate to question answering behavior. We find that reading times, saccade patterns and sensitivity to the linguistic properties of the text are all strongly and systematically conditioned on the reading task, and further interact with question answering behavior. The observed reading patterns are consistent with a rational account of cognitive resource allocation during task-based reading.


Eye Movements in Information-Seeking Reading

Normative accounts of decision-making predict that people attempt to balance the immediate rewards associated with correct responses against the costs of deliberation. However, humans frequently deliberate longer than normative models say they should. We propose that people try to optimize not only their rate of material rewards, but also their rate of information gain. A computational model that implements this idea successfully mimics human decision makers, reproducing key patterns of behavior not predicted by alternative models. Moreover, simulations reveal a normative basis for our model: An agent that exchanges even a small amount of immediate reward for information will improve its decision-making ability through learning, allowing it to earn more reward in the long run than an agent disinterested in information. Maximizing a combination of reward and information rate is a simple yet effective strategy for solving the speed-accuracy tradeoff that may resolve lingering mysteries about human decision-making.

Premium content

Perceptual Strength Norms for 510 Japanese Words, Including Ideophones: A Comparative Study with English

Next from CogSci 2023

Divergence in Word Meanings and its Consequence for Communication