United States

Many models of intuitive physical reasoning posit some kind of mental simulation mechanism, yet everyday environments frequently contain far more objects than people could plausibly represent with their limited cognitive capacity. What determines which objects are actually included in our representations? We asked participants to predict how a ball will bounce through a complex field of obstacles, and probed working memory for objects in the scene that were more and less likely to be relevant to the ball’s trajectory. We evaluate different accounts of relevance and find that successful object memory is best predicted by how frequently a ball’s trajectory is expected to contact that object under a probabilistic simulation model. This suggests that people construct representations for
mental simulation efficiently and dynamically, on the fly, by adding objects “just in time”: only when they are expected to become relevant for the next stage of simulation.

CogSci 2023

&quot;Just In Time&quot; Representations for Mental Simulation in Intuitive Physics

"Just In Time" Representations for Mental Simulation in Intuitive Physics

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Computational models of perceptual decision-making depend heavily on empirical goodness-of-fit measures for model selection. However, it is not possible to improve models' fit to data indefinitely, particularly when the data in question are variable across multiple elicitations. The completeness of a model or a theory assesses the extent to which it can predict observations in comparison with an ideal model. We measure the completeness of contemporary race models on a random dot motion discrimination task and show that the simple drift diffusion model is already close to complete in describing random dot motion discrimination data, with more complex models being in fact over-fit to datasets. We also estimate the retest reliability of the random dot motion discrimination task as part of the exercise of measuring completeness, and describe some implications of our estimates for trial-level neural models of this task.

Measuring the completeness of race models for perceptual decision-making

Humans are surrounded by various typefaces, and the preliminary corpus analysis suggests that typefaces commonly used in picture books differ from others, which implies that familiar typefaces differ between children and adults. Thus, this study investigated whether children can easily recognize fonts considered familiar to them. Japanese-speaking children (aged 7–8 years) performed a hiragana-character discrimination task, which includes different (e.g., あ–お) and same (e.g.,あ–あ) character–pair conditions. It involves various typefaces familiar to children (i.e., picture book typefaces) or adults; the study indexed discriminability using averaged reaction times to the character–pair conditions. Accuracy was sufficiently high for all typefaces (>90%), and no significant difference existed in response times between any typeface. The results suggested that children can stably discriminate characters regardless of whether or not a font is commonly used in picture books.

Discriminability of picture book typeface in children

Reader’s ability to integrate words into the preceding linguistic context via inferences making is one fundamental component of reading comprehension. However, the underlying processes that contribute to effective word-to-text integration are not yet fully understood. In this study, we investigated the contribution of variables underlying lexical representations (e.g., vocabulary), and general reading skills (text comprehension) to the integration of words across sentences in a sample of adolescents. We implemented an ERP experiment, in which we contrasted two sentential context conditions: a repetition condition and an inference condition. The cognitive cost of inferential processing was indexed by the N400 component in both conditions. Participants’ individual differences in vocabulary and reading comprehension were included to examine the role of different components in this process. The results showed that participants with lower vocabulary exhibited a larger and extended cognitive cost of inference-making in the N400 component relative to participants with a higher vocabulary.

Integration of Words to Previous Linguistic Context is Modulated by Vocabulary: An ERP study

If we want to know whether a stack of blocks will fall, we might form our judgment by using mental simulation or simple heuristics. How do we know which strategy to use? Here, we test whether people’s strategy selection is “resource rational,” balancing their utilities and costs. We had participants judge how a balance beam will fall – a task that relies on both simulation and heuristics – and imposed visuospatial cognitive load to increase the cost of mental simulation, expecting the use of simulation to then decrease and the use of heuristics to increase. When cognitive load varied between participants, higher load led to less simulation. However, when load varied across trials for an individual participant, they instead used simulation at about the same rate as high-load-only participants. Thus strategy selection is sensitive to cognitive costs but it is also sticky, not accounting for rapidly fluctuating costs.

Strategy choice for physical reasoning is (partially) sensitive to cognitive costs

Robots must be able to communicate naturally and efficiently, e.g., using concise referring forms like it, that, and the ⟨N’⟩. Recently researchers have started working on Referring Form Selection (RFS) machine learning algorithms but only evaluating them offline using traditional metrics like accuracy. In this work, we investigated how a cognitive status-informed RFS computational model might fare in actual human-robot interactions in a human-subjects study (N=36). Results showed improvements over a random baseline in task performance, naturalness, understandability, and mental workload. However, the model was not perceived to outperform a simple, naive, non-random baseline (constant use of indefinite noun phrases). We contribute several key research directions for further development of cognitive status-informed RFS models, the inclusion of multi-modality, and further development of testbeds.


Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions

Androids that strongly but imperfectly resemble humans in shape can elicit negative emotions in people, a phenomenon known as the "uncanny valley," which has been replicated in laboratory experiments. Recently, the accuracy of face recognition utilizing machine learning has increased, raising the question of whether machine learning can replicate the uncanny valley. Using FaceNet as a representative face recognition algorithm, we examined the similarity of face recognition to human evaluation and its replication of the uncanny valley. The results revealed a strong correlation between machine learning and human evaluation of human-like shapes. However, it is evident that only certain aspects of the uncanny valley were replicated. Furthermore, visualization of the activation maps suggests that localized regions, such as the mouth and chin, acted as the basis for judgment. These findings support the idea that human and machine learning have distinct areas of attention.

Does Machine Learning Replicate the Uncanny Valley? An Example using FaceNet

Working Memory (WM) plays a key role in natural language understanding and generation. To enable a human-like breadth and flexibility of language understanding and generation capabilities, cognitive systems for language-capable robots should feature a human-like WM system in a similarly central role. However, it is still quite unclear how robotic WM should be designed, as a variety of models of human WM have been proposed in cognitive psychology. Moreover, human reliance on WM during language production is sometimes to help the speaker rather than to help hearers. Thus, it is unclear whether different robotic WM systems might harm certain dimensions of interaction for the sake of the robot speaker's ostensible ease of cognitive processing. In this paper we demonstrate how different models of human WM can be implemented into robot cognitive architectures. Our results suggest that these models can be effective in terms of accuracy, perceived naturalness, and perceived human-likeness.

Forget About It: Entity-Level Working Memory Models for Referring Expression Generation in Robot Cognitive Architectures

Event cognition research has typically considered events to be contiguous in time, with defined starts and ends. However, people sometimes engage in more than one event at the same time. If this happens frequently, then theories of event cognition may require modification. This research study aims to estimate how often people engage in multitasking in daily life. Ninety-seven participants were asked whether they had been multitasking at four time points during the last 24 hours. Forty-five per cent of responses reported multitasking with a diverse range of event structures. Twenty-one per cent of reports specifically listed multiple overlapping activities. The prevalence of multitasking suggests that theories of event cognition need to be expanded to accommodate non-contiguous and simultaneous events.

The prevalence of multitasking presents challenges for theories of event segmentation

The ability to represent and reason about other people’s perspectives (‘social cognition’, or ‘Theory of Mind’ [ToM]) plays an important role in daily life. It is not currently clear how different types of interaction may impact subsequent perspective taking, and whether this differs with age. Here, we examine how experiences of social inclusion vs. social exclusion (manipulated using the Cyberball task) influence later perspective-taking efficiency, in younger (19-30 years; N = 100; M = 24.69) and older (65-80 years; N = 100; M = 69.63) participants. As expected, significant age differences were found in perspective-taking abilities, with younger participants out-performing older participants. This was specific to false-belief trials, with no difference in memory trials performance. Counter to hypothesized, no interaction between impact of social experience (inclusion vs. exclusion) and age was found, indicating that both younger and older adults are similarly impacted by social experiences. Implications will be discussed.

The Impact of Social Inclusion vs. Exclusion on Subsequent Social Cognition Abilities in Younger and Older Adults

People care about others’ welfare to varying degrees, captured by the welfare tradeoff ratio (WTR), or the weight placed on another person’s welfare compared to one’s own. People can infer another person’s WTR toward themselves and reciprocate by adjusting their own WTR toward that person. However, the evolutionary origin of such a capacity is unclear. In games with perfect information, a heuristic strategy with tit-for-tat-like reciprocity is unbeatable, and the additional computation of inferring the opponent’s WTR confers no benefit. Here we show that in games with asymmetric information, where the actor has more accurate information about the payoff structure than the observer, the heuristic strategy is prone to errors from misperception, while reciprocity based on the inference of WTRs is robust. These findings suggest that asymmetric information about social decisions, a realistic modification to the game environment, may have contributed to the evolution of people’s understanding of others’ WTRs.


Premium content

"Just In Time" Representations for Mental Simulation in Intuitive Physics

Next from CogSci 2023

Measuring the completeness of race models for perceptual decision-making