United States

This study aimed to investigate whether people produce spontaneous gestures that reflect the underlying linguistic structures of speech. Participants were asked to convey information about an animation movie using a three-word phrase in Japanese that could be interpreted in two different ways. The results showed that most participants produced gestures while uttering the phrase and the onset of nouns reflected the underlying linguistic structures and verb types. Time-series analysis revealed that occurrence of object action gestures that depict a noun’s movement tended to reflect the linguistic structures. Additionally, we observed that participants produced additional gestures that were not described in the controlled utterance. These results suggest that people spontaneously produce gestures that can both syntactically and semantically disambiguate speech.

CogSci 2023

Spontaneous co-speech gestures reflect linguistic structures

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Modification is required to differentiate potential referents in discourse context and enhances future memory for those referents. Not yet known is whether the type of modifier produced by younger and older adults differentially affects their object memory. We investigated the use of modifiers and whether it affects memory in younger and older adults. Further, it was examined whether the effects vary depending on the type of modifiers produced, namely color versus state. Participants were asked to describe an object that was accompanied by a same-category object of different color or different state, or an unrelated object. A follow-up memory task then assessed their recognition memory. Older adults overspecified more than younger adults. Although modifiers improved memory for both age groups, older adults showed better memory performance. The current finding suggests a link between language production and memory but we did not observe evidence that specific types of modifiers affected memory.

M & M: Modifiers and their Effect on Memory in Younger and Older Adults

It has been argued that cognitive processes such as attention and memory are influenced by motivational salience (high or low predictability of an outcome) rather than valance or value (gain or loss). However, whether this holds for subjective time perception remains unclear. To investigate this, a two-phase study was conducted. First, in a value learning task, a set of neutral faces was imbued with different levels of motivational salience (high or low) crossed with two levels of value (gain, loss). Thus, a specific face could acquire, for example, high motivational salience and low value by repeatedly signaling an 80% chance of losing points. Faces with these learned associations were then presented as an oddball for a varying duration (300-700 ms in steps of 50 ms) in a sequence of standard stimuli previously seen but predictive of no outcome (each presented for 500 ms). Participants assessed if the oddball was longer or shorter than the duration of the standard stimuli. The results show that an oddball associated with a high predictable outcome (80% chance of gain or loss) was perceived to last longer than an oddball associated with a low predictable outcome (20% chance of gain or loss). The present study supports previous studies by showing that a high motivational outcome, rather than valance/value, changes cognitive engagement with a stimulus. Attention may play an essential role in this interaction by modulating the subjective expansion of time.

Learned-predictiveness, not valence, modulates time perception

Perceptual training has been shown to be an effective and rapid way of training people to make simple diagnoses using medical images. However, it appears to be less effective at training people to make more complex diagnoses that require non-binary judgements. In the present study, we investigated whether perceptual training could be augmented to make it more effective and what factors limited its effectiveness. In Experiment 1, we created artificial stimuli that were designed to simulate liver ultrasound images to assess perceptual learning for a complex task that involved judgements on a 7-point scale. Whilst performance improved somewhat with training, we found that incorporating annotations into the training provided no benefits. Additionally, contrary to our expectations, training that was structured in a stepped fashion was detrimental to learning. In Experiment 2, we found that perceptual learning in a simple task with shaded disks was most impacted by the extent to which the brightness levels of each disk were discriminable but that attending to multiple locations did not result in a significant cost to performance. Our findings show that augmenting perceptual training does not increase learning and that learning is less when the relevant features are harder to discriminate.

Testing the Effectiveness of Augmenting Perceptual Training With Annotations and Steps in a Difficult Visual Discrimination Task

Predictive inference and error-driven learning are critical to optimal performance across many different contexts. However, the specific context determines the informativeness of errors in updating predictions. In this study, participants experienced two changing, unsignalled contexts with opposite optimal responses to errors; the change-point context, where errors were informative, and the oddball context, where they were not. The changes to the context occurred under two task structures: 1) a fixed task structure, with consecutive blocks of each context, and 2) a random task structure, with the context randomly selected for each new block. We modelled participants' performance using a Hierarchical Gaussian Filter (HGF) model. We found that performance was greater in the oddball than change-point context, with more accurate and precise estimates. The estimates from the fixed task structure were also more precise than those in the random task structure. We showed that consistency in context can improve precision.

Prediction and learning under unsignalled changing contexts

Intelligent tutoring systems (ITS) afford complex learning environments in which inductive reasoning is engaged, but where sparse log data complicates the classification of cognitive states. Using a simple rule learning task, we explore the feasibility of verbal think aloud protocols to contribute to machine learning algorithms designed to classify states of rule search, rule discovery, and rule following. Domain-general versus domain-specific contributions to classification are considered through the use of isomorphic rule learning tasks with numeric and spatial stimuli. We trained and tested models both within and across task domains. Models including verbal data outperform models based only on log data with particular improvement in classifying rule discovery. These results provide a foundation for real-time classification of cognitive states during ITS use.

Classification of Rule Learning Phases in Inductive Reasoning

Human physical scene understanding requires more than simply localizing and recognizing objects — we can quickly adapt our predictions about how a scene will unfold by incorporating objects' latent physics properties, such as the masses of the objects in the scene. What are the underlying computational mechanisms that allow humans to infer these physical properties and adapt their physical predictions so efficiently from visual inputs? One hypothesis is that general intuitive physics knowledge can be learned from enough raw data, instantiated as computational models that predict future video frames in large datasets of complex scenes. To test this hypothesis, we evaluate existing state-of-the-art video models. We measured both model and human performance on Physion++, a novel dataset and benchmark that rigorously evaluates visual physical prediction in humans and machines, under circumstances where accurate physical prediction relies on accurate estimates of the latent physical properties of objects in the scene. Specifically, we tested scenarios where accurate prediction relied on accurate estimates of objects' mechanical properties, including masses, friction, elasticity and deformability, and the values of these mechanical properties could only be inferred by observing how these objects moved and interacted with other objects and/or fluids. We found that models that encode objectness and physical states tend to perform better, yet there is still a huge gap compared to human performance. We also found most models' predictions correlate poorly with that made by humans. These results show that current deep learning models that succeed in some settings nevertheless fail to achieve human-level physical prediction in other cases, especially those where latent property inference is required.

Physion++: Evaluating Physical Scene Understanding with Objects Consisting of Different Physical Attributes in Humans and Machines

Agents trained using deep reinforcement learning (DRL) are capable of meeting or exceeding human-levels of performance in multi-agent tasks. However, the behaviors exhibited by these agents are not guaranteed to be human-like or human-compatible. This poses a problem if the goal is to design agents capable of collaborating with humans in cooperative or team-based tasks. Previous approaches to encourage the development of human-compatible agents have relied on pre-recorded human data during training. However, such data is not available for the majority of everyday tasks. Importantly, research on human perceptual-motor behavior has found that task-directed behavior is often low-dimensional and can be decomposed into a defined set of dynamical perceptual-motor primitives (DPMPs). Accordingly, we propose a hierarchical approach to simplify DRL training by defining the action dynamics of agents using DPMPs at the lower level, while using DRL to train the decision-making dynamics of agents at the higher level. We evaluate our approach using a multi-agent shepherding task used to study human and multi-agent coordination. Our hierarchical, DRL-DPMP approach resulted in agents which trained faster than vanilla, black-box DRL agents. Further, the hierarchical agents reached higher levels of performance not only when interacting with each other during self-play, but also when completing the task alongside agents embodying models of novice and expert human behavior. Finally, the hierarchical DRL-DPMP agents developed decision-making policies that outperformed heuristic-based agents used in previous research in human-agent coordination.

Scaffolding Deep Reinforcement Learning Agents using Dynamical Perceptual-Motor Primitives

Our memories are stored not in isolation, but in richly structured networks of knowledge. How are individual memories represented within these knowledge structures, and how do new experiences become integrated into existing structures? To address these questions, we exposed participants to novel objects from a hierarchical category structure. After participants learned these objects to criterion, we introduced new objects from the same underlying structure. Participants learned the new objects much more quickly than the old set, suggesting that recently learned knowledge rapidly scaffolded the learning of new, related information. However, fine-grained analyses of participants’ memory errors suggest these newly learned objects may not be fully integrated into prior learned category structure, but rather are held separate. These results suggest that existing knowledge can facilitate the learning of new information in the absence of assimilation of that information. Ongoing work investigates how these dynamics unfold over time and with sleep.


Measuring the rapid acquisition and integration of structured knowledge

One of the most ancient and universal forms of cognitive offloading is the outsourcing of cognitive operations onto other humans. We explored mechanisms underlying this ability by designing a novel computerised memory task where participants memorised target locations and recalled them after a brief delay. Next, they watched two virtual people compete in another memory game, where one demonstrated a reliable memory, and the other demonstrated an unreliable memory. Finally, participants completed the initial memory task again, with either the reliable-memory or unreliable-memory person being available to help with recall on each trial. Through observation and without direct instruction, participants acquired beliefs about the virtual people’s cognitive proficiencies and could readily draw upon these beliefs to inform offloading decisions. Participants were more likely to seek help from the reliable-memory person, and this tendency interacted with factors known to drive cognitive offloading more generally, like task difficulty and unaided cognitive ability.

Can you help me? Using others to offload cognition

How much can we infer about the past and future, given our knowledge of the present? Unlike temporally symmetric inferences about simple sequences, inferences about our own lives are asymmetric: we are better able to infer the past than the future, since we remember our past but not our future (i.e., the psychological arrow of time). What happens when both the past and future are unobserved, as when we make inferences about other people’s lives? We had participants view segments of a character-driven television drama. They wrote out what would happen just before or after each just-watched segment. Participants were better at inferring past (versus future) events. This asymmetry was driven by participants’ reliance on characters’ conversational references in the narrative, which tended to favor the past. Our work reveals a temporal asymmetry in how observations of other people’s behaviors can inform us about the past and future.

Downloads

Next from CogSci 2023

M & M: Modifiers and their Effect on Memory in Younger and Older Adults

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from CogSci 2023

M & M: Modifiers and their Effect on Memory in Younger and Older Adults

Downloads