United States

The ability to discriminate between large and small quantities is a core aspect of basic numerical competence in both humans and animals. In this work, we examine the extent to which the state-of-the-art neural networks designed for vision exhibit this basic ability. Motivated by studies in animal and infant numerical cognition, we use the numerical bisection procedure to test number discrimination in different families of neural architectures. Our results suggest that vision-specific inductive biases are helpful in numerosity discrimination, as models with such biases have lowest test errors on the task, and often have psychometric curves that qualitatively resemble those of humans and animals performing the task. However, even the strongest models, as measured on standard metrics of performance, fail to discriminate quantities in transfer experiments with differing training and testing conditions, indicating that such inductive biases might not be sufficient.

CogSci 2023

Evaluating Visual Number Discrimination in Deep Neural Networks

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference.<br>
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

<br>[**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
<br>[**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
<br><br>

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>
<br>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Recent work in computer vision and cognitive reasoning has given rise to an increasing adoption of the Violation-of-Expectation (VoE) paradigm in synthetic datasets. Inspired by infant psychology, researchers are now evaluating a model’s ability to label scenes as either expected or surprising with knowledge of only expected scenes. However, existing VoE-based 3D datasets in physical reasoning provide mainly vision data with little to no heuristics or inductive biases. Cognitive models of physical reasoning reveal infants create high-level abstract representations of objects and interactions. Capitalizing on this knowledge, we established a benchmark to study physical reasoning by curating a novel large-scale synthetic 3D VoE dataset armed with ground-truth heuristic labels of causally relevant features and rules. To validate our dataset in five event categories of physical reasoning, we benchmarked and analyzed human performance. We also proposed the Object File Physical Reasoning Network (OFPR-Net) which exploits the dataset's novel heuristics to outperform our baseline and ablation models. The OFPR-Net is also flexible in learning an alternate physical reality, showcasing its ability to learn universal causal relationships in physical reasoning to create systems with better interpretability.

A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories

In cognitive science, we often assume that human learning is characterized by a limited capacity to represent the information relevant for a given task. We argue that excess capacity --- using more representational resources than needed for a task at hand --- is a plausible alternative paradigm for the study of human learning. Through a review of recent results from machine learning, we show that excess capacity is consistent with high predictive ability. Through review of cognitive science literature, we demonstrate that excess capacity learning can account for a range of empirical phenomena, such as humans' simultaneous yet apparently contradictory tendency to both memorize observations and capture higher-level patterns in them, as well as a tendency to learn complex and redundant solutions to the tasks. We conclude by discussing promising directions for future inquiry under the excess capacity learning paradigm.

Excess Capacity Learning

Efficient representation of our motor memories is essential for developing and deploying the plethora of human motor skills we use every day, yet there is little cohesion in understanding how these representations are organized and develop over time. In this study, we developed a novel behavioral paradigm aimed at probing experience-dependent representational change of a motor skill with training. Across several experiments, participants trained to coordinate novel finger configurations in a speeded reaction time task and performed a similarity judgment task to probe for changes in the psychological representation of the recently acquired skill. We sought to determine if learned representations of action can be found through behavioral methods and if such representations are organized based on similarity and independent reorganization patterns predicted by theories of sensorimotor neuroscience. Together, we propose a novel behavioral measure complementary to neuroimaging capable of observing human representational change of a motor skill over time.

Experience-Dependent Representational Change During Motor Skill Learning

In cognitive science, research about mental imagery is often limited to visual, often overlooking olfactory imagery. In this study, we examined the relationship between beauty and olfactory imagery evoked by haiku. We used an odor priming that is commonly used in cognitive science to measure olfaction so that we could examine the effects of environmental aromas on the aesthetic experience. 44 participants were asked to evaluate 30 haikus. Half of them were exposed to a cypress aroma while the other half had no aroma exposure. The results showed that olfactory imagery during haiku appreciation positively influenced the beauty of haiku, and higher olfactory imagery ability led to a deeper immersion in the haiku. Odor priming did not affect evaluations, but it did affect gaze bias as measured by eye tracking. This is the first time to demonstrate the influence of olfactory imagery on aesthetic evaluation in psychology of aesthetics.

Scent of Poetry: Influence of Olfactory Imagery during Haiku Appreciation on Aesthetic Evaluation

This study (n=64) investigated the influence of manipulating the face contour on the face inversion effect (better recognition performance for upright vs inverted upside-down faces) and on face recognition in general. The study used upright and inverted scrambled faces which suffer from disruption of the configural information (spatial relationships among the facial features) typically found in normal faces. This is because we aimed to isolate the face contour information from the configural information. A delayed- matching task was adopted to ensure a high level of recognition performance, especially for inverted faces. The results revealed no significant difference between the inversion effect for scrambled faces with the normal contour vs that for scrambled faces with the manipulated (blurred) contour. Critically, we found an effect on overall recognition performance whereby scrambled faces with the blurred contour suffered greatly from the manipulation. Our results suggest that the face contour affects overall face recognition performance.

Manipulating the face contour reduces overall recognition performance for scrambled faces

Aesthetic value, beauty, is a complex concept in that it has both subjective and objective aspects. However, in previous eye-tracking studies of artworks, the target associated with the gaze has often been only the latter image feature (e.g., symmetry). By contrast, recent developments in computational aesthetics (especially aesthetic classifiers) offers a path that treats these two aspects comprehensively. Along this line, we further develop eye-tracking research. We conducted computational-model-based data analyses of eye-tracking behavior, with the aim of providing more fine-grained insights into the comprehensive concept of beauty. In contrast to previous studies that focused on Western fine-artworks, our study is distinctive in that it focuses on visual art in Eastern culture, namely Japanese ukiyo-e (the well-known works of Katsushika Hokusai and Utagawa Hiroshige). Our empirical results showed that the gaze trajectory and movement area differed significantly between highly-rated and non-highly-rated paintings. This provides positive evidence for the relationship between aesthetic value and perceptual behavior.

Relating aesthetic-value judgment to perception: An eye-tracking and computational study of Japanese art Ukiyo-e

Perceptual features, either magnitude (size, area) or non-magnitude dimensions (color, shape), affect number representation by changing signal-to-noise ratio. Clarity of signal about numerical value may explain why comparing Arabic numerals is faster than comparing dots. The present study manipulated signal-to-noise ratio to investigate the effect of perceptual noise on symbolic numeral representation. Noise was quantified based on perceptual entropy of Arabic numerals, which were presented either in a single (low entropy) or three colors (high entropy). Thirty-eight adults were asked to compare two Arabic numerals, presented in single-single, single-three, and three-three colors. RT decreased as numerical ratio (b = -0.20) and entropy ratio (b = -0.11) increased, indicating symbolic representation was affected by entropy. RT increased as sum of numerical values (b = 0.08) and sum of entropy values (b = 0.06) increased. The results suggest perceptual noise impaired numerical judgment even though adults are highly familiar with Arabic numerals.

Relation between Perceptual Entropy and Symbolic Numeral Representation

We present a novel method for image saliency prediction that leverages a cognitive model of visual attention as an inductive bias.
This is in stark contrast to recent purely data-driven
models that have achieved performance improvements mainly by increased model capacity, resulting in high computational costs and the need for large scale, domain specific training data. 
We demonstrate that by leveraging a cognitive model of visual attention, our method achieves competitive performance to the state-of-the-art across several natural image datasets while only requiring a fraction of the parameters.
Furthermore, we set the new state of the art for saliency prediction on information visualizations, demonstrating the effectiveness of our approach for cross-domain generalization.
We further provide large-scale cognitively plausible synthetic gaze data on corresponding images in the full MSCOCO and FigureQA datasets, which we used for pre-training.
These results are highly promising and underline the significant potential of bridging between first-principle cognitive and data-driven models for computer vision tasks, potentially also beyond saliency prediction, and even visual attention.

\bf Keywords: neural networks, computational models of cognition, computer vision, saliency prediction, human visual attention

Improving neural saliency prediction with a cognitive model of human visual attention

Visual emergence is the phenomenon in which the visual system obtains a holistic perception after grouping and reorganizing local signals. The picture Dalmatian dog is known for its use in explaining visual emergence. This type of image, which consists of a set of discrete black speckles, is called an emerging image. Although Gestalt theory summarizes perceptual organization into several principles, it remains ambiguous how these principles affect the perception of emerging images. This study, therefore, designed three psychological experiments to explore the factors that influence the perception of emerging images. In the first, we found that the density of speckles in the local area and the arrangements of some key speckles played a key role in the perception of an emerging case. We set parameters in the algorithm to characterize these two factors. We then automatically generated emerging-test images through the algorithm and verified their effectiveness in two subsequent experiments.


Important Clues that Facilitate Visual Emergence: Three Psychological Experiments

Predictive coding has emerged as a prominent model of how the brain learns through predictions and prediction errors. Traditional predictive coding focused primarily on sensory coding and perception. Here we propose active predictive coding (APC), a unified framework for perception, action and cognition. By learning hierarchical world models, the APC framework addresses important open problems in cognitive science and AI such as: (1) how do we learn compositional representations, e.g., part-whole hierarchies for equivariant vision? and (2) how do we solve large-scale planning problems, which are hard for traditional reinforcement learning, by composing complex action sequences from primitive policies? APC exploits hypernetworks, self-supervised learning and reinforcement learning to learn hierarchical models that combine task-invariant state transition networks and task-dependent policy networks at multiple abstraction levels. We illustrate the applicability of the APC model to visual perception using a variety of vision datasets as well as to a scalable hierarchical planning problem. Our results represent, to our knowledge, the first demonstration of a unified approach to addressing the part-whole learning problem in vision, the nested reference frames learning problem in cognition, and the integrated state-action hierarchy learning problem in reinforcement learning.

Premium content

Evaluating Visual Number Discrimination in Deep Neural Networks

Next from CogSci 2023

A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories