United States

The primary goals of computer-assisted pronunciation training (CAPT) systems are to provide a personalized interactive environment and to accurately diagnose mispronunciations. Automatic speech recognition (ASR) systems have been shown to be an effective tool for diagnosing mispronunciations. While the data ASR systems output can be difficult for the layperson to understand, presenting it in a multimodal fashion can make it easier and feeding it into an automated narrative system can produce personalized feedback. In the absence of native speech examples, synthetic examples produced by text-to-speech (TTS) engines have proven to be an adequate substitute, making data collection easier and allowing for larger CAPT systems. In this work we present the SpeakEasy pronunciation trainer, a CAPT system that leverages ASR, TTS, automated narrative systems, and multimodal data representation to provide a personalized interactive environment that tracks a user&#39;s progress over time.

CogSci 2021

SpeakEasy Pronunciation Trainer: Personalized Multimodal Pronunciation Training

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

The primary goals of computer-assisted pronunciation training (CAPT) systems are to provide a personalized interactive environment and to accurately diagnose mispronunciations. Automatic speech recognition (ASR) systems have been shown to be an effective tool for diagnosing mispronunciations. While the data ASR systems output can be difficult for the layperson to understand, presenting it in a multimodal fashion can make it easier and feeding it into an automated narrative system can produce personalized feedback. In the absence of native speech examples, synthetic examples produced by text-to-speech (TTS) engines have proven to be an adequate substitute, making data collection easier and allowing for larger CAPT systems. In this work we present the SpeakEasy pronunciation trainer, a CAPT system that leverages ASR, TTS, automated narrative systems, and multimodal data representation to provide a personalized interactive environment that tracks a user's progress over time.

poster

Deictic communication is fundamentally multimodal. Spatial demonstratives frequently co-occur with eye gaze and physical pointing to draw the attention of an addressee to an object location (e.g. this cup; that chair). Yet the relative importance of language, gesture and eye gaze in deictic reference has not this far been elucidated. In three online experiments, we manipulated the congruency of pointing, gazing and verbal cues to establish their relative importance for demonstrative choice (Experiment 1) and choice of referent (Experiments 2 and 3). Participants saw an image with a person sitting behind a table, interacting with items placed proximally or distally relative to the pictured person, with manipulation of pointing, eye gaze and language (and congruence/incongruence of these modalities). While all three modalities affected demonstrative choice (Experiment 1) and referent choice (Experiments 2 and 3), results show that pointing is the dominant deictic cue to demonstrative/referent choice.

Action Speaks Louder than Words and Gaze: The Relative Importance of Modalities in Deictic Reference

People’s desire to seek or avoid information is not only influenced by the possible outcomes of an event, but the probability of those particular outcomes occurring. There are competing explanations however as to how and why people’s desire for non-instrumental information is affected by factors including expected value, probability of outcome, and a unique formulation of outcome uncertainty. Over two experiments, we find that people’s preference for non-instrumental information is positively correlated with probability when the outcome is positive (i.e., winning money) and negatively correlated when the outcome is negative (i.e., losing money). Furthermore, at the aggregate level, we find the probability of an outcome to be a better predictor of information preference than the expected value of the event or its outcome uncertainty.

The effect of uncertainty and reward probability on information seeking behaviour

People’s causal judgments exhibit substantial variability, but the processes that lead to this variability are not currently understood. In this paper, we use a repeated-measures design to study the within-participant variability of conditional probability judgments in common-cause networks. We establish that these judgments indeed exhibit substantial within-participant variability. This variability differs by inference type and is related to the extent to which participants commit Markov violations. The consistency and systematicity of this variability suggest that it may be an important source of evidence for the cognitive processes that lead to causal judgments. The systematic study of both within- and between-person variability broadens the scope of behavior that can be studied in causal cognition and promotes the evaluation of formal models of the underlying process. The data and methods provided in this paper provide tools to enable the further study of within-participant variability in causal judgment.

Variability in causal judgments

Tversky and Kahneman (1973) found that, under time pressure, people massively underestimated the expansion of 8! (correct value 40,320), and this bias was mitigated for participants presented the descending (8x7x6x5x4x3x2x1; Median=2,250) vs. ascending order (1x2x3x4x5x6x7x8; Median=512). In a first-ever replication (N=140), we also found predominant underestimation, but no significant between-subjects descending vs. ascending order effect. However, when participants then estimated the opposite order, we reproduced this order effect within-subjects. Finally, participants received calibration information (the correct value of 6! or 10!) and again estimated both orders of 8!. Participants who received 10! made more accurate estimates for 8! (Median=38,000), which did not differ statistically from the correct value. Participants who received 6! still grossly underestimated (Median=2,678.5), despite 8! being closer to 6! than 10! in linear and log units. Thus, we surprisingly found the classic factorial estimation bias only within-subjects, and provide evidence for how calibration can reduce it.

Calibration information reduces bias during estimation of factorials: A (partial) replication and extension of Tversky and Kahneman (1973)

Humans rely on multimodal information to make judgements about events occurring in their environment. Haptic feedback, in particular, is essential to how people learn about and manipulate the objects they use daily. While much work has investigated how visual and auditory information affect the perception of causal events, little has explored how causal judgments change with the addition of haptic feedback. To begin addressing this question, we ran a psychophysical study based on the Michottean launching paradigm. We compared the use of visual and haptic with solely visual information during causally ambiguous collisions. We manipulated the offset between when the first object stops moving and the second object starts moving. Using a custom one degree-of-freedom haptic device, users in the vision and haptic condition received kinesthetic haptic feedback synchronized to the second object’s motion. The results demonstrate that adding haptic information increases causal perception for events with larger temporal offsets.

In Touch with Causation: Understanding the Impact of Kinesthetic Haptics on Causality

Theory of Mind (ToM) or mentalizing is the ability to infer mental states of oneself and other agents. Theory of mind plays a key role in social interactions as it allows one to predict other agents' likely future actions by inferring what they may intend or know. 
However, there is a wide range of ToM skills of increasing complexity. While most people are generally capable of performing complex ToM reasoning such as recursive belief inference when explicitly prompted, there is much evidence that humans do not always use ToM to their full capabilities. Instead, people often fall back to heuristics and biases, such as an egocentric bias that projects one's beliefs and perspective onto the observed agent. We explore which (internal or external) factors may influence the mentalizing processes that humans employ unsolicitedly, i.e., employ without being primed or explicitly triggered. In this paper we present an online study investigating unbalanced decision problems where one choice is significantly better than the other. Our results demonstrate that participant's are significantly less likely to exhibit an egocentric bias in such situations.

Less Egocentric Biases in Theory of Mind When Observing Agents in Unbalanced Decision Problems

Alzheimer’s disease leads to a decline in both episodic and semantic memory. Free recall tasks are commonly used in assessments designed to diagnose and monitor cognitive impairment, but tend to focus only on episodic memory. Our goal is to understand the influence of semantic memory on the sequence of free recall in a clinical data set. We develop a cognitive process model that incorporates the influence of semantic similarity and other stimulus properties on the order of free recall. The model also incorporates a decision process based on the Luce choice rule, allowing for different levels of response determinism. We apply the model to a real-world data set including free recall data from 2392 Alzheimer’s patients and their caregivers. We find that semantic similarity between items strongly influences the order of free recall, regardless of impairment. We also observe a trend for response determinism to decrease as impairment increases.

A Model-Based Analysis of Changes in the Semantic Structure of Free Recall Due to Cognitive Impairment

The relational match-to-sample (RMTS) task is used to gauge sensitivity and preference for relational content in the presence of compelling object-based alternatives. On each trial, participants see a triad of 3-element shape sequences: target item (YXY), object match (YVO), and relational match (TWT). Traditionally, human adults show a moderate relational preference — supporting the structural alignment account of similarity-based processing. In the present study, we replicate preliminary findings showing that relational responding is facilitated relative to baseline if the target item is initially presented in isolation with a query to generate a short written description. However, we also observed that initial isolated presentation of the target with no accompanying task led to a similarly elevated rate of relational responding. These findings suggest that a minimal manipulation of presenting the target by itself prior to revealing the full RMTS triad promotes relational responding. We discuss implications for underlying mechanisms driving RMTS performance.

Promoting Relational Responding: The Role of Prior Exposure to the Sample

Previous studies suggest that environment affect human spatial navigation ability. It remains in question when and how the environment assert influence on the developmental changes of spatial cognition. To answer this question, current study compared the general navigation ability and the specific ability in using allocentric vs. egocentric cues between urban and rural children in China. Our results showed urban and rural children did not differ in their general navigation ability when they can use various cognitive strategies in a free navigation task. However, the 9 to12-yrs old rural children performed significantly better than the age-matched urban children when they were instructed to use allocentric cues in a similar maze navigation task. No such difference was found between the 5 to 8-yrs old rural and urban children, suggesting that living environment plays an important role in the development of the allocentric processing ability for navigation during the middle childhood.

Impact of Living Environment on the Development of Cognitive Navigation Strategy in Chinese Urban and Rural Children

Iterated models of pragmatic reasoning, such as the Rational Speech Act model (RSA; Frank & Goodman, 2012), aim to explain how meaning is understood in context. We propose an optimal experiment design approach for teasing apart such models, in which contexts are optimized for differentiating model predictions in reference games. We use this approach to compare RSA with RD-RSA (Zaslavsky et al., 2020), a recent variant of RSA grounded in Rate-Distortion theory. First, we show that our optimal experiment design approach finds cases in which the two models yield qualitatively different predictions, in contrast to previous experimental settings for which these models generate similar predictions. Next, we test the models on newly collected experimental data using our optimal design. Our results show that in this experimental setting RD-RSA robustly outperforms the standard RSA model. This finding supports the idea that Rate-Distortion theory may characterize human pragmatic reasoning.

SpeakEasy Pronunciation Trainer: Personalized Multimodal Pronunciation Training

Downloads

Next from CogSci 2021

Action Speaks Louder than Words and Gaze: The Relative Importance of Modalities in Deictic Reference

Similar lecture

Distributional learning of recursive structures