United States

Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language models (LMs)---informed solely by textual input---on a prevalent phenomenon in cognitive science: typicality. Inspired by experiments that involve language processing and show robust typicality effects in humans, we propose two tests for LMs. Our first test targets whether typicality modulates LM probabilities in assigning taxonomic category memberships to items. The second test investigates sensitivities to typicality in LMs&#39; probabilities when extending new information about items to their categories. Both tests show modest---but not completely absent---correspondence between LMs and humans, suggesting that text-based exposure alone is insufficient to acquire typicality knowledge.

CogSci 2021

Do language models learn typicality judgments from text?

### Conference Welcome

Welcome to the third Cultural Evolution Society Conference, CES2021. In the conference, which will be held from June 9-11, 2021, about 180 researchers from East Asia, Oceania, Middle East, Europe, North and South America give presentations on their latest research advancing the evolutionary understanding of human and nonhuman animal culture.

### Schedule and Program

[![](https://assets.underline.io/uploads/markdown_image/1/image/1ef06ffd05f80957b4a5b685f63ee3ba.png)](https://www.chain.hokudai.ac.jp/CES2020/wp-content/uploads/2021/06/fullprogram.pdf)
[![](https://assets.underline.io/uploads/markdown_image/1/image/12285eb22580949635121d15d5b46325.png)](https://www.chain.hokudai.ac.jp/CES2020/wp-content/uploads/2021/06/abstracts.pdf)

**Program-at-glance**

![](https://assets.underline.io/uploads/markdown_image/1/image/c0d0d4c8fcdad49d12631574fd966ae8.png)

### About the conference 
 
**Help-desk:** 
Please first visit [Help-desk](https://underline.io/events/120/help-desk) where you can find the instructions and manuals for exploring the conference site. 

**Plenary sessions:**
Five leading researchers give a live talk. It is like a TV show. The audience can submit a question from the chat window. Chat messages are immediately delivered to speakers and moderators.

**Q&A sessions (parallel):**
Each presenter first gives a 2-minute live lightning talk followed by an 8-minute live discussion with the audience. It uses Zoom.

**Discussion sessions:** 
The audience can approach presenters and discuss their research. The sessions are held in a dedicated room in Gathertown.
 
**Gathertown:**
It is a platform dedicated to live virtual interactions.
Gather Town is available for all participants 24 hours during the conference.
You can meet and talk with anybody at a lounge or at a table in Gathertown.
Further Gather Town instruction and how to use short keys please find in our [Help desk page](https://underline.io/events/120/help-desk). 

**Videos and slides:**

* Plenary sessions and the Q&A sessions will be recorded and uploaded after the sessions.
* All the videos and slides are available for 60 days after the conference but only to registered ones. 

**Text-based chatting with the speakers:** 
You can chat with any speaker in the Q&A and discussion sessions. A chat window is available on each speaker’s page, and you can leave your questions there. 

**Announcement to the other participants:**
If you want to make an announcement to the other participants (e.g., “Hey, let’s meet at the lounge in Gathertown from 9pm CEST!), please use twitter (#CESConf2021).



### Code of conduct

See the full details of [the code of conduct](https://www.chain.hokudai.ac.jp/CES2020/code-of-conduct/).
&nbsp; 
 CES is dedicated to providing a welcoming and harassment-free conference experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age, religion, nationality, or the presence of or care for dependents. 
We do not tolerate harassment of conference participants in any form. Conference participants violating these rules may be sanctioned or expelled from the conference at the discretion of the conference organizers.

&nbsp;
**Harassment can occur in-person or virtually, and includes, but is not limited to:**
* Comments (verbal, text, etc.) that reinforce social structures of domination related to gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age, religion, nationality
* Sexual images in public spaces (physical or virtual)
* Deliberate intimidation, stalking, or following
* Harassing photography or recording (including sharing of presentation materials without consent)
* Sustained disruption of talks or other events (including “Zoombombing”)
* Inappropriate physical contact or unwelcome sexual attention
* Advocating for, or encouraging, any of the above behaviour 

&nbsp;
 **Inclusiveness Committee**:
&nbsp;
 To facilitate implementing the code of conduct and foster a climate of inclusion, we have developed an inclusiveness committee. Below is their contact information:
 **Hiromi Matsumae**:	matsumae.hiromi.g@tokai.ac.jp
 **Patrick Savage**:	psavage@sfc.keio.ac.jp
 **Sarah Mathew**:	Sarah.Mathew@asu.edu
 **Gmail:** inclusivenessCES2021@gmail.com
 See [more details about reporting the harassment.](https://www.chain.hokudai.ac.jp/CES2020/code-of-conduct/)

### Organizing committee

**Honorary Conference Organizer:** [Kenichi Aoki (Meiji University)](https://sites.google.com/site/kenichi1aoki/home) 

**Chair:** [Masanori Takezawa (Hokkaido University)](https://lalalarakko.github.io)

**Scientific Committee:** [Kenichi Aoki (Meiji University)](https://sites.google.com/site/kenichi1aoki/home), [Yasuo Ihara (the University of Tokyo)](https://researchmap.jp/read0209180?lang=en), [Hiromi Matsumae (Tokai University)](https://sites.google.com/site/hiromimatsumae/eng), [Naoko Matsumoto (Okayama University)](https://researchmap.jp/nmatsumoto_okayama-u), [Masako Myowa (Kyoto University)](https://myowa.educ.kyoto-u.ac.jp/en), [Hisashi Nakao (Nanzan University)](https://www.hisashinakao.com/), [Patrick E. Savage (Keio University)](https://sites.google.com/view/comp-music-lab/home)

**Local Organizing Committee:** [Yutaka Horita (Teikyo University)](https://sites.google.com/site/yhoritapage/), [Masanori Takezawa (Hokkaido University)](https://lalalarakko.github.io/), [Ayaka Takimoto (Hokkaido University)](https://lynx.let.hokudai.ac.jp/~takimoto/), [Kohei Tamura (Tohoku University)](https://researchmap.jp/7000017854)

**Inclusiveness Committee:** [Sarah Mathew (Arizona State University)](https://www.sarahmathew.net/), [Hiromi Matsumae (Tokai University)](https://sites.google.com/site/hiromimatsumae/eng), [Patrick E. Savage (Keio University)](https://sites.google.com/view/comp-music-lab/home)


![](https://assets.underline.io/uploads/markdown_image/1/image/967110a00d200a6b20d6f4fa6556ba13.png)

For attending the conference, you first need to register and pay fees from [the link here](https://www.chain.hokudai.ac.jp/CES2020/registration/). After the payment of the registration fees, you will receive a ticket, which is necessary for accessing the conference site. After the payment of the registration fees, you will receive a ticket, which is necessary for accessing the conference site

Registration fee

Cultural Evolution Society 2021

Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language models (LMs)---informed solely by textual input---on a prevalent phenomenon in cognitive science: typicality. Inspired by experiments that involve language processing and show robust typicality effects in humans, we propose two tests for LMs. Our first test targets whether typicality modulates LM probabilities in assigning taxonomic category memberships to items. The second test investigates sensitivities to typicality in LMs' probabilities when extending new information about items to their categories. Both tests show modest---but not completely absent---correspondence between LMs and humans, suggesting that text-based exposure alone is insufficient to acquire typicality knowledge.

technical paper

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year! 

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).

The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

Learning the sounds of a new language depends on the ability to learn novel auditory categories. Multidimensional categories, whether speech or nonspeech, can be learned through feedback and different category structures are proposed to recruit separate cognitive and neural mechanisms. There is substantial individual variability in learning; however, it is rare to compare learning of different categories in the same individuals. Understanding the sources of variability has theoretical implications for category learning. In this study, we trained the same participants on three types of multidimensional auditory categories. Participants learned nonspeech rule-based, nonspeech information-integration, and Mandarin speech categories. Learning of all category types was related across individuals and differences in working memory similarly supported learning across tasks. There was substantial variability in learning outcomes and strategies used to learn the categories. There are multiple paths to successful learning and appreciation of individual variability is essential to understanding the mechanisms of learning.

Individual Variability in Strategies and Learning Outcomes in Auditory Category Learning

Two assumptions of psycholinguistic research are that text corpora can be used as a proxy of the language that people have been exposed to and that the reaction time with which people recognize words decreases with the probability (or frequency) of the words in a corpus. We propose a method that produces topic-specific word probabilities from a text corpus using latent Dirichlet allocation, then combines them to fit lexical decision reaction times and re-estimates word probabilities. We evaluated how well independent lexical decision reaction times could be predicted from re-estimated word probabilities compared to original probabilities, using independent lexical decision data. In an experiment designed to prove the concept, the re-estimated word frequency model explained up to 9.6% of additional variability in reaction times on group level and up to 2.9% on level of individual participants.

Word Probability Re-Estimation Using Topic Modeling and Lexical Decision Data

Many category learning experiments use supervised learning (i.e., trial-by-trial feedback). Most of those procedures use deterministic feedback, teaching participants to classify exemplars into consistent categories (i.e., the stimulus i is always classified in category k). Though some researchers suggest that natural learning conditions are more likely to be inconsistent, the literature using probabilistic feedback in category learning experiments is sparse. Our analysis of the literature suggests that part of the reason for this sparsity is a relative lack of flexibility of current paradigms and procedures for designing probabilistic feedback experiments. The work we report here offers a novel paradigm (the Probabilistic Prototype Distortion task) which allows researchers greater flexibility when creating experiments with different p(category|feature) probabilities, and also allows parametrically manipulating the amount of randomness in an experimental task. In the current work, we offer a detailed procedure, implementation, experimental results and discussion of this novel procedure. Our results suggest that by designing experiments with our procedures, the experimental setup allows subjects to achieve the desired classification performance.

Designing probabilistic category learning experiments: The probabilistic prototype distortion task

Traditional statistics instruction emphasizes a .05 significance level for hypothesis tests. Here, we investigate the consequences of this training for researchers’ mental representations of probabilities –– whether .05 becomes a boundary, i.e., a discontinuity of the mental number line, and alters their perception of differences between p-values. Graduate students (n = 25) with statistical training viewed pairs of p-values and judged whether they were ‘similar’ or ‘different’. After controlling for covariates, participants were more likely and faster to judge p-values as ‘different’ when they crossed the .05 boundary (e.g., .047 vs. .052) compared to when they did not (e.g., .027 vs. .032). This categorical perception effect suggests that traditional statistical instruction creates a psychologically real divide between so-called statistically significant and non-significant p-values. Such a distortion is undesirable given modern approaches to statistical reasoning that de-emphasize dichotomizing p-values.

Categorical perception of p-values

Studies on similarity-based interference in subject-verb number agreement dependencies have found a consistent facilitatory effect in ungrammatical sentences but no conclusive effect in grammatical sentences. Existing models propose that interference is caused either by a faulty representation of the input (encoding-based models) or by difficulty in retrieving the subject based on cues at the verb (retrieval-based models). Neither class of model captures the observed patterns in human reading time data. We propose a new model that integrates a feature encoding mechanism into an existing cue-based retrieval model. Our model outperforms the cue-based retrieval model in explaining interference effect data from both grammatical and ungrammatical sentences. These modeling results yield a new insight into sentence processing, encoding modulates retrieval. Nouns stored in memory undergo feature distortion, which in turn affects how retrieval unfolds during dependency completion.

Feature encoding modulates cue-based retrieval: Modeling interference effects in both grammatical and ungrammatical sentences

Snoddy and Kurtz (2020) demonstrated spontaneous transfer of relational categories to new learning. Recognition memory data suggested that transfer was driven by schematization during learning. In the present study, we explored whether schema abstraction underlies transfer and recognition effects. Participants were assigned to condition based on the type of initial learning: classification with comparison, supervised observation with comparison, single-item supervised observation, or baseline (no learning). After initial learning, participants underwent a study phase and recognition test on novel stimuli followed by a target category learning task involving the same underlying category structures expressed in a new domain. During the recognition test, all conditions led to increased false alarms relative to baseline. Only the comparison conditions exhibited analogical transfer on the target category learning task. Results suggest that comparison facilitates the transfer of relational categories (due to schema abstraction), but recognition memory effects may be driven by more general categorization mechanisms.

Comparison Promotes the Spontaneous Transfer of Relational Categories

How people represent categories—and how those representations change over time—is a basic question about human cognition. Previous research has suggested that people categorize objects by comparing them to category prototypes in early stages of learning but use strategies that consider the individual exemplars within each category in later stages. However, many category learning experiments do not accurately reflect the environmental statistics of the real world, where the probability that we encounter an object changes over time. Our goal in this study was to introduce memory constraints by presenting each stimulus at intervals corresponding to the power-law function of memory decay. Since the exemplar model relies on the individual’s ability to store and retrieve previously seen exemplars, we hypothesized that adding memory constraints that better reflect real environments would favor the exemplar model more early on compared with later. Confirming our hypothesis, the results illustrate that under realistic environmental statistics with memory constraints, the exemplar model’s advantage over the the prototype model decreases over time.

The Dynamics of Exemplar and Prototype Representations Depend on Environmental Statistics

Computational model simulations have been very fruitful for gaining insight into how the systematic structure we observe in the world's natural languages could have emerged through cultural evolution. However, these model simulations operate on a toy scale compared to the size of actual human vocabularies, due to the prohibitive computational resource demands that simulations with larger lexicons would pose. Using computational complexity analysis, we show that this is not an implementational artifact, but instead it reflects a deeper theoretical issue: these models are (in their current formulation) computationally intractable. This has important theoretical implications, because it means that there is no way of knowing whether or not the properties and regularities observed for the toy models would scale up. All is not lost however, because awareness of intractability allows us to face the issue of scaling head-on, and can guide the development of our theories.

Why is scaling up models of language evolution hard?

Co-creating meaning in conversation is challenging. Success is often determined by people’s abilities to coordinate their language in strategic ways to signal problems and to align mental representations. Here we explore one set of grounding mechanisms, known as interactive linguistic alignment, that makes use of the ways people re-use, i.e., “align to,” the lexical, semantic, and syntactic forms of others’ utterances. In particular, the focus is on the temporal development of multi-level linguistic alignment and examine how its expression is related to communicative outcomes within a unique collaborative problem-solving paradigm. The primary task, situated within an educational video game, requires creative thinking between three people where the paths for possible solutions are highly variable. We find that over time interactions are marked by decreasing lexical and syntactic alignment, with a trade-off of increasing semantic alignment. However, a greater semantic alignment does not necessarily translate into better team performance. Overall, these findings provide greater clarity on the role of grounding mechanisms in complex and dynamic collaborative problem-solving tasks.

Multi-Level Linguistic Alignment in a Virtual Collaborative Problem-Solving Task

Legal documents, in the form of terms of service agreements and other private contracts, are now an increasingly prevalent part of everyday life. While legal documents have long been acknowledged to be difficult to understand without training, it remains an open question whether the ever-increasing exposure to contracts might have mitigated this difficulty. Moreover, insofar as this difficulty has persisted, there remains no systematic analysis of which linguistic structures contribute most heavily to the processing difficulty of legal texts, nor whether this difficulty is heightened for those with less language experience. Here, we investigate these issues, and in a well-powered experiment find evidence that (a) both recall and comprehension of legal propositions in a contract are hindered by use of a legal register relative to plain-English translations; (b) certain linguistic structures, such as center-embedding, hinder recall to a greater degree than others, such as passive voice; and (c) language experience influences comprehension of legal propositions. Surprisingly, language experience did not influence recall, nor was there an interaction between legal register and language experience on recall or comprehension. These findings suggest that legal language poses heightened difficulties for those with less language experience--who tend to be of lower socioeconomic status and with diminished access to the justice system--and that eliminating complex features of legalese would benefit those of all reading levels.

Do language models learn typicality judgments from text?

Downloads

Next from CogSci 2021

Individual Variability in Strategies and Learning Outcomes in Auditory Category Learning

Similar lecture

Social Complexity, Affluence, and the Emergence of Moralizing Gods in the Ancient Mediterranean

Do language models learn typicality judgments from text?

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from CogSci 2021

Individual Variability in Strategies and Learning Outcomes in Auditory Category Learning

Similar lecture

Social Complexity, Affluence, and the Emergence of Moralizing Gods in the Ancient Mediterranean

Downloads