United States

As a prime step in the empirical research cycle, we rely on language to define the constituted concepts. In the plethora of scholarly output, we often find a wide range of discrepant definitions of a given concept manifested by varied linguistic expressions. Capturing the linguistic variability of de facto concept definitions helps researchers assess how convergent our understanding of a concept has become and how falsifiable the associated theory is. To estimate the falsifiability of operational definitions, we proposed and validated proxy measures using natural language processing (NLP) techniques. As a use case study, we quantified the variability of operational definitions of 216 subordinate concepts associated with &quot;open scholarship&quot;. We aggregated 2212 operational definitions from online thesauruses and scientific texts and explored the validity of the proposed proxies of definitions&#39; informativeness. Our study brings new perspectives on reappraising the role of language in constructing falsifiable theories.

CogSci 2023

In search of proxy measures of falsifiability of operational definitions: A cognitive linguistic perspective

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

As a prime step in the empirical research cycle, we rely on language to define the constituted concepts. In the plethora of scholarly output, we often find a wide range of discrepant definitions of a given concept manifested by varied linguistic expressions. Capturing the linguistic variability of de facto concept definitions helps researchers assess how convergent our understanding of a concept has become and how falsifiable the associated theory is. To estimate the falsifiability of operational definitions, we proposed and validated proxy measures using natural language processing (NLP) techniques. As a use case study, we quantified the variability of operational definitions of 216 subordinate concepts associated with "open scholarship". We aggregated 2212 operational definitions from online thesauruses and scientific texts and explored the validity of the proposed proxies of definitions' informativeness. Our study brings new perspectives on reappraising the role of language in constructing falsifiable theories.

poster

The 45th Annual Meeting of the Cognitive Science Society will take place in Sydney, Australia as a modified hybrid conference. Presentations from the main-session room will be live-streamed every day for all events. This will include all invited talks and award sessions. Recordings of all CogSci 2023 sessions will be made available for on-demand viewing after the conference. 
The in-person component will be held at the International Convention Centre Sydney at Darling Harbour. The Conference will take place Wednesday July 26th – Saturday July 29th, 2023 (Wednesday, July 26th is the Workshops Day).

Join us!

 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2023/07/COgSci-PAAG-V2.pdf)
 [**The Detailed Program can be found here**](https://online.fliphtml5.com/ebtyf/mswz/)
 

#### **CogSci 2023 Program Book**
<iframe style="width:700px;height:400px" src="https://online.fliphtml5.com/ebtyf/tzgk/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe>

The 45th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

The existence of a domain-general ‘cognitive map’ system that can be used flexibly to represent physical spaces and conceptual spaces has been suggested in recent work. Activation in brain regions commonly associated with spatial navigation, such as the hippocampus and entorhinal cortex - a critical hub for grid cells - has been observed during tasks such as representing a social network or abstract conceptual relations. To demonstrate a behavioral parallel between abstract and physical navigation, we developed a novel task in which participants represented relational statements (e.g., Joe is smarter than Steve) in space along two dimensions. Uniformity, defined as how closely the representations adhered to a grid cell-like pattern, correlated with self-reported sense of direction and spatial habits of mind. This supports extant research proposing a flexible cognitive map system used for representing both physical space and conceptual relations, providing insight about the inherent grid structure utilized in spatial representations.

Uniform Patterns in Spatial Representations of Abstract Information Are Predicted by Sense of Direction and Spatial Habits of Mind

Following the abandonment of the three-year strict zero-COVID strategy in China, the National Health Commission of China announced the renaming of COVID-19 in Chinese from “新冠肺炎” (novel coronavirus pneumonia, NCP) to “新冠感染” (novel coronavirus infection, NCI) in December, 2022, which may help to reduce people’s fear of COVID-19. We investigated whether the name change affected Chinese people’s (N=1256) perception of COVID immediately after the announcement. The results showed that when explicitly asked about the difference between the two names, about 65% of the individuals perceived NCP as a more serious threat to health and more frightening than NCI. However, when the questions were posed implicitly, the effect of renaming interacted with individuals’ direct and indirect COVID experience. This study thus provides insight into the role of language in shaping people's perception, particularly its interaction with people’s bodily and knowledge/social experience.

What’s new in a name? Chinese people’s perception of the renaming of COVID-19

Over the last decade, cognitive science and allied fields have been criticized for being excessively “WEIRD,” i.e., overly reliant on participants from Western, Educated, Industrialized, Rich, and Democratic societies. The lack of diversity among research participants is now widely acknowledged, but it’s a rather superficial problem, symptomatic of other more fundamental ones. This poster outlines what we see as four overlapping problems. Cognitive science is WEIRD not only in terms of who makes up its participant pool, but also in terms of its theoretical commitments (e.g., individualism and universalism), methodological assumptions (e.g., measurement and analysis methods), and institutional structures (e.g., funding and publishing). Merely solving the problem of WEIRD participants by sampling more widely is insufficient to address cognitive science’s more fundamental WEIRD theoretical, methodological and institutional problems. Coming to terms with this is necessary if we wish to make cognitive science relevant for all humanity.

How WEIRD is Cognitive Science?

Recent studies and models have been successful in describing human search in choice-rich environments in terms of different generalization and exploration strategies. However, it remains an open question how and to what extent people adapt these strategies to different environments. Therefore, we used a within-subject manipulation to study human search in two classes of environments, where the reward distributions differed in the strength of spatial correlations. These environments were chosen through simulations designed to maximize the need for deploying different generalization strategies. We describe participant behavior with the parameters of a model using Gaussian Process function learning to generalize and an Upper Confidence Bound to sample from unexplored options. Our results show that while participants adapted their exploration strategies, their generalization remained unchanged across environments. Together, these findings offer important first insights into how people adapt decision-making strategies depending on environmental demands in vast decision spaces.

Environment-sensitive generalization and exploration strategies

There has been a recent surge of interest in how people assign
subjective value to outcomes that are both risky and delayed,
referred to as risky intertemporal gambles. In most studies,
subjective value has been determined through preferences between
binary choices, yet subjective value and preference can
be determined by willingness-to-pay values or other prices
assigned to each prospect. The present study aims to compare
factors that influence the subjective value of risky intertemporal
prospects between a choice task and a pricing
task. Preferences during choice and pricing were assessed
when outcomes were beneficial (gains) or harmful (losses).
Cognitive models of each task quantify both common parameters
that influence subjective value and specific parameters
involved in each response process. Results suggest that differences
extend beyond simple differences in elicitation procedure:
during pricing, people paid greater attention to payoffs
while exhibiting greater self-control and risk aversion compared
to choice.

Modeling the subjective value of monetary outcomes under risk and delay: gains, losses, and choice-price reversals

Inattentional blindness, the failure to detect obvious and 
unexpected events when attention is elsewhere, is consistently 
underestimated. Instead, people overestimate the ability to 
notice unexpected events in hindsight. No studies have yet 
determined whether beliefs about the noticing of unexpected 
events differ depending on whose visual experience is being 
evaluated. In the current experiment, one-hundred and fortyeight participants read four vignettes depicting legally-relevant 
scenarios. Half of participants imagined that they were the 
individual in the scenario, while the other half imagined that 
the individual was someone else. Participants rated the 
likelihood of noticing each unexpected event and their 
confidence in their judgment. Perspective did not affect 
likelihood ratings or confidence in isolation, but the effect of 
perspective on likelihood judgments did depend on the 
scenario. Additionally, there were differences in likelihood 
judgments and confidence based on scenario, with participants 
believing an event would be more likely to be noticed in more 
relatable legal scenarios (e.g., witness and driving) compared 
to highly specialized scenarios (e.g., CCTV and medicine). 
Thus, visual metacognitive judgments do not appear to 
reliability differ based on the individual being evaluated; 
instead, features of the scenario may be more important.

Does visual metacognition differ based on perspective?

While people report a multitude of factors that determine their food choices (Marcone et al., 2020), laboratory studies often focus on taste and health over other attributes, such as price and convenience, which are often considered irrelevant. In two studies, we found that convenience, a previously overlooked attribute, impacts lab-based food choice. Study 1 (N=118) used a thought-listing paradigm, identifying convenience (22.7%) as a frequently-considered thought during food choice, after taste (43.7%) and health (32.8%). Importantly, in study 2 (N = 94) we found that convenience (β=0.06, p<.001) predicted lab-based food choice, even after controlling for taste (β=0.40, p<.001) and health (β=0.08, p<.001). However, revealed food preferences from lab-based choices were not associated with self-reported preferences from questionnaires, suggesting that integrating more real-world considerations is needed to bridge between laboratory and naturalistic food choices.

Real-world convenience shapes laboratory food choices even when irrelevant

Fermi problems are often taught in science and engineering as a technique for difficult numerical estimation in which a question is broken down into sub-questions that people can more confidently estimate. Thus, they may be tools providing insight into how people estimate numbers. However, there are no empirical studies into the effectiveness of Fermi problems. This study presented participants with the same questions as both Fermi problems (with sub-questions) and as non-Fermi problems (no sub-questions) and tested the hypothesis that participants’ estimates would be more accurate when presented with Fermi problems than non-Fermi problems. The same data tested a hypothesis that participants would better fit to Benford’s law when presented with Fermi problems than non-Fermi problems. Neither hypothesis was supported, although the strong fit of peoples estimates to Benford’s law was replicated. This first attempt to empirically examine Fermi problems pointed to ways to more rigorously test these hypotheses.

Fermi problems as tools for understanding number generation and Benford’s law

We propose a method to use anchor-biased numerical estimates to enhance the wisdom of the crowd (i.e., aggregating individual estimates becomes more accurate). We predicted that better wisdom of crowds could be achieved by combining estimates affected by two sufficiently different anchors. This prediction was based on the hypothesis that anchoring effects could be used to collect estimates from diverse perspectives, and better crowd wisdom could be achieved by aggregating anchor-biased estimates. To test this hypothesis, we conducted a behavioral experiment on anchoring effects and computer simulations of the wisdom of crowds. The behavioral experiment revealed that anchor-biased estimates did not always produce larger errors but produced more accurate or less accurate estimates depending on the relationship between the anchor and true values. We also found that even in the absence of anchoring effects, the estimates were biased. Thus, estimates without anchor bias were not always more accurate than anchor-biased estimates. The results of computer simulations revealed that robust and better wisdom of crowds could be achieved using the proposed method. Contrastingly, aggregating estimates that were not biased by anchors did not always lead to better wisdom of crowds.

Effective utilization of anchor-biased estimates for the wisdom of crowds

Existing studies of the serial reproduction paradigm focused on stimuli that were statistically independent of each other. We explored serial reproductions of random walk series and examined whether Bayesian models previously built for independent stimuli could be adapted to autocorrelated stimuli. We found that Bayesian models captured most of the empirical results qualitatively, but could be further improved by incorporating recency effects. Besides, given that the optimal strategy of iterative prediction of random walk was to reproduce the current stimuli, we also compared serial prediction of random walk to serial reproduction. We found that serially reproduced and predicted series both decorrelate as a function of chain position and that the means of the series increase in both tasks, which matched qualitative predictions of the Bayesian models.

Premium content

Downloads

Next from CogSci 2023

Uniform Patterns in Spatial Representations of Abstract Information Are Predicted by Sense of Direction and Spatial Habits of Mind

Similar lecture

Definitions Matter: Guiding GPT for Multi-label Classification