CogSci 2025

•

July 31, 2025

•

San Francisco, United States

keywords:

behavioral science

group behaviour

interactive behavior

computational modeling

mathematical modeling

computer science

decision making

human-computer interaction

psychology

artificial intelligence

In human-AI decision-making, understanding the factors that maximize overall accuracy remains a critical challenge. This study highlights the role of metacognitive sensitivity—the agent's ability to assign confidence scores that reliably distinguish between correct and incorrect predictions. We propose a theoretical framework to evaluate the impact of accuracy and metacognitive sensitivity in hybrid decision-making contexts. Our analytical results establish conditions under which an agent with lower accuracy but higher metacognitive sensitivity can enhance overall decision accuracy when paired with another agent. Empirical analyses on a real-world image classification dataset confirm that stronger metacognitive sensitivity—whether in AI or human agents—can improve joint decision outcomes. These findings advocate for a more comprehensive approach to evaluating AI and human collaborators, emphasizing the joint optimization of accuracy and metacognitive sensitivity for enhanced decision-making.

Downloads

Paper

Next from CogSci 2025

Mental Sampling in Social Judgment: Examining Variability in Judgments for the Self, Close, and Distant Others
poster

Mental Sampling in Social Judgment: Examining Variability in Judgments for the Self, Close, and Distant Others

CogSci 2025

Derek Powell
Tingyue Li and 1 other author

31 July 2025

Similar lecture

Explaining Algorithm Aversion with Metacognitive Bandits
poster

Explaining Algorithm Aversion with Metacognitive Bandits

CogSci 2021

Aakriti Kumar
+1
Aakriti Kumar and 3 other authors

27 July 2021