Netherlands

Extensive instruction tuning of large language models(LLMs)
has proven to be a powerful technique, extending the
outstanding performance of general LLMs to new tasks.
Consequently, the integration of state-of-the-art(SOTA)
general open-source models with specific domains leverages
instruction tuning to unlock the emerging capabilities of large
language models(LLMs) in those domains. Current practices
in instruction tuning often rely on expanding the size of the
dataset without a clear strategy to ensure data quality. This can
inadvertently introduce noise and degrade model performance.
Furthermore, there is currently no unified approach for the
quantity, quality, and diversity of instruction tuning data, as
well as the methods for instruction tuning. As a result,
this severely hampers the practicality and universality of
instruction tuning.
Addressing these issues, we propose a UNIfied Framework
for Instruction Tuning(UNIFIT), namely Concept Tree
generation for instruction tuning data, instruction following
difficulty selecting data for high-quality instruction tuning, and
the incorporation of random noise embeddings(NE) to enhance
model performance during tuning. Through experiments
involving multiple models, domains, and orders of magnitude,
our proposed instruction tuning framework not only enhances
the diversity of instruction tuning data but also achieves
a remarkable 60% reduction in training time consumption,
with a mere 6% of all instruction tuning data, surpassing
the performance of using all instruction tuning data by
11%. This universally applicable instruction tuning framework
signifies a substantial advancement in the generality of large
language model instruction tuning, marking a revolutionary
leap forward and promising efficiency gains while being
resource-conscious.

**Authors:**

Qiang Huang: XinJiang university; Feng Huang: Xinjiang university; DeHao Tao: Tsinghua University; BingKun Wang: Zhengzhou College of Finance and Economics; YongFeng Huang: TsingHua University

CogSci 2024

UNIFIT: A Unified Framework For Instruction Tuning To Improve Instruction Following Ability For Large Language Models

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

poster

The 46th Annual Meeting of the Cognitive Science Society was an in-person meeting held in Rotterdam, The Netherlands at the Postillion Hotel & Conference Centre.

**ON-DEMAND PROGRAM ACCESS AFTER THE CONFERENCE** 
Recordings of the invited program are now available. You can access them by clicking on the 'Schedule' icon on the left. Select view by week and navigate to the day and time slot of the recording you wish to view. Click on the time slot to access the recording.

Recordings available (click on each recording to view):

**Thursday, July 25** 
0900: [Keynote Speaker Morgan Barense on Enhancing real-world event memory](https://underline.io/events/465/sessions/17997/lecture/99084-dynamics-between-minds-and-the-environment) 
1000: [Gleitman Award Winner Isabelle Dautriche on Language Foundations: Insights from acquisition, communication, cognition, and more](https://underline.io/events/465/sessions/18000/lecture/99085-gleitman-talk) 
1415: [Invited Symposium: Dynamics between minds and the environment](https://underline.io/events/465/sessions?eventSessionId=18016&searchGroup=lecture) 
1700: [Rumelhart Prize Presentation Speaker Alison Gopnik on Exploit, explore, empower: Three ages and three intelligences](https://underline.io/events/465/sessions/18028/lecture/99161-rp1-rumelhart-prize-presentation) 

**Friday, July 26** 
0900: [C.L. de Carvalho-Heineken Prize Keynote Speaker Kia Nobre on Focusing in memory](https://underline.io/events/465/sessions/18035/lecture/99162-hp1-c-l-de-carvalho-heineken-prize-keynote-address) 
1030: [Rumelhart Symposium: Childhood as exploration](https://underline.io/events/465/sessions/18038/lecture/100346-childhood-as-exploration) 
1415: [Elman Prize Symposium](https://underline.io/events/465/sessions?eventSessionId=18053&searchGroup=lecture) 
1600: [Invited Symposium: Dynamics between minds](https://underline.io/events/465/sessions?eventSessionId=18065&searchGroup=lecture) 
1745: [Keynote Speaker Andrea E. Martin on Neural dynamics encode the structure and statistics of language](https://underline.io/events/465/sessions/18077/lecture/99271-neural-dynamics-encode-the-structure-and-statistics-of-language) 

**Saturday, July 27** 
0900: [Keynote Speaker Gregor Schöner on How higher cognition emerges from the dynamics of strongly interacting neural populations](https://underline.io/events/465/sessions/18083/lecture/99273-dynamics-within-the-mind) 
1000: [Glushko Talks](https://underline.io/events/465/sessions?eventSessionId=18086&searchGroup=lecture) 
1500: [Invited Symposium: Dynamics within minds](https://underline.io/events/465/sessions?eventSessionId=18100&searchGroup=lecture) 


#### **CogSci 2024 Program Book**
<div style="position:relative;padding-top:0;width:900px;height:500px;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/cble/" seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


**The Program Book can be downloaded [here](https://drive.google.com/file/d/1yRVp1cBqRnAbdYi4GXWV_pZeyWNsRquv/view?usp=sharing).**

#### **Letter from the President**

Welcome to the 2024 Meeting of the Cognitive Science Society in Rotterdam! CogSci 2024 brings together a large community of cognitive scientists who have traveled here from around the world, as well as a smaller group of remote presenters. I would like to extend an especially warm welcome to our first-time CogSci meeting attendees, and hope that 
their experience inspires them to join our future meetings for many years to come.

I want to thank this year’s conference co-chairs, Larissa K Samuelson, Stefan Frank, Mariya Toneva, Allyson Mackey, and Eliot Hazeltine, for putting together an amazing series of invited talks and symposia around the theme of Dynamics of Cognition. The co-chairs also deserve credit for handling a record-breaking number of submissions and producing a truly exciting conference program 

Beyond the invited talks and symposia dedicated to this year’s conference theme, I would like to highlight the presentations and events honoring the 
Rumelhart, Elman, Gleitman and C .L . de Carvalho Heineken prize winners, including the Rumelhart reception on Thursday. As you may know, we are recording all of the invited talks and symposia, and all prize-winners’ talks, and will be making this content available after the conference. 

The Cognitive Science Society relies on the volunteer efforts of our Governing Board members who work on matters related to our membership; conference policies; diversity and inclusion, international, and outreach initiatives; prizes and much more. We welcome interest from Society members who would like to become more involved in what we do. The 
Society is deeply grateful to Rick Dale and Andrea Bender, who serve as editors of the two Society journals, Cognitive Science and Topics in Cognitive Science respectively. We especially acknowledge our editors’ generosity in donating their stipends each year to D&I and outreach efforts. 

This year’s Annual Meeting depends critically on the support of Marischal De Armond and his team at Podium Conference Specialists: Sharon Zwack, Cendrine De Vis, Sarah-Kate Burke, and Rachel Waller worked tirelessly to secure the venue and handled a multitude of logistical and organizational issues, from arranging coffee breaks and receptions to making sure that a huge number of posters fits comfortably into the available space. Finally, and importantly, I would like to acknowledge the Cognitive Science Society Executive Officer, Erica Wojcik, for managing the complex and ever expanding sphere of our Society’s activities.

I hope you enjoy this year’s meeting and the many different opportunities to engage with the vibrant community of cognitive scientists gathered here. I also hope you get a chance to explore the great city of Rotterdam with its many attractions, museums, restaurants and scenic views. If you’d like to talk more about our Society, or simply want to say hello, please stop by!

![](https://assets.underline.io/markdown_image/1/image/3c66d0e21fff30e204df1138ffd15173.jpeg)

**Anna Papafragou** 
President, Cognitive Science 
Society 2023-2024

**CODE OF CONDUCT** 
By attending the CogSci 2024 Conference, you are required to adhere to the society’s [Code of Conduct](https://drive.google.com/file/d/16-6KkptF0Gn3ZYGJDlpqpwTdImw45Ng0/view?usp=drive_link).

**ABOUT THE COGNITIVE SCIENCE SOCIETY** 
The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education.

The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

You need to log in with the email address you registered with. Access credentials have been sent to your email. 

Please be sure to check your spam and other email folders if you do not see an email confirmation right away.

Please log in to explore this event.

It looks like you are not registered for this event. 

To access the site please register [**here**](https://cognitivesciencesociety.org/registration/). 

Please register!

The 46th Annual Meeting of the Cognitive Science Society presents the latest research across cognitive science and highlights the theme of Cognition in Context.

Despite the fact that beliefs are mental states that cannot be directly observed, humans talk about each others' beliefs on a regular basis, often using rich compositional language to describe what others think and know. What explains this capacity to interpret the hidden epistemic content of other minds? In this paper, we take a step towards an answer by grounding the semantics of belief statements in a Bayesian theory-of-mind: By modeling how humans jointly infer coherent sets of goals, beliefs, and plans that explain an agent's actions, then evaluating statements about the agent's beliefs against these inferences via epistemic logic, our framework provides a conceptual role semantics for belief, explaining the gradedness and compositionality of human belief attributions, as well as their intimate connection with goals and plans. We evaluate this framework by studying how humans attribute goals and beliefs while watching an agent solve a doors-and-keys gridworld puzzle that requires instrumental reasoning about hidden objects. In contrast to pure logical deduction, non-mentalizing baselines, and mentalizing that ignores the role of instrumental plans, our model provides a much better fit to human goal and belief attributions, demonstrating the importance of theory-of-mind for a semantics of belief.

**Authors:**

Lance Ying: Harvard University; Tan Zhi-Xuan: Massachusetts Institute of Technology ; Lionel Wong: Massachusetts Institute of Technology; Vikash Mansinghka: MIT; Josh Tenenbaum: MIT

Grounding Language about Belief in a Bayesian Theory-of-Mind

Many aspects of human learning have been proposed as a process of constructing mental programs: from acquiring symbolic number representations to intuitive theories about the world. In parallel, there is a long-tradition of using information processing to model human cognition through Rate Distortion Theory (RDT). Yet, it is still poorly understood how to apply RDT when mental representations take the form of programs. In this work, we adapt RDT by proposing a three way trade-off among rate (description length), distortion (error), and computational costs (search budget). We use simulations on a melody task to study the implications of this trade-off, and show that constructing a shared program library across tasks provides global benefits. However, this comes at the cost of sensitivity to curricula, which is also characteristic of human learners. Finally, we use methods from partial information decomposition to generate training curricula that induce more effective libraries and better generalization.

**Authors:**

Hanqi Zhou: University of Tübingen; David G. Nagy: University of Tübingen; Charley M Wu: University of Tübingen

Harmonizing Program Induction with Rate-Distortion Theory

We identify three shortcomings of word vectors as representations of the meaning of words: 1) the dimensions of the vectors are implicit and uninterpretable, 2) the word vectors entangle all the meanings and uses of words, and 3) the vectors are unstructured. We proposed solutions to each of these shortcomings, and we explore the implications.

**Authors:**

Jerry T. Ball: Independent; Stuart Rodgers: Institute for Defense Analyses; Roger Schvaneveldt: Independent Researcher; Alan Ball: Independent Researcher

Creating Meaningful Word Vectors and Examining their use as Representations of Word Meaning

This study explores whether Large Language Models (LLMs) can mimic human cognitive processes, particularly pragmatic reasoning in language processing. Focusing on how humans tend to offer semantically similar alternatives in response to negated statements, the research examines if GPT models, both base and fine-tuned, exhibit this behavior. The experiment involves a cloze task, where the models provide completions to negative sentences. Findings reveal that chat models closely resemble human behavior, while completion models do not align with human responses. This indicates that mere linguistic input statistics might be inadequate for LLMs to develop behaviours consistent with pragmatic reasoning. Instead, conversational fine-tuning appears to enable these models to adopt behaviors akin to human pragmatic reasoning. This research not only sheds light on LLMs' capabilities but also prompts further inquiry into language acquisition, especially the role of conversational interactions in developing pragmatic reasoning.

**Authors:**

Francesca Capuano: University of Tübingen; Barbara Kaup: University of Tübingen

Pragmatic Reasoning in GPT Models: Replication of a Subtle Negation Effect

Many recent works in the field of multi-agent reinforcement learning via communication focus on learning what messages to send, when to send and whom to address such messages. Those works indicate that communication is useful for higher cumulative reward or task success. However, one important limitation is that most of them ignore the importance of enforcing agents' ability to understand the received information. In this paper, we notice that observation and communication signals are from separate information sources. Thus, we enhance the communicating agents with the capability to integrate crucial information from different sources. Specifically, we propose a multi-modal communication method, which modulates agents' observation and communication signals as different modalities and performs multi-modal fusion to allow knowledge transferring across different modalities. We evaluate the proposed method on a diverse set of cooperative multi-agent tasks with several state-of-the art algorithms. Results demonstrate the effectiveness of our method in incorporating knowledge and gaining deeper understanding from various information sources.

**Authors:**

Han Wang: Xidian University; Yufeng Xie: Xidian University; BingCheng He: Xidian University; Prof. Qingshan Li: Xidian University

Multi-Agent Communication With Multi-Modal Information Fusion

Part-whole relations and their representation play a vital role in perceptual organization and conceptual reasoning. It is critical for humans to parse visual scenes into objects and parts, and organize them into hierarchies. Few studies have examined how well neural networks learn part-whole hierarchies from visual inputs. In this paper, we introduce a new diagnostic dataset, CChar, to facilitate their understanding. It contains frame-based images of writing 6,840 Chinese characters and annotations on hierarchical structures. The results show that RNN and transformer models could recognize a part of high-level components above strokes and illustrate a certain ability in learning part-whole hierarchies. However, these models do not have robust compositional reasoning. To identify the role of conceptual guidance in predicting hierarchical structures, we prepare visual features extracted by self-supervised and fine-tuned models, test them on generating hierarchical sequences, and observe that conceptual guidance is important to learn part-whole hierarchies. In addition, we also explore the relationship between the depth of hierarchies and model performance. It is found that RNNs perform worse as the hierarchies deepen, but the performance of transformers becomes better with increasing depth.

**Authors:**

Meng Li: University of Potsdam; David Schlangen: University of Potsdam; Dietrich Klakow: Saarland University

Learning Part-whole Hierarchies from the Sequence of Handwriting

In this work, we propose a neural network model for free recall tasks that draws direct parallels between neural machine translation (NMT) models and a cognitive model of human memory search, specifically the Context Maintenance and Retrieval model (CMR). Despite originating in separate domains, these models share strikingly similar architectural features. We hypothesize that NMT advancements such as attention mechanisms (Luong et al., 2015) closely resemble how humans are able to reactivate prior experienced contexts (or “jump back in time”; Tulving, 1985). To demonstrate these parallels, we implement a basic encoder-decoder model with attention mechanism as a cognitive model of human memory search, and train and evaluate its performance against available human free recall data. We find that at intermediate levels of training, the model can capture several phenomena observed in human free recall experiments (Kahana et al., 2022); and upon completion of training, the model demonstrates the same optimal behavior as previously derived by the CMR model (Zhang, Griffiths, & Norman, 2023). When the attention mechanism is removed in an ablation study, the behavioral difference between the model with and without attention aligns with behavior previously observed in hippocampal amnesia patients and healthy controls (Palombo, Di Lascio, Howard, & Verfaellie, 2019). Our interdisciplinary approach not only sheds light on the mechanisms underlying NMT but also provides a novel modeling framework for understanding human memory search.

**Authors:**

Nikolaus Salvatore: Rutgers University; Qiong Zhang: Rutgers University - New Brunswick

Parallels between Neural Machine Translation and Human Memory Search: A Cognitive Modeling Approach

We investigated the human capacity to acquire multiple visuomotor mappings for de novo skills. Using a grid navigation paradigm, we tested whether contextual cues implemented as different "grid worlds", allow participants to learn two distinct key-mappings more efficiently. Our results indicate that when contextual information is provided, task performance is significantly better. The same held true for meta-reinforcement learning agents that differed in whether or not they receive contextual information when performing the task. We evaluated their accuracy in predicting human performance in the task and analyzed their internal representations. The results indicate that contextual cues allow the formation of separate representations in space and time when using different visuomotor mappings, whereas the absence of them favors sharing one representation. While both strategies can allow learning of multiple visuomotor mappings, we showed contextual cues provide a computational advantage in terms of how many contexts can be learned.

**Authors:**

Carlos Alan Velazquez-Vargas: Princeton University; Isaac Christian: Princeton University; Jordan Taylor: Princeton University; Sreejan Kumar: Princeton Neuroscience Institute

Learning to Abstract Visuomotor Mappings using Meta-Reinforcement Learning

Prior literature shows that some mechanisms, e.g., commitment, could give rise to cooperation. However, participants' diverse propensities to cooperate may limit such mechanisms' effectiveness. Thus, we bring individual differences in their propensities to cooperate into the reasoning of long-term social dynamics of cooperation through an agent-based modeling approach. Our results suggest that commitment may still guarantee cooperation when individuals have different propensities to cooperate but has weaker effects, and the setups of commitment are also important. Our study highlights the importance of integrating individual preferences in analyzing collective dynamics of a population consisting of individuals of heterogeneous characteristics, thus offering implications to facilitate cooperation in rich real-world scenarios.

**Authors:**

Wei Wang: Beijing University of Posts and Telecommunications; Luzhan Yuan: Beijing University of Posts and Telecommunications; Zheng Jiang: Beijing University of Posts and Telecommunications; Gaowei Zhang: Beijing University of Posts and Telecommunications; Yi Wang: Beijing University of Posts and Telecommunications

The Dynamics of Cooperation with Commitment in A Population of Heterogeneous Preferences--An ABM Study

Termed the 'Made-by-Human Hypothesis,' I challenge the commonly accepted notion that Artificial Intelligence (AI) is exclusively crafted by humans, emphasizing its impediment to progress. I argue that influences beyond human agency significantly shape AI's trajectory. I then introduce the 'Hybrid Hypothesis,' suggesting that the creation of AI is multi-sourced; methods like evolutionary algorithms influencing AI originate from diverse sources and manifest varying impacts. I contend that the genesis of AI models will increasingly involve a 'Human+' hybrid composition, where the expertise of human developers intertwines with AI's intrinsic mechanisms and may be influenced by non-human sources. The Hybrid Hypothesis extends AI's origin beyond human influence and initiates a profound exploration of unresolved epistemological issues.

**Authors:**

Renee Ye: Ruhr-Universität Bochum

Premium content

Downloads

Next from CogSci 2024

Grounding Language about Belief in a Bayesian Theory-of-Mind

Similar lecture

Automatic Evaluation of Generative Models with Instruction Tuning