Singapore

How do sequence models represent their decision-making process? Prior work suggests that Othello-playing neural network learned nonlinear models of the board state (Li et al., 2023a). In this work, we provide evidence of a closely related linear representation of the board. In particular, we show that probing for “my colour” vs. “opponent’s colour” may be a simple yet powerful way to interpret the model’s internal state. This precise understanding of the internal representations allows us to control the model’s behaviour with simple vector arithmetic. Linear representations enable significant interpretability progress, which we
demonstrate with further exploration of how the world model is computed.

EMNLP 2023

Emergent Linear Representations in World Models of Self-Supervised Sequence Models

mechanistic interpretability

world models

language models

### Welcome!
The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. Our Virtual Poster Sessions will take place online Thursday, August 22, 2024.

You are required to register for this event. **Please register [here](https://2024.aclweb.org/registration). **

If you have already registered, please check your inbox for an email from Underline granting you access to ACL 2024 content.

Please register!

ACL 2024

The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) will take place in Bangkok, Thailand from August 11th to 16th, 2024. More information will be announced soon.

workshop paper

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

Explanations are meaningful only if it is understandable for human users. In daily life, humans explain to each other in natural language. However, eXplainable AI (XAI) identifies, e. g., important attributes from tabular data as explanations to a prediction, which are often difficult for humans to understand, especially when they
have no AI background. To solve the limitation, we aim to generate text-based comprehensive explanations conditioned on the important features selected by an XAI method. We expect the explanations can borrow strength from XAI and Large Language Models (LLMs), integrating the concrete information extracted by XAI from tabular data and the general domain knowledge retained in LLM from large corpus.

Natural Language Explanation for Tabular Data

We train a transformer network on the task of performing additions of two integers, achieving near perfect test-set accuracy with just 8000 positive training examples. We compute both learning curves (accuracy as a function of training set size) and model selection curves (accuracy as a function of embedding dimension and of number of layers). We observe that the number of dimension plays the biggest role, whereas the number of layers does not make a big difference after four layers. While the network generalises well from training to testing set, we did not observe any transfer-learning benefits when learning subtraction, having to relearn it from scratch. Current work involves probing the attention heads to understand the different roles played by positional embedding and the digit embedding (our conjecture is that query and key should be attending to position, while value should be attending to the actual digit). This distinction would not be possible when analysing words.

Learning to Add as a way to Understand Transformers

Examples are a powerful tool that help us understand complex concepts and connections. In computational linguistics research, looking at example system output and example corpus entries can offer a wealth of insights that are not otherwise accessible. This paper describes the open-source software VULCAN, a visualization tool for strings, graphs, trees, alignments, attention and more. 
VULCAN's unique ability to visualize both linguistic structures and properties of neural models make it particularly relevant for neuro-symbolic models. Neuro-symbolic models, combining neural networks with often linguistically grounded structures, offer a promise of increased interpretability in an age of purely neural black-box end-to-end models. VULCAN aims to facilitate this interpretability in practice.
VULCAN is designed to be both easy to use and powerful in its capabilities.

Introducing VULCAN: A Visualization Tool for Understanding our Models and Data by Example

Flexible Model Interpretability through Natural Language Model Editing

Several methodologies have recently been proposed to evaluate the ability of Pretrained Language Models (PLMs) to interpret negation. In this article, we build on Gubelmann and Hand- schuh (2022), which studies the modification of PLMs’ predictions as a function of the polarity of inputs, in English. Crucially, this test uses “self-contained” inputs ending with a masked position: depending on the polarity of a verb in the input, a particular token is either semantically ruled out or allowed at the masked position. By replicating Gubelmann and Handschuh (2022) experiments, we have uncovered flaws that weaken the conclusions that can be drawn from this test. We thus propose an improved version, the Self-Contained Neg Test, which is more controlled, more systematic, and entirely based on examples forming minimal pairs varying only in the presence or absence of verbal negation in English.
When applying our test to the roberta and bert base and large models, we show that only roberta-large shows trends that match the expectations, while bert-base is mostly insensitive to negation. For all the tested models though, in a significant number of test instances the top-1 prediction remains the token that is semantically forbidden by the context, which shows how much room for improvement remains for a proper treatment of the negation phenomenon.

The Self-Contained Negation Test Set

Investigating the Effect of Discourse Connectives on Transformer Surprisal: Language Models Understand Connectives, Even So They Are Surprised

Collostructional analysis is a technique devised to find correlations between particular words and linguistic constructions in order to analyse meaning associations of these constructions. Contrasting collostructional analysis results with output from BERT might provide insights into the way BERT represents the meaning of linguistic constructions. This study tests to what extent English BERT's meaning representations correspond to known constructions from the linguistics literature by means of two tasks that we propose. Firstly, by predicting the words that can be used in open slots of constructions, the meaning associations of more lexicalized constructions can be observed. Secondly, by finding similar sequences using BERT's output embeddings and manually reviewing the resulting sentences, we can observe whether instances of less lexicalized constructions are clustered together in semantic space. These two methods show that BERT represents constructional meaning to a certain extent, but does not separate instances of a construction from a near-synonymous construction that has a different form.

Using Collostructional Analysis to evaluate BERT's representation of linguistic constructions

In this work-in-progress, we investigate the multi-step reasoning mechanisms of
transformer-based language models by adopting a reverse-engineering approach, inspired by work on mechanistic interpretability. To this end, we study a 6-layer transformer trained on a simplified tree traversal task. We find that the model forms backtracking heads which allow the model to backtrack a certain number of steps from the goal node. In addition, we find that the MLPs in the final layers are cru-
cial for finding paths when the goal is further away, implying that the model combines different internal mechanisms to solve even simple algorithmic tasks.

Planning in Transformers: Evidence for Backtracking Circuits in Models Trained on Tree Traversal

Probing contextualized representations typically involves comparing task-specific model predictions against ground truth linguistic labels. Although this methodology shows what information can be recovered by a classifier, it does not reveal how a classifier uses the representation to make its decision. To address the latter problem, we ask: Do task-classifiers rely on representation- and task-independent geometric patterns in the embedding space? We explore this question by developing METAPROBE, an approach that uses geometric properties of representations to predict the behavior of task-specific classifiers (i.e., their predictions as opposed to the ground truth). Our experiments reveal the existence of universal geometric patterns across representations that can predict classifier predictions. Consequently, this allows us to posit a geometric explanation for the impressive performance of contextualized representations.

Downloads

Next from EMNLP 2023

Natural Language Explanation for Tabular Data

Similar lecture

CheckersGPT: Learning World Models through Language Modeling

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EMNLP 2023

Natural Language Explanation for Tabular Data

Similar lecture

CheckersGPT: Learning World Models through Language Modeling

Downloads