United States

Because of the compositionality of natural language, syntactic structure which contains the information about the relationship between words is a key factor for semantic understanding. However, the widely adopted Transformer is hard to learn the syntactic structure effectively in dialogue generation tasks. To explicitly model the compositionaity of language in Transformer Block, we restrict the information flow between words by constructing directed dependency graph and propose Dependency Relation Attention (DRA). Experimental results demonstrate that DRA can further improve the performance of state-of-the-art models for dialogue generation.

NAACL 2022

Modeling Compositionality with Dependency Graph for Dialogue Generation

dependency parse

conversation ai

natural language generation

### **Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!**

<iframe width=1280 height=720 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/player/crQbitV66dw?width=1280&height=720&ff=1&title=0" allowfullscreen="true"></iframe>

Computational creativity is an interdisciplinary endeavor that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, social sciences and the arts. The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:

* to construct a program or computer capable of showing some level of creativity
* to better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans
* to design programs that can enhance human creativity without necessarily being creative themselves

 The ICCC conference series organized by the [**Association for Computational Creativity**](https://computationalcreativity.net/) since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.

To access the Twelfth International Conference on Computational Creativity, ICCC’21, you need to register for the conference.  Instructions on how to access the event will be provided directly to attendees and included on the Twelfth International Conference on Computational Creativity, ICCC’21 conference website. 

[Register now](https://computationalcreativity.net/iccc21/registration/)

Register now

ICCC'21

Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

In this study, we developed the first baseline readability model for the Cebuano language. Cebuano is the second most-used native language in the Philippines with about 27.5 million speakers. As the baseline, we extracted traditional or surface-based features, syllable patterns based from Cebuano's documented orthography, and neural embeddings from the multilingual BERT model. Results show that the use of the first two handcrafted linguistic features obtained the best performance trained on an optimized Random Forest model with approximately 87% across all metrics. The feature sets and algorithm used also is similar to previous results in readability assessment for the Filipino language showing potential of crosslingual application. To encourage more work for readability assessment in Philippine languages such as Cebuano, we open-sourced both code and data.

LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification

Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaustive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations from a knowledge base schema. Most existing works are pipelines prone to error accumulation, and all approaches are only applicable to unrealistically small numbers of entities and relations. We introduce GenIE (generative information extraction), the first end-to-end autoregressive formulation of closed information extraction. GenIE naturally exploits the language knowledge from the pre-trained transformer by autoregressively generating relations and entities in textual form. Thanks to a new bi-level constrained generation strategy, only triplets consistent with the predefined knowledge base schema are produced. Our experiments show that GenIE is state-of-the-art on closed information extraction, generalizes from fewer training data points than baselines, and scales to a previously unmanageable number of entities and relations. With this work, closed information extraction becomes practical in realistic scenarios, providing new opportunities for downstream tasks. Finally, this work paves the way towards a unified end-to-end approach to the core tasks of information extraction.

GenIE: Generative Information Extraction

Content-based collaborative filtering (CCF) predicts user-item interactions based on both users' interaction history and items' content information. Recently, pre-trained language models (PLM) have been used to extract high-quality item encodings for CCF. However, it is resource-intensive to train a PLM-based CCF model in an end-to-end (E2E) manner, since optimization involves back-propagating through every content encoding within a given user interaction sequence. To tackle this issue, we propose GRAM (GRadient Accumulation for Multi-modality in CCF), which exploits the fact that a given item often appears multiple times within a batch of interaction histories. Specifically, Single-step GRAM aggregates each item encoding's gradients for back-propagation, with theoretic equivalence to the standard E2E training. As an extension of Single-step GRAM, we propose Multi-step GRAM, which increases the gradient update latency, achieving a further speedup with drastically less GPU memory. GRAM significantly improves training efficiency (up to 146x) on five datasets from two task domains of Knowledge Tracing and News Recommendation. Our code is available at https://github.com/yoonseok312/GRAM.

GRAM: Fast Fine-tuning of Pre-trained Language Models for Content-based Collaborative Filtering

Presentation on the paper "Multi-sense Language Modelling": recap, observations and findings

Hierarchical Relation-Guided Type-Sentence Alignment for Long-Tail Relation Extraction with Distant Supervision

To tailor a learning system to the student's level and needs, we must consider the characteristics of the learning content, such as its difficulty. While natural language processing allows us to represent text efficiently, the meaningful representation of mathematical formulas in an educational context is still understudied. This paper adopts structural embeddings as a possible way to bridge this gap. Our experiments validate the approach using publicly available datasets to show that incorporating syntactic information can improve performance in predicting the exercise difficulty.

SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation

Intent classification (IC) and slot filling (SF) are two fundamental tasks in modern Natural Language Understanding (NLU) systems. Collecting and annotating large amounts of data to train deep learning models for such systems are not scalable. This problem can be addressed by learning from few examples using fast supervised meta-learning techniques such as prototypical networks. In this work, we systematically investigate how contrastive learning and data augmentation methods can benefit these existing meta-learning pipelines for jointly modelled IC/SF tasks. Through extensive experiments across standard IC/SF benchmarks (SNIPS and ATIS), we show that our proposed approaches outperform standard meta-learning methods: contrastive losses as a regularizer in conjunction with prototypical networks consistently outperform the existing state-of-the-art for both IC and SF tasks, while data augmentation strategies primarily improve few-shot IC by a significant margin

Strategies to Improve Few-shot Learning for Intent Classification and Slot Filling

Reasoning over natural language is a long-standing goal for the research community. However, studies have shown that existing language models are inadequate in reasoning. To address the issue, we present POET, a new pre-training paradigm. Through pre-training language models with programs and their execution results, POET empowers language models to harvest the reasoning knowledge possessed in program executors via a data-driven approach. POET is conceptually simple and can be instantiated by different kinds of programs. In this paper, we show three empirically powerful instances, i.e., POET-Math, POET-Logic, and POET-SQL. Experimental results on six benchmarks demonstrate that POET can significantly boost model performance on natural language reasoning, such as numerical reasoning, logical reasoning, and multi-hop reasoning. Taking the DROP benchmark as a representative example, POET improves the F1 metric of BART from 69.2% to 80.6%. Furthermore, POET shines in giant language models, pushing the F1 metric of T5-11B to 87.6% and achieving a new state-of-the-art performance on DROP. POET opens a new gate on reasoning-enhancement pre-training and we hope our analysis would shed light on the future research of reasoning like program executors.

Reasoning Like Program Executors

"As the generation capability of language models improves, a critical emerging need is to have humans in the loop during story infilling, where a language model assists humans in writing creative content.
In our work, we explore the setting in story generation called outline-driven story infilling.  That is, given an incomplete story and a high-level outline, we want the system to complete the story writing as grounded in the outline. We experiment with several variants of language models for this task, including encoder-decoder and decoder-only pre-trained language models. 
We leverage five existing datasets from commonsense reasoning, fictional plot generation, and short story generation. Our results of extensive experiments in automated evaluation metrics, human evaluation, and case analysis reveal that our proposed approaches can successfully leverage outlines for improving generation quality compared to other representative story generation models."

Outline-driven Story Infilling

The increasing adoption of message-based behavioral therapy enables new approaches to assessing mental health using linguistic analysis of patient-generated text. Word counting approaches have demonstrated utility for linguistic feature extraction, but deep learning methods hold additional promise given recent advances in this area. We evaluated the utility of emotion features extracted using a BERT-based model in comparison to emotions extracted using word counts as predictors of symptom severity in a large set of messages from text-based therapy sessions involving over 6,500 unique patients, accompanied by data from repeatedly administered symptom scale measurements. BERT-based emotion features explained more variance in regression models of symptom severity, and improved predictive modeling of scale-derived diagnostic categories. However, LIWC categories that are not directly related to emotions provided valuable and complementary information for modeling of symptom severity, indicating a role for both approaches in inferring the mental states underlying patient-generated language.

Modeling Compositionality with Dependency Graph for Dialogue Generation

Downloads

Next from NAACL 2022

LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification

Similar lecture

Evaluating Natural Language Descriptions Generated in a Workspace-Based Architecture

Modeling Compositionality with Dependency Graph for Dialogue Generation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification

Similar lecture

Evaluating Natural Language Descriptions Generated in a Workspace-Based Architecture

Downloads