Spain

Text-to-SQL systems offers natural language interfaces to databases, which can automatically generates SQL queries given natural language questions. On the WikiSQL benchmark, state-ofthe-art text-to-SQL systems typically take a slot-filling approach by building several specialized models for each type of slot. Despite being effective, such modularized systems are complex and also fall short in jointly learning for different slots. To solve these problems, this paper proposes a novel approach that formulates the task as a question answering problem, where different slots are predicted by a unified machine reading comprehension (MRC) model. For this purpose, we use a BERT-based MRC model, which can also benefit from intermediate training on other MRC datasets. The proposed method can achieve competitive results on WikiSQL, suggesting it being a promising direction for text-to-SQL.

COLING 2020

SQL Generation via Machine Reading Comprehension

text-to-sql

semantic parsing

question answering

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

poster

This paper proposed a supervised visual attention mechanism for multimodal neural machine translation (MNMT), trained with constraints based on manual alignments between words in a sentence and their corresponding regions of an image. 
The proposed visual attention mechanism captures the relationship between a word and an image region more precisely than a conventional visual attention mechanism trained through MNMT in an unsupervised manner. 
Our experiments on English-German and German-English translation tasks using the Multi30k dataset and on English-Japanese and Japanese-English translation tasks using the Flickr30k Entities JP dataset show that a Transformer-based MNMT model can be improved by incorporating our proposed supervised visual attention mechanism and that further improvements can be achieved by combining it with a supervised cross-lingual attention mechanism (up to +1.61 BLEU, +1.7 METEOR).

Supervised Visual Attention for Multimodal Neural Machine Translation

We present a large-scale corpus of e-mail conversations with domain-agnostic and two-level dialogue act (DA) annotations towards the goal of a better understanding of asynchronous conversations. To assess the difficulty of DA recognition on our corpus, we evaluate several models, including a pre-trained contextual representation model, as our baselines. The experimental results show that BERT outperforms other neural network models, including previous state-of-the-art models, but falls short of a human performance. An evaluation of a model trained on our corpus against other domains of asynchronous conversation reveals the domain independence of our DA annotations.

A Large-Scale Corpus of E-mail Conversations with Standard and Two-Level Dialogue Act Annotations

Multi-turn response selection has been extensively studied and applied to many real-world applications in recent years. However, current methods typically model the interactions between multi-turn utterances and candidate responses with iterative approaches, which is not practical as the turns of conversations vary. Besides, some latent features, such as user intent and conversation topic, are under-discovered in existing works. In this work, we propose Intra-/Inter-Interaction Network (I3) with latent interaction modeling to comprehensively model multi-level interactions between the utterance context and the response. In specific, we first encode the intra- and inter-utterance interaction with the given response from both individual utterance and the overall utterance context. Then we develop a latent multi-view subspace clustering module to model the latent interaction between the utterance and response. Experimental results show that the proposed method substantially and consistently outperforms existing state-of-the-art methods on three multi-turn response selection benchmark datasets.

Intra-/Inter-Interaction Network with Latent Interaction Modeling for Multi-turn Response Selection

We wish to explore the problem of discovering conversational intent from past human-human conversation dataset. We assume task-oriented conversational intent where one user is trying to accomplish and the agent is helping to solve that problem. We extract short text representation of a conversation using a novel feature extraction pipeline. By contrast, we formulate the problem of conversational intent mining as a short text clustering. We found that the conversational intents are unbalanced in nature. Since density based clustering can find clusters of arbitrary shapes, we investigate popular algorithms such as DBSCAN, HDBSCAN for unbalanced data clustering problem. We also propose a simple formulation of the density based clustering algorithm (ITER-DBSCAN) and propose a parallel version for large dataset. Our algorithm achieves state-of-the art results on six intent mining, one conversational dataset and one short text clustering dataset. Codes and dataset are published on GitHub for future research.

Intent Mining from past conversations for Conversational Agent

This paper introduces a method that efficiently reduces the computational cost and parameter size of Transformer. The proposed model, refer to as \textit{Group-Transformer}, splits feature space into multiple groups, factorizes the calculation paths, and reduces computations for the group interaction.
Extensive experiments on two benchmark tasks, enwik8 and text8, prove our model's effectiveness and efficiency in small-scale Transformers.
To the best of our knowledge, Group-Transformer is the first attempt to design Transformer with the group strategy, widely used for efficient CNN architectures.

Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model

In this paper, we separate the model into a sentence reconstruction module and a style module. We use the Transformer-based autoencoder model for sentence reconstruction and the adaptive style embedding is learned directly in the style module. Because of this separation, each module can better focus on its own task. Moreover, we can vary the style strength of the generated sentence by changing the style of the embedding expression.

How Positive Are You:Text Style Transfer using Adaptive Style Embedding

We propose a distance-based model aiming at temporal knowledge graph embedding (TKGE). The temporal evolution of an entity embedding is defined as a rotation from the initial time to the current time in the complex vector space. Each relation is represented as a pair of dual complex embeddings to handle the beginning and the end of the relation, respectively. We use different time division methods and time granularities for different datasets. Experimental Results show that our model achieves the state-of-the-art performances on four TKG datasets.


TeRo: A Time-aware Knowledge Graph Embedding Model via Temporal Rotation

Lexical substitution, i.e. generation of plausible words that can replace a particular target word in a given context, is an extremely powerful technology that can be used as a backbone of various NLP applications, including word sense induction and disambiguation, lexical relation extraction, data augmentation, etc. In this paper, we present a large-scale comparative study of lexical substitution methods employing both rather old and most recent language and masked language models (LMs and MLMs), such as context2vec, ELMo, BERT, RoBERTa, XLNet. We show that already competitive results achieved by SOTA LMs/MLMs can be further substantially improved if information about the target word is injected properly. Several existing and new target word injection methods are compared for each LM/MLM using both intrinsic evaluation on lexical substitution datasets and extrinsic evaluation on word sense induction (WSI) datasets. On two WSI datasets we obtain new SOTA results. Besides, we analyze the types of semantic relations between target words and their substitutes generated by different models or given by annotators.

Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution

Common uncertainty estimation approaches remain as black-boxes without explaining which features have led to the uncertainty of a prediction. We introduce an approach to decompose and visualize the uncertainty of text classifiers at the level of words. We aim to enable a deeper inspection and reasoning about unreliable model behaviours by providing detailed explanations of uncertainties.

Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs

This poster introduces the SORTS Subject-Object Resolution Test Suite of German minimal sentence pairs for model introspection. The full test suite consists of 18,502 transitive clauses with manual annotations of 8 word order patterns, 5 morphological and syntactic and 11 semantic property classes. The test suite has been constructed such that sentences are minimal pairs with respect to a property class. Each property has been selected with a particular focus on its effect on subject-object resolution, the second-most error-prone task within syntactic parsing of German after prepositional phrase attachment. The size and detail of annotations make the test suite a valuable resource for natural language processing applications with syntactic and semantic tasks. We use dependency parsing to demonstrate how the test suite allows insights into the process of subject-object resolution. Based on the test suite annotations, word order and case syncretism can be identified as most important factors that affect subject-object resolution.

Downloads

Next from COLING 2020

Supervised Visual Attention for Multimodal Neural Machine Translation

Similar lecture

A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from COLING 2020

Supervised Visual Attention for Multimodal Neural Machine Translation

Similar lecture

A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsing

Downloads