United States

The quality and quantity of parallel sentences are known as very important training data for constructing neural machine translation (NMT) systems. However, these resources are not available for many low-resource language pairs. Many existing methods need strong supervision and hence are not suitable. Although there have been several attempts at developing unsupervised models, they ignore the language-invariant between languages. In this paper, we propose an approach based on transfer learning to mine parallel sentences in an unsupervised setting. With the help of bilingual corpora of rich-resource language pairs, we can mine parallel sentences without bilingual supervision of low-resource language pairs. Experiments show that our approach improves the performance of mined parallel sentences compared with previous methods. In particular, we achieve good results at two real-world low-resource language pairs.

NAACL 2021

Parallel sentences mining with transfer learning in an unsupervised setting

2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

**Whova App** 
Stay in touch with your fellow conference attendees via the [Whova App](https://whova.com/portal/webapp/nacon_202106/)

**Conference Structure**
https://2021.naacl.org/blog/conference-structure/

**Walkthrough video of how to NAACL 2021** 

Please take a moment to view this video explaining how to navigate the platform, attend sessions network with other attendees. 


<figure class="video_container">
  <iframe src="https://screencast-o-matic.com/watch/crhwbGVh3vx?v=6&ff=1&title=0&controls=1" width=640  height=350  frameborder="0" allowfullscreen="true"> </iframe>
</figure>

<br>
<br>

technical paper

"Word sense disambiguation (WSD) is a long-standing problem in natural language processing. One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution. For instance, 84% of the annotated words have less than 10 examples in the SemCor training data. This issue is more pronounced as the imbalance occurs in both word and sense distributions. In this work, we propose MetricWSD, a non-parametric few-shot learning approach to mitigate this data imbalance issue. By learning to compute distances among the senses of a given word through episodic training, MetricWSD transfers knowledge (a learned metric space) from high-frequency words to infrequent ones. MetricWSD constructs the training episodes tailored to word frequencies and explicitly addresses the problem of the skewed distribution, as opposed to mixing all the words trained with parametric models in previous work. Without resorting to any lexical resources, MetricWSD obtains strong performance against parametric alternatives, achieving a 75.1 F1 score on the unified WSD evaluation benchmark (Raganato et al., 2017b). Our analysis further validates that infrequent words and senses enjoy significant improvement.

Non-Parametric Few-Shot Learning for Word Sense Disambiguation

Word representations empowered with additional linguistic information have been widely studied and proved to outperform traditional embeddings. Current methods mainly focus on learning embeddings for words while embeddings of linguistic information (referred to as grain embeddings) are discarded after the learning. This work proposes a framework field embedding to jointly learn both word and grain embeddings by incorporating morphological, phonetic, and syntactical linguistic fields. The framework leverages an innovative fine-grained pipeline that integrates multiple linguistic fields and produces high-quality grain sequences for learning supreme word representations. A novel algorithm is also designed to learn embeddings for words and grains by capturing information that is contained within each field and that is shared across them. Experimental results of lexical tasks and downstream natural language processing tasks illustrate that our framework can learn better word embeddings and grain embeddings. Qualitative evaluations show grain embeddings effectively capture the semantic information.

Field Embedding: A Unified Grain-Based Framework for Word Representation

Automated metaphor detection is a challenging task to identify the metaphorical expression of words in a sentence. To tackle this problem, we adopt pre-trained contextualized models, e.g., BERT and RoBERTa. To this end, we propose a novel metaphor detection model, namely metaphor-aware late interaction over BERT. Our model not only leverages contextualized word representation but also benefits from linguistic metaphor identification theories to detect whether the target word is metaphorical. Our empirical results demonstrate that MelBERT outperforms several strong baselines on four benchmark datasets, i.e., VUA-18, VUA-20, MOH-X, and TroFi.

MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theories

Fine-grained opinion mining (OM) has achieved increasing attraction in the natural language processing (NLP) community, which aims to find the opinion structures of ``Who expressed what opinions towards what'' in one sentence.

A Unified Span-Based Approach for Opinion Mining with Syntactic Constituents

The goal of stance detection is to identify whether the author of a text is in favor of, neutral or against a specific target. Despite substantial progress on this task, one of the remaining challenges is the scarcity of annotations. Data augmentation is commonly used to address annotation scarcity by generating more training samples. However, the augmented sentences that are generated by existing methods are either less diversified or inconsistent with the given target and stance label. In this work, we formulate the data augmentation of stance detection as a conditional masked language modeling task and augment the dataset by predicting the masked word conditioned on both its context and the auxiliary sentence that contains target and label information. Moreover, we propose another simple yet effective method that generates target-aware sentence by replacing a target mention with the other. Experimental results show that our proposed methods significantly outperforms previous augmentation methods on 11 targets.

Target-Aware Data Augmentation for Stance Detection

It's an about 9 minutes presentation of the paper: Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction, which introduces the task description, motivation, methodology and experiment results briefly. The topics are the target-oriented opinion words extraction and aspect-opinion pairs extraction.

Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction

Does syntax matter for Aspect-level Sentiment Classification？ No, the RoBERTa with an MLP layer is able to achieve SOTA or near SOTA performance in this task.

Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa

Domain divergences are a tool to measure domain shifts. Given a plethora of divergence measures, In this talk, we provide a taxonomy of divergences consisting of three classes - 1) Geometric 2) Information Theoretic 3) Higher order measures. We detail their use cases in domain adaptation for 1) Data Selection 2) Learning domain invariant representations 3) Making decisions in the wild. While divergence measures have a practical use-case of predicting performance drops, it is still not clear which divergence measure is suitable for a NLP task. Also, most of the works in the literature calculate divergence measures based on simple frequency based word distributions or word embeddings. Does using contextual word representations have any advantage in predicting the drop in performance? We perform a large-scale empirical analysis consisting of more than 130 domain adaptation scenarios, 3 NLP tasks and 12 divergence measures. We find that divergences calculated using traditional word distributions still serve as strong baselines.

Domain Divergences: A Survey and Empirical Analysis

**Read the paper on the folowing link:**

https://www.aclweb.org/anthology/2021.naacl-srw.14/

**Abstract:**

0

Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation

For a computer to naturally interact with a human, it needs to be human-like. In this paper, we propose a neural response generation model with multi-task learning of generation and classification, focusing on emotion. Our model based on BART (Lewis et al., 2020), a pre-trained transformer encoderdecoder model, is trained to generate responses and recognize emotions simultaneously. Furthermore, we weight the losses for the tasks to control the update of parameters. Automatic evaluations and crowdsourced manual evaluations show that the proposed model makes generated responses more emotionally aware.

Parallel sentences mining with transfer learning in an unsupervised setting

Downloads

Next from NAACL 2021

Non-Parametric Few-Shot Learning for Word Sense Disambiguation

Similar lecture

Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems

Parallel sentences mining with transfer learning in an unsupervised setting

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2021

Non-Parametric Few-Shot Learning for Word Sense Disambiguation

Similar lecture

Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems

Downloads