Ireland

We address a low-resource code-mixed (Spanish-Catalan) and domain-specific (Stroke) problem of clinical texts de-identification. We empirically find the strong presence of few-shot cross-lingual transfer in mBERT and apply it to our problem by annotating a few-shot training corpus and a development corpus. Our models achieved a 14%+ gain over zero-shot and have the potential for other low-resource applications.

ACL 2022

Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts

cross-lingual adaptation

few-shot ner

low-resource

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

workshop paper

In-domain language model pretraining has been shown to benefit various domain-specific down-stream tasks. In the biomedical domain, natural language generation (NLG) tasks such as medical dialogue system are of critical importance, while understudied. We emphasize the lack of in-domain generative language model and the unsystematic generative downstream benchmarks. We introduce the generative language model BioBART that adapts BART to the biomedical domain and collate various biomedical language generation tasks for model evaluation.

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

The introduction and experimental analysis of the proposed approach, Memory-aligned Knowledge Graph for Clinically Accurate RadiologyImage Report Generation.

Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation

Data augmentation is important in addressing data sparsity and low resources in NLP. Unlike data augmentation for other tasks such as sentence-level and sentence-pair ones, data augmentation for named entity recognition (NER) requires preserving the semantic of entities. To that end, in this paper we propose a simple semantic-based data augmentation method for biomedical NER. Our method leverages semantic information from pre-trained language models for both entity-level and sentence-level. Experimental results on two datasets: i2b2-2010 (English) and VietBioNER (Vietnamese) showed that the proposed method could improve NER performance.

Simple Semantic-based Data Augmentation for Named Entity Recognition in Biomedical Texts

Probing factual knowledge in Pre-trained Language Models (PLMs) using prompts has indirectly implied that language models (LMs) can be treated as knowledge bases. To this end, this phenomena has been effective, especially when these LMs are fine-tuned towards not just data, but also to the style or linguistic pattern of the prompts themselves. We observe that, satisfying a particular linguistic pattern in prompts is an unsustainable, time-consuming constraint in the probing task, especially because, they are often manually designed and the range of possible prompt template patterns can vary depending on the prompting task. To alleviate this constraint, we propose using a position-attention mechanism to capture positional information of each word in a prompt relative to the mask to be filled, hence avoiding the need to re-construct prompts when the prompts’ linguistic pattern changes. Using our approach, we demonstrate the ability of eliciting answers (in a case study on health outcome generation) to not only common prompt templates like Cloze and Prefix, but also rare ones too, such as Postfix and Mixed patterns whose masks are respectively at the start and in multiple random places of the prompt. More so, using various biomedical PLMs, our approach consistently outperforms a baseline in which the default PLMs representation is used to predict masked tokens.


Position-based Prompting for Health Outcome Generation

Text summarization (TS) is an important NLP task. Pre-trained Language Models (PLMs) have been used to improve the performance of TS. However, PLMs are limited by their need of labelled training data and by their attention mechanism, which often makes them unsuitable for use on long documents. To this end, we propose a hybrid, unsupervised, abstractive-extractive approach, in which we walk through a document, generating salient textual fragments representing its key points. We then select the most important sentences of the document by choosing the most similar sentences to the generated texts, calculated using BERTScore. We evaluate the efficacy of generating and using salient textual fragments to guide extractive summarization on documents from the biomedical and general scientific domains. We compare the performance between long and short documents using different generative text models, which are finetuned to generate relevant queries or document titles. We show that our hybrid approach out-performs existing unsupervised methods, as well as state-of-the-art supervised methods, despite not needing a vast amount of labelled training data.

GenCompareSum: a hybrid unsupervised summarization method using salience

Named entity recognition (NER) is one of the core technologies for knowledge acquisition from text and has been used for knowledge extraction of chemicals and medicine. 
As one of the NER improvement approaches, multitask learning that learns a model from multiple training data has been used. Among multitask learning, an auxiliary learning method, which uses training data of an auxiliary task for improving its target task, has shown higher NER performance than conventional multitask learning for improving all the tasks simultaneously. The conventional auxiliary learning method uses only one auxiliary training dataset. 
We propose Multiple Utilization of NER Corpora Helpful for Auxiliary BLESsing
(MUNCHABLES). MUNCHABLES utilizes multiple training datasets as auxiliary training data by the following methods : the first one is to fine-tune the NER model of the target task by sequentially performing auxiliary learning for each auxiliary training dataset, and the other is to use all training datasets in one auxiliary learning. We evaluate MUNCHABLES on eight chemical/biomedical/scientific domain NER tasks, where seven training datasets are used as auxiliary training data. The experiment results show that our proposed methods achieve higher NER performance than conventional multi-task learning methods on average and that NER performance can be improved by using multiple auxiliary training data. Furthermore, the proposed models outperform stateof-the-art models on the datasets

Auxiliary Learning for Named Entity Recognition with Multiple Auxiliary Biomedical Training Data

Self-supervised pre-training methods have brought remarkable breakthroughs in the understanding of text, image, and speech. Recent developments in genomics has also adopted these pre-training methods for genome understanding. However, they focus only on understanding haploid sequences, which hinders their applicability towards understanding genetic variations, also known as single nucleotide polymorphisms (SNPs), which is crucial for genome-wide association study. In this paper, we introduce SNP2Vec, a scalable self-supervised pre-training approach for understanding SNP. We apply SNP2Vec to perform long-sequence genomics modeling, and we evaluate the effectiveness of our approach on predicting Alzheimer's disease risk in a Chinese cohort. Our approach significantly outperforms existing polygenic risk score methods and all other baselines, including the model that is trained entirely with haploid sequences. We release our code and dataset on https://github.com/HLTCHKUST/snp2vec.

SNP2Vec: Scalable Self-Supervised Pre-Training for Genome-Wide Association Study

Automatic speech recognition (ASR) systems
usually incorporate postprocessing mechanisms
to remove disfluencies, facilitating the
generation of clear, fluent transcripts that are
conducive to many downstream NLP tasks.
However, verbal disfluencies have proved to
be predictive of dementia status, although little
is known about how various types of verbal
disfluencies, nor automatically detected disfluencies,
affect predictive performance. We experiment
with an off-the-shelf disfluency annotator
to tag disfluencies in speech transcripts
for a well-known cognitive health assessment
task. We evaluate the performance of this
model on detecting repetitions and corrections
or retracing, and measure the influence of goldannotated
versus automatically detected verbal
disfluencies on dementia detection through a
series of experiments. We find that removing
both gold and automatically-detected disfluencies
negatively impacts dementia detection
performance, degrading classification accuracy
by 5:6% and 3% respectively

How You Say It Matters: Measuring the Impact of Verbal Disfluency Tags on Automated Dementia Detection

We introduce Doctor XAvIer —a BERT-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. We present a novel performance plot and evaluation metric for feature attribution methods —Feature Attribution Dropping (FAD) curve and its Normalized Area Under the Curve (N-AUC). FAD curve analysis shows that integrated gradients outperforms Shapley values in explaining diagnosis classification. Doctor XAvIer outperforms the baseline with 0.97 F1-score in named entity recognition and symptom pertinence classification and 0.91 F1-score in diagnosis classification.

Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation

The PICO framework focuses on the Population, Intervention, Comparator, and Outcomes of clinical trial studies. Manually looking for PICO descriptions from the thousands of clinical studies is vital but one of the most resource-consuming tasks for writing systematic reviews. One of the significant barriers to fully automating PICO recognition is the lack of large manually annotated corpora. Distant supervision is one of the weak supervision methods used to obtain large entity annotated corpora. We propose a distant supervision method using open-access and open-source resources and use it to get a sizeable "Intervention" and "Comparator" entity annotated dataset called DISTANT-CTO. We use DISTANT-CTO to train downstream NER models and cross the previous state of the art on "Intervention" and "Comparator" extraction.

Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts

Downloads

Next from ACL 2022

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

Similar lecture

Adapting Coreference Resolution Models through Active Learning

Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2022

BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model

Similar lecture

Adapting Coreference Resolution Models through Active Learning

Downloads