Croatia

Automatically generated emotion arcs, which capture how an individual or a population feels over time, are widely used in industry and research. However, there is little research on evaluating the accuracy of these generated arcs. This is partly due to the difficulty of establishing the true (gold) emotion arc. Our work, for the first time, systematically and quantitatively evaluates automatically generated emotion arcs. We also compare two common ways of generating emotion arcs: Machine-Learning (ML) models and Lexicon-Only (LexO) methods. Using 36 diverse datasets, we systematically study factors that impact the quality of generated arcs. We show that despite being markedly poor at instance level, LexO methods are highly accurate at generating emotion arcs by aggregating information from hundreds of instances. These findings have wide-spread implications for both commercial development as well as research in fields such as psychology, public health, digital humanities, and social sciences, as LexO methods are simple, interpretable, have low carbon footprint, and do not require domain-specific training data or extensive programming expertise.

EACL 2023

Frustratingly Easy Sentiment Analysis of Text Streams: Generating High-Quality Emotion Arcs Using Emotion Lexicons

emotion lexicons

emotion arcs

sentiment analysis

cross-lingual nlp

poster

### Welcome to the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL) 
Continuing its mission of expanding and involving the science community of all European countries, EACL had selected the Ukrainian community for the 16th EACL, which took place online due to the COVID pandemic. Unfortunately, the ongoing war made the organisation in Kyiv impossible. Considering the importance of physical interaction among researchers, especially after the restrictions imposed by the COVID pandemic, in addition to an online mode, the conference will be held in Dubrovnik, Croatia, from 2 to 6 of May, 2023. The original aim of strengthening the connection with the Ukrainian community will not change as our program will feature a dedicated session and a workshop to highlight work on Ukrainian language technologies. As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

You need to register for the conference in order to access this site. Please visit https://2023.eacl.org/registration for more information.

As the flagship European conference in the field of computational linguistics, EACL welcomes European and international researchers covering a broad spectrum of research areas that are concerned with computational approaches to natural language.

Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reliable question answering systems. Furthermore, multilingual and cross-lingual resources in emergent domains are scarce, leading to few or no such systems.
In this paper, we demonstrate a cross-lingual open-retrieval question answering system for the emergent domain of COVID-19.
Our system adopts a corpus of scientific articles to ensure that retrieved documents are reliable. To address the scarcity of cross-lingual training data in emergent domains, we present a method utilizing automatic translation, alignment, and filtering to produce English-to-all datasets.
We show that a deep semantic retriever greatly benefits from training on our English-to-all data and significantly outperforms a BM25 baseline in the cross-lingual setting.
We illustrate the capabilities of our system with examples and release all code necessary to train and deploy such a system.

Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains

In recent years, COVID-19 has impacted all aspects of human life. As a result, numerous publications relating to this disease have been issued. Due to the massive volume of publications, some retrieval systems have been developed to provide researchers with useful information. In these systems, lexical searching methods are widely used, which raises many issues related to acronyms, synonyms, and rare keywords. In this paper, we present a hybrid relation retrieval system, CovRelex-SE, based on embeddings to provide high-quality search results.

CovRelex-SE: Adding Semantic Information for Relation Search via Sequence Embedding

The development of poetry generation system mainly focuses on enhancing the capacity of generation model. However, the demands of customization and polishing are generally ignored, which highly reduces the scope of application. In this work, we present Yu Sheng, a web-based poetry generation system that is featured a human-in-loop generation framework, providing various customization options for users with different backgrounds to engage in the process of poetry composition. To this end, we propose two methods and train the models that can perform constrained generation and fine-grained polishing. The automatic and human evaluation results show that our system has a strong ability to generate and polish poetry compared to other vanilla models. Our system is publicly accessible at: https://yusheng.cis.um.edu.mo.

Yu Sheng: Human-in-Loop Classical Chinese Poetry Generation System

We develop FISH, an interactive system that extracts and highlights crucial textual signals from financial statements efficiently and precisely. The system is available online at https://cfda.csie.org/FISH/, and a short video for introduction is at https://youtu.be/ZbvZQ09i6aw.

FISH: A Financial Interactive System for Signal Highlighting

We present GATE Teamware 2: an open-source web-based platform for managing teams of annotators working on document classification tasks. GATE Teamware 2 is an entirely re-engineered successor to GATE Teamware, using contemporary web frameworks. The software allows the management of teams of multiple annotators, project managers and administrators - including the management of annotators - across multiple projects. Projects can be configured to control and monitor the annotation statistics and have a highly flexible JSON-configurable annotation display which can include arbitrary HTML. Optionally, documents can be uploaded with pre-existing annotations and documents are served to annotators in a random order by default to reduce bias. Crucially, annotators can be trained on applying the annotation guidelines correctly and then screened for quality assurance purposes, prior to being cleared for independent annotation. GATE Teamware 2 can be self-deployed, including in container orchestration environments, or provided as private, hosted cloud instances. GATE Teamware 2 is an open-source software and can be downloaded from Github.

GATE Teamware 2: An open-source tool for collaborative document classification annotation

Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it’s promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. Furthermore, we suggest several use cases where the data collected with CoTEVer can be utilized for enhancing the faithfulness of explanations. Our toolkit is publicly available at https://github.com/SeungoneKim/CoTEVer

CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification

We present TrialsSummarizer, a system that aims to automatically summarize
evidence presented in the set of randomized controlled trials most relevant to a given query.
Building on prior work, the system retrieves trial publications matching a query
specifying a combination of condition, intervention(s), and outcome(s), and ranks
these according to sample size and estimated study quality.The top-k such studies are passed through a neural multi-document summarization system, yielding a synopsis of these trials. We consider two architectures: A standard sequence-to-sequence model based on BART, and a multi-headed architecture intended to provide greater transparency to end-users.Both models produce fluent and relevant summaries of evidence retrieved for queries, but their tendency to introduce unsupported statements render them inappropriate for use in this domain at present. The proposed architecture may help users verify outputs allowing users the ability to interact with outputs and to trace generated tokens back to inputs.

Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges

State-of-the-art models for identifying offensive language often fail to generalize over more nuanced or implicit cases of offensive and hateful language. Understanding model performance on complex cases is key for building robust models that are effective in real-world settings. To help researchers efficiently evaluate their models, we introduce OLEA, a diagnostic, open-source, extensible Python library that provides easy-to-use tools for error analysis in the context of detecting offensive language in English. OLEA packages analyses and datasets proposed by prior scholarship, empowering researchers to build effective, explainable and generalizable offensive language classifiers.

OLEA: Tool and Infrastructure for Offensive Language Error Analysis in English

Electronic Medical Records are integral parts of modern healthcare. Part of the records are clinical notes that healthcare providers take during encounters with patients. Notes are key to differential analysis which is the reasoning process leading to diagnosis and treatment. This paper presents DAVE, a differential analysis automation and visualization to assist healthcare professionals through the differential analysis process. DAVE takes as input clinical notes as they are being written by professionals and suggests candidate diagnostic algorithms. We digitized textbook diagnostic algorithms into directed acyclic graphs. We trained a distributional semantics model using an annotated corpora of electronic medical records and text from diagnostic algorithm descriptions. The model, boosted with PUBMed-based semantic similarity metrics, ranks the diagnostic algorithm graphs, and suggests the top three. The model achieved 74.3% success rate and was highly accepted by multiple medical professionals for usability

DAVE: Differential Diagnostic Analysis Automation and Visualization from Clinical Notes

We introduce an SSMT (Speech to Speech Machine Translation, aka Speech to Speech Video Translation) Pipeline 1 , as a web application for translating videos from one language to another by cascading multiple language modules. Our speech translation system combines highly accurate speech to text (ASR) for Indian English, pre-possessing modules to bridge ASR-MT gaps such as spoken disfluency and punctuation, robust machine translation (MT) systems for multiple language pairs, SRT module for translated text, text to speech (TTS) module and a module to render translated synthesized audio on the original video. It is user-friendly, flexible, and easily accessible system. We aim to provide a complete configurable speech translation experience to users and researchers with this system. It also supports human intervention where users can edit outputs of different modules and the edited output can then be used for subsequent processing to improve overall output quality. By adopting a human-in-the-loop approach, the aim is to configure technology in such a way where it can assist humans and help to reduce the involved human efforts in speech translation involving English and Indian languages. As per our understanding, this is the first fully integrated system for English to Indian languages (Hindi, Telugu, Gujarati, Marathi, and Punjabi) video translation. Our evaluation shows that one can get 3.5+ MOS score using the developed pipeline with human intervention for English to Hindi. A short video demonstrating our system is available at https://youtu.be/b3hDRhLGxns.

Downloads

Next from EACL 2023

Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains