Singapore

EMNLP 2023

Re-Examining Summarization Evaluation across Multiple Quality Criteria | VIDEO

rouge

summarization evaluation

evaluation metrics

summarization

evaluation

workshop paper

**Welcome to EMNLP 2023!**

On behalf of the EMNLP 2023 Organizing Committee, I extend a warm and heartfelt welcome to all of you to the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). It is with immense pleasure and excitement that we gather here in Singapore, a vibrant hub of innovation and technological advancement.

The conference program is packed with insightful presentations, thought-provoking workshops, and engaging networking opportunities. In addition to the technical sessions, we have also planned several social events that will provide you with the opportunity to connect with your colleagues.

The conference is held in person at the Resorts World Convention Centre in Singapore, and available on-line with the help of Underline.

For those in Singapore, I hope that you will find time to explore the exciting Sentosa Island, Gardens by the Bay, Singapore Botanic Gardens and the many other attractions unique to Singapore.

Of course, we are grateful to our sponsors and partners for their generous support of EMNLP 2023, whose contributions make it possible for us to host this world-class event.

Yuji Matsumoto (RIKEN AIP) 
EMNLP 2023 General Chair

To access the **EMNLP 2023** event page on Underline, you need to register for the Conference. 
Please follow **[this link](https://2023.emnlp.org/registration/)** for more details.

EMNLP 2023 took place in Singapore from Dec 6th to Dec 10th, 2023.

FREDSum: A Dialogue Summarization Corpus for French Political Debates | VIDEO

Mitigating Framing Bias with Polarity Minimization Loss | VIDEO

Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders

LLM aided semi-supervision for efficient Extractive Dialog Summarization

Citance-Contextualized Summarization of Scientific Papers

Evaluating the Impact of Retrieval on Multi-document Summarization

Can you Summarize my learnings? Towards Perspective-based Educational Dialogue Summarization

Using LLM for Improving Key Event Discovery: Temporal-Guided News Stream Clustering with Event Summaries

Synthesize, if you do not have: An Effective Synthetic Dataset Creation Strategy for Unsupervised Opinion Summarization in E-commerce

Recently, the emergence of ChatGPT has attracted wide attention from the computational linguistics community. Many prior studies have shown that ChatGPT achieves remarkable performance on various NLP tasks in terms of automatic evaluation metrics. However, the ability of ChatGPT to serve as an evaluation metric is still underexplored. Considering assessing the quality of natural language generation (NLG) models is an arduous task and NLG metrics notoriously show their poor correlation with human judgments, we wonder whether ChatGPT is a good NLG evaluation metric. In this report, we provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. In detail, we regard ChatGPT as a human evaluator and give task-specific (e.g., summarization) and aspect-specific (e.g., relevance) instruction to prompt ChatGPT to evaluate the generated results of NLG models. We conduct experiments on five NLG meta-evaluation datasets (including summarization, story generation and data-to-text tasks). Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments in most cases. In addition, we find that the effectiveness of the ChatGPT evaluator might be influenced by the creation method of the meta-evaluation datasets. For the meta-evaluation datasets which are created greatly depending on the reference and thus are biased, the ChatGPT evaluator might lose its effectiveness. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.

Is ChatGPT a Good NLG Evaluator? A Preliminary Study

https://d3smihljt9218e.cloudfront.net/transcript/59337/subtitle/aa79491c4d8a49e4151f07d7bd9dd0e0.vtt

English (automatic)

Downloads

Next from EMNLP 2023

FREDSum: A Dialogue Summarization Corpus for French Political Debates | VIDEO