United States

News headline generation has seen significant advancements in resource-rich languages like the English language, leveraging sophisticated natural language processing (NLP) techniques. However, similar progress has not been observed for low-resource Indian languages, particularly Telugu. We focus on implementing news headline generation given the news article using an abstractive summarization approach, which enables the generation of contextually rich headlines by interpreting and rephrasing content using deep learning techniques. We create a dataset of articles and headlines scraped from the Telugu news website `Sakshi&#39;. We use pre-trained language models such as mBART50, MT5, and IndicBART for fine-tuning using our dataset. Our findings show the effectiveness of fine-tuning pre-trained models for news headline-generation tasks in Telugu.

EMNLP 2024

News Headline Generation in Telugu

mbart50

artificial intelligence in telugu

headlines in telugu

indian languages processing

automated headline generation

indicbart

sentence similarity

bleu

rouge

news headline generation

web scraping

computational linguistics

text summarization

dataset creation

machine learning

natural language processing

deep learning

fine-tuning

language models

# Welcome everyone to ACL 2022!

The 60th Annual Meeting of the Association for Computational Linguistics is taking place May 22-27, 2022 as a hybrid event, in Dublin and online. We are happy to welcome all of you to this anniversary edition with an almost 50-50 in-person and virtual participation. 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, three amazing keynote events and two new initiatives of invited talks: Spotlight Talks for Young Rising Stars (STIRS) and The Next Big Idea Talks. Posters (including Findings of ACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop will have an oral session and a poster session as part of Poster Session 1. The program also features eight Tutorials and 28 Workshops. 

 
We wish you a wonderful conference! 
[**The ACL 2022 Organizing Committee**](https://www.2022.aclweb.org/organisers)
 
[**Conference Handbook**](https://drive.google.com/file/d/1_BUCMfhMVrjG9E2e71aHdHeE28KSje0l/view?usp=sharing) 
[**Mini Handbook**](https://drive.google.com/file/d/1qlBKl0wzmlVF1oCeMQl3BahLd9nLP5Ce/view?usp=sharing) 
[**Posters and Demo guides**](https://drive.google.com/file/d/1UucMAoCNncIOaH1rMMDa0owuG9qgvJTG/view?usp=sharing)

ACL 2022

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing (NLP). 

News headline generation has seen significant advancements in resource-rich languages like the English language, leveraging sophisticated natural language processing (NLP) techniques. However, similar progress has not been observed for low-resource Indian languages, particularly Telugu. We focus on implementing news headline generation given the news article using an abstractive summarization approach, which enables the generation of contextually rich headlines by interpreting and rephrasing content using deep learning techniques. We create a dataset of articles and headlines scraped from the Telugu news website `Sakshi'. We use pre-trained language models such as mBART50, MT5, and IndicBART for fine-tuning using our dataset. Our findings show the effectiveness of fine-tuning pre-trained models for news headline-generation tasks in Telugu.

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

Automatic metrics are extensively used to evaluate natural language processing systems. However, there has been increasing focus on how they are used and reported by practitioners within the field. In this paper, we have con ducted a survey on the use of automatic metrics, focusing particularly on natural language gen eration (NLG) tasks. We inspect which metrics are used as well as why they are chosen and how their use is reported. Our findings from this survey reveal significant shortcomings, in cluding inappropriate metric usage, lack of im plementation details and missing correlations with human judgements. We conclude with rec ommendations that we believe authors should follow to enable more rigour within the field.

Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices

Modern biomedical concept representations are mostly trained on synonymous concept names from a biomedical knowledge base, ignoring the inter-concept interactions and a concept's local neighborhood in a knowledge base graph. In this paper, we introduce Biomedical Entity Representation with a Graph-Augmented Multi-Objective Transformer (BERGAMOT), which adopts the power of pre-trained language models (LMs) and graph neural networks to capture both inter-concept and intra-concept interactions from the multilingual UMLS graph. We apply contrastive loss on textual and graph representations to make them less sensitive to surface forms and enable intermodal knowledge exchange between two uni-modal encoders. BERGAMOT achieves state-of-the-art results in zero-shot entity linking without task-specific supervision on three monolingual datasets and Mantra multilingual benchmark.

Biomedical Entity Representation with Graph-Augmented Multi-Objective Transformer

Automated Emotion Recognition in language is a challenging task that has attracted considerable attention especially in recent years. We present a summary of our findings and observations based on a thorough analysis of recent resources and related practices.

Emotion-Annotated Data in NLP: Perspectives on Recent Resources and Practices

The textual content that is fed to Large Language Models (LLMs) can sometimes be noisy con- taining misspelled words. We investigate the effect of such mistakes on the performance of 9 language models, with parameters ranging from 0.2B to 13B, in 3 different NLP tasks, namely Natural Language Inference (NLI), Name Entity Recognition (NER), and Intent Classification (IC). We perform our experiments on 6 different languages (English, German, French, Spanish, Hindi, and Turkish) and build a dictionary of real-world noise for them using Wikipedia edit history. We show that the performance gap of the studied models on the clean and noisy test data averaged across all the datasets and languages ranges from 2.3 to 4.3 absolute percentage points. In addition, mT5 models, in general, show more robustness compared to BLOOM, Falcon, and BERT-like models. In particular, mT5 (13B), was the most robust on average overall, across the 3 tasks, and in 4 out of the 6 languages.

Exploring Robustness of Multilingual LLMs on Real-World Noisy Data

Large language models (LLMs) have significantly improved their reasoning and decision-making capabilities, as seen in methods like ReAct. However, despite its effectiveness in tackling complex tasks, ReAct faces two main challenges: losing focus on the original question and becoming stuck in action loops. To address these issues, we introduce Focused ReAct, an enhanced version of the ReAct paradigm that incorporates reiteration and early stop mechanisms. These improvements help the model stay focused on the original query and avoid repetitive behaviors. Experimental results show accuracy gains of 18% to 530% and a runtime reduction of up to 34% compared to the original ReAct method.

Focused ReAct: Improving ReAct through Reiterate and Early Stop

We investigate the knowledge of object af- fordances in pre-trained language models (LMs) and pre-trained Vision-Language mod- els (VLMs). A growing body of literature shows that PTLMs fail inconsistently and non- intuitively, demonstrating a lack of reasoning and grounding. To take a first step toward quan- tifying the effect of grounding (or lack thereof), we curate a novel and comprehensive dataset of object affordances – TEXT2AFFORD, charac- terized by 15 affordance classes. Unlike affor- dance datasets collected in vision and language domains, we annotate in-the-wild sentences with objects and affordances. Experimental results reveal that PTLMs exhibit limited rea- soning abilities when it comes to uncommon object affordances. We also observe that pre- trained VLMs do not necessarily capture ob- ject affordances effectively. Through few-shot fine-tuning, we demonstrate improvement in affordance knowledge in PTLMs and VLMs. Our research contributes a novel dataset for lan- guage grounding tasks, and prese

Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text

To predict upcoming text, language models must in some cases retrieve in-context information verbatim. In this report, we investigated how the ability of language models to retrieve arbitrary in-context nouns developed during training (across time) and as language models trained on the same dataset increase in size (across scale). We then asked whether learning of in-context retrieval correlates with learning of more challenging zero-shot benchmarks. Furthermore, inspired by semantic effects in human short-term memory, we evaluated the retrieval with respect to a major semantic component of target nouns, namely whether they denote a concrete or abstract entity, as rated by humans. We show that verbatim in-context retrieval developed in a sudden transition early in the training process, after about 1% of the training tokens. This was observed across model sizes (from 14M and up to 12B parameters), and the transition occurred slightly later for the two smallest models. We further found that the development of verbatim in-context retrieval is positively correlated with the learning of zeroshot benchmarks. Around the transition point, all models showed the advantage of retrieving concrete nouns as opposed to abstract nouns. In all but two smallest models, the advantage dissipated away toward the end of training.

Transformer verbatim in-context retrieval across time and scale

Interpretability studies have played an important role in the field of NLP. They focus on the problems of how models encode information or, for instance, whether linguistic capabilities allow them to prefer grammatical sentences to ungrammatical. Recently, several studies examined whether the models demonstrate patterns similar to humans and whether they are sensitive to the phenomena of interference like humans' grammaticality judgements, including the phenomenon of agreement attraction. In this paper, we probe BERT and GPT models on the syntactic phenomenon of agreement attraction in Russian using the psycholinguistic data with syncretism. Working on the language with syncretism between some plural and singular forms allows us to differentiate between the effects of the surface form and of the underlying grammatical feature. Thus we can further investigate models' sensitivity to this phenomenon and examine if the patterns of their behaviour are similar to human patterns. Moreover, we suggest a new way of comparing models' and humans' responses via statistical testing. We show that there are some similarities between models' and humans' results, while GPT is somewhat more aligned with human responses than BERT. Finally, preliminary results suggest that surface form syncretism influences attraction, perhaps more so than grammatical form syncretism.

Of Models and Men: Probing Neural Networks for Agreement Attraction with Psycholinguistic Data

The ability to compare by analogy, metaphorically or not, lies at the core of how humans understand the world and communicate. In this paper, we study the likelihood of metaphoric outputs, and the capability of a wide range of pretrained transformer-based language models to identify metaphors from other types of analogies, including anomalous ones. In particular, we are interested in discovering whether language models recognise metaphorical analogies equally well as other types of analogies, and whether the model size has an impact on this ability. The results show that there are relevant differences using perplexity as a proxy, with the larger models reducing the gap when it comes to analogical processing, and for distinguishing metaphors from incorrect analogies. This behaviour does not result in increased difficulties for larger generative models in identifying metaphors in comparison to other types of analogies from anomalous sentences in a zero-shot generation setting, when perplexity values of metaphoric and non-metaphoric analogies are similar.

How Are Metaphors Processed by Language Models? The Case of Analogies

Stories are central for interpreting experiences, communicating, and influencing each other via films, medical, media, and other narra tives. Quantifying the similarity between sto ries has numerous applications including detect ing IP infringement, detecting hallucinations, search/recommendation engines, and guiding human-AI collaborations. Despite this, tradi tional NLP text similarity metrics are limited to short text distance metrics like n-gram over laps and embeddings. Larger texts require pre processing with significant information loss through paraphrasing or multi-step decomposi tion. This paper introduces AIStorySimilarity, a novel benchmark to measure the semantic dis tance between long-text stories based on core structural elements drawn from narrative the ory and script writing. Based on four narrative elements (characters, plot, setting, and themes) as well 31 sub-features within these, we use a SOTA LLM (gpt-3.5-turbo) to extract and evaluate the semantic similarity of a diverse set of major Hollywood movies. In addition, we compare human evaluation with story simi larity scores computed three ways: extracting elements from film scripts before evaluation (Elements), directly evaluating entire scripts (Scripts), and extracting narrative elements from the parametric memory of SOTA LLMs without any provided scripts (GenAI). To the best of our knowledge, AIStorySimilarity is the first benchmark to measure long-text story similarity using a comprehensive approach to narrative theory. All code, data, and plot image f iles are available at https://github.com/ jon-chun/AIStorySimiliarity.

Next from EMNLP 2024

Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices

Similar lecture

Updated Headline Generation: Creating Updated Summaries for Evolving News Stories