United States

We introduce a novel continued pre-training method, MELT (MatEriaLs-aware continued pre-Training), specifically designed to efficiently adapt the pre-trained language models (PLMs) for materials science. Unlike previous adaptation strategies that solely focus on constructing domain-specific corpus, MELT comprehensively considers both the corpus and the training strategy, given that materials science corpus has distinct characteristics from other domains. To this end, we first construct a comprehensive materials knowledge base from the scientific corpus by building semantic graphs. Leveraging this extracted knowledge, we integrate a curriculum into the adaptation process that begins with familiar and generalized concepts and progressively moves toward more specialized terms. We conduct extensive experiments across diverse benchmarks to verify the effectiveness and generality of MELT. A comprehensive evaluation convincingly supports the strength of MELT, demonstrating superior performance compared to existing continued pre-training methods. In-depth analysis also shows that MELT enables PLMs to effectively represent materials entities compared to the existing adaptation methods, thereby highlighting its broad applicability across a wide spectrum of materials science.

EMNLP 2024

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science

adaptive unlearning

multilingual unlearning

machine unlearning

**General Chair Welcome**

![](https://assets.underline.io/markdown_image/1/image/d6e6dddc08bf2371b9136ea9c18dc5ab.png)              

Welcome to ICALEO® 2023!
	
We were excited to host the world’s leading international conference on Lasers and Electro-Optics in Chicago. The Laser has been shown to have the potential to revolutionize solutions for major global challenges like energy usage and supply, global warming, and CO2 footprint. Even the fusion energy breakthrough is based on laser technology. The conference, with peer-reviewed scientific and technical presentations, focused on the newest results in Micro Applications, Macro Applications, Additive Manufacturing, Beam Shaping and Frontiers in Laser Applications. Additionally, the Battery-Manufacturing Track gave an overview of the process combined with the newest advances in laser application results. 

The Business Session brought the business aspect and environment of the laser to the audience and the possibility to discuss with seasoned industry leaders from around the world. 

Whether this is the first time you are hearing about ICALEO or you are a longtime attendee, you will find something of interest, new ideas, and an excellent opportunity to network with other laser specialists worldwide. 

**Klaus Löffler **  
ICALEO General Chair   
Managing Director, Precitec GmbH

**What is ICALEO?**

ICALEO® brings together the brightest minds, seasoned professionals, and pioneering researchers from around the world. With a laser-focused agenda, ICALEO offers an exceptional platform to delve into the cutting-edge advancements, diverse applications, and transformative research that define lasers and electro-optics.

Participants will be immersed in a dynamic environment that fosters collaborative learning, networking, and idea exchange. Through engaging plenary presentations, technical sessions, an industry business panel, and networking events, attendees gain insights into the forefront of laser technology. From industrial applications to medical breakthroughs, additive manufacturing to materials processing, ICALEO covers the entire spectrum of laser-related disciplines. Moreover, the event serves as a launching pad for synergistic partnerships, paving the way for professionals to forge connections that shape the future of our industry.

Join us in Chicago and be part of an unparalleled gathering that challenges the boundaries of possibility within lasers and electro-optics. As we convene to share knowledge, explore innovations, and cultivate collaborations, ICALEO stands as a pivotal force driving the evolution of this transformative technology.

**Who attended?**

ICALEO is tailor-made for professionals, researchers, academics, and enthusiasts who are deeply passionate about the world of lasers and electro-optics. If you're an industry trailblazer, an academic luminary, an aspiring expert, or a postgraduate student looking to dive into the latest developments and trends, this is an event you won't want to miss.

Industrial leaders seeking to stay ahead of the curve in their fields, engineers working on laser applications, and manufacturing experts interested in additive manufacturing and materials processing will find ICALEO to be an invaluable platform. Researchers and scientists exploring the forefront of laser technology, from fundamental principles to groundbreaking applications, will also discover a wealth of knowledge and networking opportunities.

Medical professionals investigating laser-based medical advancements, entrepreneurs seeking to innovate with lasers, educators aiming to stay updated with the latest in laser education, and policy makers shaping the industry's landscape are all essential attendees. Whether you're an established expert or a rising star, the 42nd annual ICALEO promises to offer a diverse ecosystem of expertise, connections, and knowledge sharing that caters to the laser and electro-optics communities.

**Fields Represented:**

* Entrepreneurs and Innovators
* Manufacturing Experts
* Students and Graduates
* Policy Makers and Regulators
* Consultants and Advisors
* Technology Enthusiasts
* Suppliers and Manufacturers
* Startups and Innovators
* Researchers in Laser Education

**[View All Proceedings Here](https://icaleo.conferencespot.org/event-data)**

### **Program Tracks & Chairs**

To access the site please register [**here**](https://icaleo.org/attend). 

Please register for the ICALEO 2023 Conference to join the event.

ICALEO 2023

ICALEO® brings together the brightest minds, seasoned professionals, and pioneering researchers from around the world. With a laser-focused agenda, ICALEO offers an exceptional platform to delve into the cutting-edge advancements, diverse applications, and transformative research that define lasers and electro-optics.

poster

## Welcome to EMNLP 2024! 
We are excited to welcome you to one of the most prominent conferences in the field of Natural Language Processing. This year, EMNLP 2024 is being held in a hybrid format,
offering both virtual and in-person participation in beautiful Miami. Due to a record-breaking number of submissions, we've expanded the total number of accepted papers to accommodate more cutting-edge research from around the globe.
### [Conference Handbook](https://drive.google.com/file/d/1WPROgxjLAC96AJL7Ugy0tEnYm7dkrbHt/view?usp=sharing)

You are required to register for this event. **Please register [here](https://2024.emnlp.org/registration/).** The EMNLP 2024 event page on Underline will be open to public one week prior to the event.

Please register!

EMNLP 2024 will take place in Miami, Florida from Nov 12th to Nov 16th, 2024, at the Hyatt Regency Miami Hote and on Underline for remote participants.

Pretrained language models memorize vast amounts of information, including private and copyrighted data, raising significant safety concerns. Retraining these models after excluding sensitive data is prohibitively expensive, making machine unlearning a viable, cost-effective alternative. Previous research has focused on machine unlearning for monolingual models, but we find that unlearning in one language does not necessarily transfer to others. This vulnerability makes models susceptible to low-resource language attacks, where sensitive information remains accessible in less dominant languages. This paper presents a pioneering approach to machine unlearning for multilingual language models, selectively erasing information across different languages while maintaining overall performance. Specifically, our method employs an adaptive unlearning scheme that assigns language-dependent weights to address different language performances of multilingual language models. Empirical results demonstrate the effectiveness of our framework compared to existing unlearning baselines, setting a new standard for secure and adaptable multilingual language models.

Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models

Large Language Models (LLMs) demonstrate remarkable translation capabilities in high-resource language tasks, yet their performance in low-resource languages is hindered by insufficient multilingual data during pre-training. To address this, we conduct extensive multilingual continual pre-training on the LLaMA series models, enabling translation support across more than 100 languages. Through a comprehensive analysis of training strategies, such as vocabulary expansion and data augmentation, we develop LLaMAX. Remarkably, without sacrificing its generalization ability, LLaMAX achieves significantly higher translation performance compared to existing open-source LLMs (by more than 10 spBLEU points) and performs on-par with specialized translation model (M2M-100-12B) on the Flores-101 benchmark. Extensive experiments indicate that LLaMAX can serve as a robust multilingual foundation model. The code \footnote{\url{https://github.com/CONE-MT/LLaMAX/.}} and the models \footnote{\url{https://huggingface.co/LLaMAX/.}} are publicly available.

XLLaMA2: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages

The evolution of large language models (LLMs) has enhanced the planning capabilities of language agents in diverse real-world scenarios. Despite these advancements, the potential of LLM-powered agents to comprehend ambiguous user instructions for reasoning and decision-making is still under exploration. In this work, we introduce a new task, Proactive Agent Planning, which requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction, invoke external tools to collect valid information, and generate a plan to fulfill the user's demands. To study this practical problem, we establish a new benchmark dataset, Ask-before-Plan. To tackle the deficiency of LLMs in proactive planning, we propose a novel multi-agent framework, Clarification-Execution-Planning ($\texttt{CEP}$), which consists of three agents specialized in clarification, execution, and planning. We introduce the trajectory tuning scheme for the clarification agent and static execution agent, as well as the memory recollection mechanism for the dynamic execution agent. Extensive evaluations and comprehensive analyses conducted on the Ask-before-Plan dataset validate the effectiveness of our proposed framework.

Ask-before-Plan: Proactive Language Agents for Real-World Planning

Cross-platform topic dissemination is one of the research subjects that delved into media analysis; sometimes it fails to grasp the authentic topics due to platform-induced biases, which may be caused by aggregating documents from multiple platforms and running them on an existing topic model. This work deals with the impact of unique platform characteristics on the performance of topic models and proposes a new approach to enhance the effectiveness of topic modeling. The data utilized in this study consisted of a total of 1.5 million posts collected using the keyword ''ChatGPT'' on the three social media platforms. The devised model reduces platform influence in topic models by developing a platform-invariant contrastive learning algorithm and removing platform-specific jargon word sets. The proposed approach was thoroughly validated through quantitative and qualitative experiments alongside standard and state-of-the-art topic models and showed its supremacy. This method can mitigate biases arising from platform influences when modeling topics from texts collected across various platforms.

Platform-Invariant Topic Modeling via Contrastive Learning to Mitigate Platform-Induced Bias

Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of high-quality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-FACT, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-FACT includes factuality annotations of $112,276$ events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-FACT is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-FACT also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. We will release our dataset and codes to facilitate further research on event factuality detection.

MAVEN-FACT: A Large-scale Event Factuality Detection Dataset

When extracting structured information from text, recognizing entities and extracting relationships are essential. Recent advances in both tasks generate a structured representation of the information in an autoregressive manner, a time-consuming and computationally expensive approach. This naturally raises the question of whether autoregressive methods are necessary in order to achieve comparable results. In this work, we propose ITER, an efficient encoder-based relation extraction model, that performs the task in three parallelizable steps, greatly accelerating a recent language modeling approach: ITER achieves an inference throughput of over 600 samples per second for a large model on a single consumer-grade GPU. Furthermore, we achieve state-of-the-art results on the relation extraction datasets ADE and ACE05, and demonstrate competitive performance for both named entity recognition with GENIA and CoNLL03, and for relation extraction with SciERC and CoNLL04.

ITER: Iterative Transformer-based Entity Recognition and Relation Extraction

Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots employed responsibly for social good can be an enabler of positive individual and social change. Existing methods rely on fine-tuning persuasive chatbots with task-specific training data which is costly, if not infeasible, to collect. Furthermore, they employ only a handful of pre-defined persuasion strategies. 

We propose PersuaBot, a zero-shot chatbot based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies. PersuaBot uses an LLM to first generate a natural responses, from which the strategies used are extracted. To combat hallucination of LLMs, Persuabot replace any unsubstantiated claims in the response with retrieved facts supporting the extracted strategies. We applied our chatbot, PersuaBot, to three significantly different domains needing persuasion skills: donation solicitation, recommendations, and health intervention. 

Our experiments on simulated and human conversations show that our zero-shot approach is more persuasive than prior work, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.

Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval

Previous research leveraged Large Language Models (LLMs) in numerous ways in the educational domain. Here, we show that they can be used to answer exam questions simulating students of different skill levels and share a prompt, engineered for GPT-3.5, that enables the simulation of varying student skill levels on questions from different educational domains. We evaluate the proposed prompt on three publicly available datasets (one from science exams and two from English reading comprehension exams) and three LLMs (two versions of GPT-3.5 and one of GPT-4), and show that it is robust to different educational domains and capable of generalising to data unseen during the prompt engineering phase. We also show that, being engineered for a specific version of GPT-3.5, the prompt does not generalise well to different LLMs, stressing the need for prompt engineering for each model in practical applications. Lastly, we find that there is not a direct correlation between the quality of the rationales obtained with chain-of-thought prompting and the accuracy in the student simulation task.

Using LLMs to simulate students' responses to exam questions

Recent approaches to zero-shot commonsense reasoning have enabled Pre-trained Language Models (PLMs) to learn a broad range of commonsense knowledge without being tailored to specific situations. However, they often suffer from human reporting bias inherent in textual commonsense knowledge, leading to discrepancies in understanding between PLMs and humans. In this work, we aim to bridge this gap by introducing an additional information channel to PLMs. We propose Imagine (Machine Imagination-based Reasoning), a novel zero-shot commonsense reasoning framework designed to complement textual inputs with visual signals derived from machine-generated images. To achieve this, we enhance PLMs with imagination capabilities by incorporating an image generator into the reasoning process. To guide PLMs in effectively leveraging machine imagination, we create a synthetic pre-training dataset that simulates visual question-answering. Our extensive experiments on diverse reasoning benchmarks and analysis show that Imagine outperforms existing methods by a large margin, highlighting the strength of machine imagination in mitigating reporting bias and enhancing generalization capabilities.

Zero-shot Commonsense Reasoning over Machine Imagination

Many datasets have been developed to train and evaluate document-level relation extraction (RE) models. Most of these are constructed using real-world data. It has been shown that RE models trained on real-world data suffer from factual biases. To evaluate and address this issue, we present CovEReD, a counterfactual data generation approach for document-level relation extraction datasets using entity replacement. We first demonstrate that models trained on factual data exhibit inconsistent behavior: while they accurately extract triples from factual data, they fail to extract the same triples after counterfactual modification. This inconsistency suggests that models trained on factual data rely on spurious signals such as specific entities and external knowledge -- rather than on context -- to extract triples. We show that by generating document-level counterfactual data with CovEReD and training models on them, consistency is maintained with minimal impact on RE performance. We release our CovEReD pipeline as well as Re-DocRED-CF, a dataset of counterfactual RE documents, to assist in evaluating and addressing inconsistency in document-level RE.

Downloads

Next from EMNLP 2024

Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models

Similar lecture

The Melt Pool Dynamics on Different Substrate Materials In High-Speed Laser Directed Energy Deposition Process