United States

Large language models (LLMs) can generate jokes that rival human comedy in quality, yet their originality remains uncertain. This study develops and validates automated, corpus-based methods for quantifying joke originality by focusing on core &quot;topic handles&quot; – the key nouns or short noun phrases that capture a joke&#39;s script opposition and logical mechanism (per the General Theory of Verbal Humor). Using a reference corpus of one million English jokes from Reddit, we compute Pointwise Mutual Information (PMI) on handle pairs in three variants (raw co-occurrence, semantic-cluster smoothing, and word- decomposition) and two embedding-based measures (handle-level conceptual distance and full-text corpus novelty via Sentence-BERT). We tested the measures on 400 LLM- generated jokes (200 from GPT-4o and 200 from GPT-5.4 prompted for original two-sentence jokes) and 80 benchmark jokes from the Witscript-powered JEST benchmark (including non- humorous controls). Three professional comedians rated all items on 0–3 scales of originality and funniness. Automated measures, especially raw PMI average (ρ = .386) and corpus novelty (ρ = .372), correlated significantly with human originality ratings but showed no relation to funniness. Benchmark jokes were rated most original, and the originality– funniness relationship differed sharply across sources (positive for benchmark, negative for GPT-4o). The results demonstrate that handle-based PMI and semantic novelty metrics offer practical, quantitative tools for assessing originality in both human and AI-generated humor, advancing objective evaluation of computational creativity while highlighting distinct patterns in how general-purpose LLMs versus specialized humor systems produce novel jokes.

ACL 2026

Navigating the Joke Space: Towards Automated Originality Assessment of AI-Generated Humor

joke space

originality

computational humor

### **Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!**

<iframe width=1280 height=720 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/player/crQbitV66dw?width=1280&height=720&ff=1&title=0" allowfullscreen="true"></iframe>

Computational creativity is an interdisciplinary endeavor that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, social sciences and the arts. The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:

* to construct a program or computer capable of showing some level of creativity
* to better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans
* to design programs that can enhance human creativity without necessarily being creative themselves

<br>The ICCC conference series organized by the [**Association for Computational Creativity**](https://computationalcreativity.net/) since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.



To access the Twelfth International Conference on Computational Creativity, ICCC’21, you need to register for the conference.  Instructions on how to access the event will be provided directly to attendees and included on the Twelfth International Conference on Computational Creativity, ICCC’21 conference website. 

[Register now](https://computationalcreativity.net/iccc21/registration/)

Register now

ICCC'21

Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!

Large language models (LLMs) can generate jokes that rival human comedy in quality, yet their originality remains uncertain. This study develops and validates automated, corpus-based methods for quantifying joke originality by focusing on core "topic handles" – the key nouns or short noun phrases that capture a joke's script opposition and logical mechanism (per the General Theory of Verbal Humor). Using a reference corpus of one million English jokes from Reddit, we compute Pointwise Mutual Information (PMI) on handle pairs in three variants (raw co-occurrence, semantic-cluster smoothing, and word- decomposition) and two embedding-based measures (handle-level conceptual distance and full-text corpus novelty via Sentence-BERT). We tested the measures on 400 LLM- generated jokes (200 from GPT-4o and 200 from GPT-5.4 prompted for original two-sentence jokes) and 80 benchmark jokes from the Witscript-powered JEST benchmark (including non- humorous controls). Three professional comedians rated all items on 0–3 scales of originality and funniness. Automated measures, especially raw PMI average (ρ = .386) and corpus novelty (ρ = .372), correlated significantly with human originality ratings but showed no relation to funniness. Benchmark jokes were rated most original, and the originality– funniness relationship differed sharply across sources (positive for benchmark, negative for GPT-4o). The results demonstrate that handle-based PMI and semantic novelty metrics offer practical, quantitative tools for assessing originality in both human and AI-generated humor, advancing objective evaluation of computational creativity while highlighting distinct patterns in how general-purpose LLMs versus specialized humor systems produce novel jokes.

workshop paper

It is my great pleasure and honor to welcome you to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), held in San Diego, California, USA, July 2–7, 2026. ACL 2026 continues our fields tradition of excellence in scholarship, innovation, and inclusivity, and I am deeply grateful to the many volunteers who have worked tirelessly to bring this event to life.

It is exciting to see the many developments in the field reflected at our conference, such as the advancement of large language models (born out of our community) into many application directions. This is not without challenges, partly due to unsolved problems, and partly due to misuse of the technology. All this is reflected well at ACL 2026. The conference also reflects that the field is not only about advancing technology, but also the science of fundamentally understanding language and how it is use by many different speaker communities. <br>
[Full Message from General Chair Philipp Koehn](https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing) <br><br><br>
<html><button style="display: inline-flex; align-items: center; justify-content: center; white-space: nowrap; border-radius: 9999px; font-weight: bold; background: #7c3aed; color: white; font-family: 'Space Grotesk', sans-serif; height: 40px; font-size: 16px; padding: 0 20px; border: none; cursor: pointer" onclick="window.open('https://drive.google.com/file/d/1QiJFOmMjvDgsDo3i1XzlQnImxe3RU_Cy/view?usp=sharing')">Conference Handbook</button></html>
<br><br><br><br>

To access this event page, you need to log in with the **email address you registered with**. <br>Access credentials will be sent to your email from Underline -  subject line "Welcome to ACL 2026". Please be sure to check your spam email folder if you do not see an email confirmation right away.

To access this event page, you are required to register.
Please complete your registration to continue.

We recommend reading [**the registration information**](https://2026.aclweb.org/registration/) first.

**Online Registration Form**: https://acl.swoogo.com/acl2026

Registration Required

The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) will take place in San Diego, California, from July 2 - 7, 2026.

Computational mental health (CMH) classifiers often degrade under distribution shift because human annotators and distant-supervision pipelines reward different linguistic signals. 
 
 We introduce TSS (Triple-Stream Stress probe), a multi-channel diagnostic framework that decomposes text into (A) lexical character n-grams, (B) a small, mostly content-free morpho-syntactic channel, and (C) a 154-feature psycholinguistic style channel. Across four English datasets (N = 12,906), TSS reveals a lexical interference effect: adding lexical features to the style channel reduces Macro-F1 on human-labeled data (mean drop 0.072, p < 10⁻⁴) but not on auto-labeled data. 
 
 We propose Degree of Divergence (DoD), a difference-in-differences statistic adapted from econometrics for label-source auditing, with instance-level bootstrap inference; the headline estimate is DoD(BC–A) = 0.0374, 95% CI [0.0097, 0.0651], p = 0.0032. 
 
 A platform-stratified Twitter-only DoD (which removes the Reddit vs. Twitter contrast) reproduces the pattern with bootstrap inference: DoDᵀᵥ,BC–A = +0.096 (p < 0.001) and DoDᵀᵥ,AC–A = −0.089 (p < 0.001). Interventional masking (pos_only) retains ∼95–99% of Channel C's performance after destroying content words on human datasets, indicating that the style channel does not rely primarily on lexical surface form. 
 
 TSS is positioned as a diagnostic audit framework, not a clinical screening tool: it flags label-source-specific shortcut learning before generalization claims are made.

The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

Large language models achieve strong performance on biomedical question answering and summarization benchmarks, yet traditional evaluation metrics often fail to detect clinically significant factual errors. We introduce a unified evaluation framework that combines reference-based measures with evidence-grounded factuality verification to assess biomedical text generation. Evaluating four open-source models across three benchmarks (BioASQ, PubMedQA, MedLFQA), we find that 13.4–24.7% of generated claims are contradicted and 23–41% are unsupported, despite high lexical overlap scores. Our proposed Fact-Aligned Score (FAS) correlates strongly with claim-level verifiability (rho=0.68), substantially outperforming ROUGE-L (rho=0.41). We release an open-source toolkit with model outputs and analysis scripts to support reproducible factuality evaluation and safer deployment of biomedical LLMs.

Towards Unified Factuality Evaluation for Biomedical QA and Summarization: Aligning Metrics with Clinical Use-Cases

Seizure freedom is a key clinical outcome for people with epilepsy (PWE) yet it is primarily recorded in free-text notes and letters in the United Kingdom, making it difficult to aggregate and track at scale. This paper introduces a generative LLM-based pipeline boosted by synthetic data to identify a PWE's seizure freedom status in clinicians' records. We fine-tuned seven different LLMs with between 4-14 billion parameters using LoRA to compare models trained on synthetic records against those trained on expert annotated records. The best performing configuration, based on Qwen-2.5-14B, was trained entirely on synthetic records and used chain-of-thought (CoT) reasoning (both generated by GPT-5). This achieved an F1 score of 0.90±0.02 on double-annotated test data and outperformed the equivalent model trained on authentic clinician records, which achieved 0.87±0.04. The synthetically trained models also have the benefit of outputting their CoT reasoning process for greater decision-making transparency and can also make use of the unused supervised training data for significantly increased test examples. This work has implications for monitoring a key treatment outcome for PWE automatically and at scale.

Using Synthetic Records to Improve Automated Identification of Seizure Freedom in Clinical Text about People with Epilepsy

This paper studies how to improve biomedical named entity recognition (NER) using large language models (LLMs), especially for low-resource languages like Bangla and Basque. The main goal is to understand how different prompt styles and output formats affect model performance.
 
 The study finds that the way we design prompts is very important. Among all methods, question-style prompting works best across all languages. It helps the model understand the biomedical task more clearly and improves accuracy. In fact, improvements are much greater in Bangla and Basque compared to high-resource languages like English and Spanish.
 
 Another key finding is about the output format. Traditional BIO tagging (labeling each word) performs poorly with LLMs because it is strict and sensitive to small errors. Instead, span-based extraction (directly extracting text phrases) works much better and gives higher F1 scores. This is because LLMs naturally generate text spans rather than token-level labels.
 
 The paper also analyzes errors. Common problems include hallucination, missing entities, and boundary mistakes. Translation-based prompts can reduce hallucination, while question-style prompts reduce empty outputs in biomedical NER.
 
 Overall, the study shows that choosing the right prompt and output format is very important, especially for low-resource high-vocabulary languages. It provides useful guidance for building better multilingual medical information extraction systems.

Analyzing Prompt Design Choices in Biomedical Information Extraction for Low-Resource Languages

Extracting structured knowledge from unstructured text is a fundamental challenge in machine learning, particularly for concepts organized within complex hierarchical ontologies. In genomics, identifying phenotypes from clinical narratives is crucial for diagnostic precision, yet current methods struggle with contextual interpretation and subtle clinical descriptions.
 We present a hierarchy-aware workflow for ontology-based phenotype linking that combines semantic and hierarchical signals. Our approach integrates Large Language Models for span detection with retrieval and a hybrid reranking strategy using both Euclidean (semantic) and hyperbolic (hierarchical) embeddings trained on the Human Phenotype Ontology.
 We show that while hyperbolic embeddings alone do not outperform standard semantic retrieval, they provide complementary structural signals that improve performance over strong baselines when combined with Euclidean representations. In particular, the hybrid approach outperforms existing state-of-the-art methods and yields more hierarchically coherent predictions, especially in settings involving implicit phenotype mentions.
 Experiments on a public benchmark (ID-68) and a newly released clinical dataset (CHU-50), publicly released with code and data, highlight both performance gains and improved alignment with ontology structure. We further introduce a hierarchy-aware evaluation framework that reflects clinical relevance beyond exact-match metrics.

Hierarchy-Aware Hyperbolic and Semantic Reranking for Ontology-Based Phenotype Linking

The MedCPT model has demonstrated that strong biomedical retrievers can be trained using proprietary PubMed search logs. In this work, we study whether freely available citation sentences are sufficient to train similarly effective models. We construct a large-scale training dataset of ~ 62 million citation sentence-abstract pairs extracted from PubMed Central. We train a lightweight BERT-based retriever-reranker model called CiteRec on this dataset and evaluate it across three benchmark settings: (a) the biomedical subset of BEIR for information retrieval, (b) SciRepEval for generalizable scientific document embeddings, and (c) CitancePlus, a new set of ~ 90 thousand citation sentence-abstract pairs for PubMed-scale citation recommendation.
 
 We show that CiteRec performs competitively with MedCPT on the biomedical BEIR subset and outperforms it on SciRepEval. On CitancePlus, CiteRec achieves strong performance for citation recommendation over the full PubMed corpus, outperforming both MedCPT and a substantially larger Qwen3-Embedding-8B retriever.

Training Biomedical Retrievers From Large-Scale Citation Contexts

Standard clinical Natural Language Processing (NLP) benchmarks often yield inflated metrics by forcing deterministic classification on ambiguous instances, thereby obscuring the clinical risks of overconfident predictions. To bridge this gap, we propose a risk-aware hybrid selective classification framework, evaluated on early Human Immunodeficiency Virus suspicion identification in Spanish clinical notes. Our dual-verification approach explicitly decouples aleatoric uncertainty through Mondrian conformal prediction and epistemic uncertainty using a Multi-Centroid Mahalanobis Distance veto. Empirical evaluations reveal that standard uncertainty metrics and baseline classifiers are structurally insufficient for safe medical triage, suffering severe coverage collapse when forced to operate under strict reliability constraints. In contrast, by demanding that clinical narratives pass both probabilistic and geometric safeguards, the proposed framework successfully isolates a highly trustworthy operational domain.The obtained results show that explicit, decoupled uncertainty quantification is essential for translating biomedical NLP into responsible clinical practice.

Reliable Automated Triage in Spanish Clinical Notes: A Hybrid Framework for Risk-Aware HIV Suspicion Identification

Retrieval strategy selection is a critical but understudied design decision in biomedical RAG systems. Existing evaluations rely on lexical metrics that miss answer grounding, or require proprietary infrastructure that limits reproducibility. We present BioRAG, a head-to-head ablation of seven retrieval strategies on BioASQ-13b, evaluated using four RAGAs metrics with a locally deployed judge at zero monetary cost. Hybrid BM25 plus dense retrieval with Reciprocal Rank Fusion achieves faithfulness of 0.534 and context recall of 0.507, improvements of 50% and 85% over naive dense retrieval, confirmed across three random seed re-samples. HyDE improves faithfulness by 14% but reduces context precision by 52%, a tradeoff not previously documented on BioASQ. No single strategy dominates all four metrics, indicating that strategy selection must be application-driven. Sensitivity analysis across k in {3,5,10} confirms ranking stability. A domain mismatch diagnostic confirms 2% corpus coverage failure. The full pipeline runs on consumer hardware without paid APIs, directly addressing BioNLP 2026's emphasis on reproducibility and evaluation frameworks for health-related applications.

BioRAG: A Systematic Ablation Study of Retrieval Strategies for Biomedical Question Answering

Drug repurposing methods rely heavily on knowledge graph (KG) embeddings, but building and curating these graphs takes considerable effort. We present two findings on the Hetionet drug-disease benchmark and an epilepsy ranking task. First, PubMedBERT text embeddings, fed through the same downstream classifiers and identical 10-fold splits as four re-trained KG baselines (TransE, ComplEx, DistMult, RotatE), reach AUROC $0.910$, above all four (best: RotatE, $0.854$); a Random Forest on the same vectors scores $0.880$. The comparison is asymmetric in one important way: PubMedBERT was pretrained on the literature Hetionet was curated from, so the result is best read as ``text-with-literature-supervision vs.graph-only,'' and a head-to-head with text-augmented KG methods (KG-BERT, TxGNN) is left as follow-up. Second, across all seven combinations of text, molecular (ECFP4), and gene expression (LINCS L1000) features, cross-attention fusion of weaker modalities into text consistently degrades performance, despite a gated mechanism intended to suppress unhelpful modalities; the residual path forces the strong modality to absorb noise. The model also ranks proconvulsants (amoxapine, flumazenil) near the top, because text embeddings encode strength of association with a disease but not its direction.

Beyond Knowledge Graphs: PubMedBERT Embeddings as a Competitive Standalone Modality for Drug Re-purposing

Clinical NLP systems are increasingly used for triage support, prediction, and decision assistance in EHR-based settings, where demographic fairness is a critical concern. A common evaluation approach is counterfactual demographic perturbation: modifying attributes such as age or sex while holding clinical evidence fixed and measuring output changes. 
 However, we show that such counterfactual audits can be misleading when interpreted in isolation. Across three clinical LLMs, we find that non-demographic control perturbations (e.g., paraphrases) often induce output variability comparable to or greater than demographic edits. 
 This can contribute to overestimation or misinterpretation of demographic bias.
 To address this, we propose a baseline-aware audit framework that explicitly compares demographic perturbations against control baselines. 
 Our analysis reveals that (i) label-level stability can mask substantial variation in generated rationales and recommendations, and (ii) age-based perturbations generally induce larger effects than sex-based ones in borderline cases. 
 Crucially, we identify a high intrinsic instability ("noise floor"; 0.46--0.71 Jaccard instability) in clinical LLM generations, while additional matched-metric analyses show that demographic perturbations are often comparable to non-demographic baseline variability.
 These findings highlight a key limitation of existing fairness evaluations: without establishing appropriate baselines, apparent demographic sensitivity may be over- or mis-attributed to bias rather than broader generative instability. We argue that baseline-aware counterfactual audits, which explicitly compare demographic effects against intrinsic model noise, provide a more reliable lens for evaluating clinical NLP systems in high-stakes settings.

Downloads

Next from ACL 2026

The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

Similar lecture

Witscript: A System for Generating Improvised Jokes in a Conversation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ACL 2026

The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

Similar lecture

Witscript: A System for Generating Improvised Jokes in a Conversation

Downloads