Mexico

We describe a real-time system that receives a live audio stream from a jam session and generates lyric lines that are congruent with the live music being played. Two novel approaches are proposed to align the learned latent spaces of audio and text representations that allow the system to generate novel lyric lines matching live instrumental music. One approach is based on adversarial alignment of latent representations of audio and lyrics, while the other approach learns to transfer the topology from the music latent space to the lyric latent space. A user study with music artists using the system showed that the system was useful not only in lyric composition, but also encouraged the artists to improvise and find new musical expressions. Another user study demonstrated that users preferred the lines generated using the proposed methods to the lines generated by a baseline model.

ICCC&#39;21

LyricJam: A system for generating lyrics for live instrumental music

music audio representation learning

co-creative system

lyric generation

generative adversarial networks

natural language generation

variational autoencoders

### **Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!**

<iframe width=1280 height=720 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/player/crQbitV66dw?width=1280&height=720&ff=1&title=0" allowfullscreen="true"></iframe>

Computational creativity is an interdisciplinary endeavor that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, social sciences and the arts. The goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends:

* to construct a program or computer capable of showing some level of creativity
* to better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans
* to design programs that can enhance human creativity without necessarily being creative themselves

<br>The ICCC conference series organized by the [**Association for Computational Creativity**](https://computationalcreativity.net/) since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.



To access the Twelfth International Conference on Computational Creativity, ICCC’21, you need to register for the conference.  Instructions on how to access the event will be provided directly to attendees and included on the Twelfth International Conference on Computational Creativity, ICCC’21 conference website. 

[Register now](https://computationalcreativity.net/iccc21/registration/)

Register now

ICCC'21

Welcome to the Twelfth International Conference on Computational Creativity, ICCC’21!

technical paper

ML Design for Creative Pursuits

A chatbot is perceived as more humanlike and likeable if it includes some jokes in its output. But most existing joke generators were not designed to be integrated into chatbots. This paper presents Witscript, a novel joke generation system that can improvise original, contextually relevant jokes, such as humorous responses during a conversation. The system is based on joke writing algorithms created by an expert comedy writer. Witscript employs well-known tools of natural language processing to extract keywords from a topic sentence and, using wordplay, to link those keywords and related words to create a punch line. Then a pretrained neural network language model that has been fine-tuned on a dataset of TV show monologue jokes is used to complete the joke response by filling the gap between the topic sentence and the punch line. A method of internal scoring filters out jokes that don't meet a preset standard of quality. Human evaluators judged Witscript's responses to input sentences to be jokes more than 40% of the time. This is evidence that Witscript represents an important next step toward giving a chatbot a humanlike sense of humor.

Witscript: A System for Generating Improvised Jokes in a
Conversation

We explore a masked-language model based on BERT for shifting the meaning of text towards a target theme.
Content words in the original text are masked and the model provides a list of filling candidates, out of which one is selected based on its similarity to the theme and constraints regarding morphology and metre.
Experimentation is performed with Portuguese song lyrics and trade-offs between grammaticality, semantics, form, and novelty are analysed.
We confirm that BERT is a useful tool for creative text transformation.

Exploring a Masked Language Model for Creative TextTransformation

We present an interactive computational approach to fill curatorial gaps between artworks in a fine arts exhibition. We describe the algorithmic details of our semantic
approach based on word embedding of keywords and how we include additional curatorial constraints. We present an installation at a museum exhibition and discuss lessons learned during our arts based research.

Computational filling of curatorial gaps in a fine arts exhibition

We present a system that automatically generates music from visual art based on the perceived emotion of the given input. We propose the generated music as a framing device that can enhance the aesthetic experience of people viewing Computational Creativity (CC) outputs. In this paper, we carry out a first study to test this by comparing the aesthetic experience of viewing paintings generated by CC systems accompanied by either textual framing, our proposed musical framing or both. We evaluate our system by means of qualitative user evaluations, which require participants to rank their aesthetic experience from best to worst. The results from the study demonstrated that the musical framing generated by our system provided a better aesthetic experience for users compared to the textual framing. Furthermore, the results suggest that with more work, a combination of textual and musical framing could be used to further improve the aesthetic experience for people viewing visual CC art.

Framing through Music: A Pilot Study

We ask the question: to what extent can recent large-scale language and image generation models blend visual concepts? Given an arbitrary object, we identify a relevant object and generate a single-sentence description of the blend of the two using a language model. We then generate a visual depiction of the blend using a text-based image generation model. Quantitative and qualitative evaluations demonstrate the superiority of language models over classical methods for conceptual blending, and of recent large-scale image generation models over prior models for the visual depiction.

Visual Conceptual Blending with Large-scale Language and Vision Models

Computer vision research into image aesthetic quality assessment seeks to use machine learning to measure the aesthetic quality of images, which bears resemblance to a variety of topics within computational creativity, but has not been discussed in our community. To foster a conversation around this research literature, we trace the development of computer vision algorithms foraesthetic judgment over the past fifteen years and critically consider whether these algorithms actually cross the “aesthetic gap” proposed by the first researchers in this space. We then build towards a more fundamental question regarding machine learning and subjectivity.

What Does it Take to Cross the Aesthetic Gap? The Development of Image Aesthetic Quality Assessment in Computer Vision

MuSyFI is a system that tries to model an inspirational
computational creative process. It uses images as source
of inspiration and begins by implementing a possible
translation between visual and musical features. Results
of this mapping are fed to a Genetic Algorithm (GA)
to try to better model the creative process and produce
more interesting results. Three different musical artifacts
are generated: an automatic version, a co-created
version, and a genetic version. The automatic version
maps features from the image into musical features nondeterministically;
the co-created version adds harmony
lines manually composed by us to the automatic version;
finally, the genetic version applies a genetic algorithm
to a mixed population of automatic and co-created
artifacts.
The three versions were evaluated for six different
images by conducting surveys. They evaluated whether
people considered our musical artifacts music, if they
thought the artifacts had quality, if they considered the
artifacts ’novel’, if they liked the artifacts, and lastly if
they were able to relate the artifacts with the image in
which they were inspired. We gathered a total of 300
answers and overall people answered positively to all
questions, which confirms our approach was successful
and worth further exploring.

MuSyFI - Music Synthesis From Images

Intelligent learning environments have demonstrated effectiveness for providing individualized instruction to students of computer science (CS). However, the great potential of intelligent agents has not yet been explored within expressive environments, which are increasingly common for supporting and motivating K-12 students. This paper presents the prototype design and implementation of a novel Co-creative Artificial Intelligence (CAI) integrated within EarSketch, an online environment for learning introductory computing concepts through code-driven, sample-based music remixing. CAI is intended to scaffold student learning from EarSketch's expressive computing curriculum by co-creating algorithmic music alongside a human learner. This paper presents an initial version of CAI, which engages with EarSketch users by offering menu-based dialogue and suggestions based on the state of a project. We report a pilot study in classrooms, showing promising results in students' satisfaction with the system's capabilities. The findings of this pilot study suggest the ability of a co-creative agent to support users in learning and creative objectives, and should inspire research into combined computational and creative user models.

LyricJam: A system for generating lyrics for live instrumental music

Downloads

Next from ICCC'21

ML Design for Creative Pursuits

Similar lecture

Witscript: A System for Generating Improvised Jokes in a Conversation

LyricJam: A system for generating lyrics for live instrumental music

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICCC'21

ML Design for Creative Pursuits

Similar lecture

Witscript: A System for Generating Improvised Jokes in a Conversation

Downloads