Portugal

We present Lemotif, an integrated natural language processing and image generation system that uses machine learning to (1) parse a text-based input journal entry describing the user&#39;s day for salient themes and emotions and (2) visualize the detected themes and emotions in creative and appealing image motifs. Synthesizing approaches from artificial intelligence and psychology, Lemotif acts as an affective visual journal, encouraging users to regularly write and reflect on their daily experiences through visual reinforcement. By making patterns in emotions and their sources more apparent, Lemotif aims to help users better understand their emotional lives, identify opportunities for action, and track the effectiveness of behavioral changes over time. We verify via human studies that prospective users prefer motifs generated by Lemotif over corresponding baselines, find the motifs representative of their journal entries, and think they would be more likely to journal regularly using a Lemotif-based app.

ICCC&#39;20

Lemotif: An Affective Visual Journal Using Deep Neural Networks

creative ai natural language processing image synthesis neural networks affective journaling visualizing emotions

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

We present Lemotif, an integrated natural language processing and image generation system that uses machine learning to (1) parse a text-based input journal entry describing the user's day for salient themes and emotions and (2) visualize the detected themes and emotions in creative and appealing image motifs. Synthesizing approaches from artificial intelligence and psychology, Lemotif acts as an affective visual journal, encouraging users to regularly write and reflect on their daily experiences through visual reinforcement. By making patterns in emotions and their sources more apparent, Lemotif aims to help users better understand their emotional lives, identify opportunities for action, and track the effectiveness of behavioral changes over time. We verify via human studies that prospective users prefer motifs generated by Lemotif over corresponding baselines, find the motifs representative of their journal entries, and think they would be more likely to journal regularly using a Lemotif-based app.

technical paper

Computational creativity is the art, science, philosophy and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative. As a field of research, this area is thriving, with progress in formalising what it means for software to be creative, along with many exciting and valuable applications of creative software in the sciences, the arts, literature, gaming and elsewhere.

The ICCC conference series organized by the Association for Computational Creativity since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.


ICCC'20

The ICCC conference series organized by the Association for Computational Creativity since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.
Computational creativity is the art, science, philosophy and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviors that unbiased observers would deem to be creative. As a field of research, this area is thriving, with progress in formalizing what it means for software to be creative, along with many exciting and valuable applications of creative software in the sciences, the arts, literature, gaming and elsewhere.



"This work is on the transformation of text relying mostly on a common analogy vector operation, computed in a distributional model, i.e., static word embeddings.
Given a theme, original song lyrics have their words replaced by new ones, related to the new theme as the original words are to the original theme.
As this is not enough for producing good lyrics, towards more coherent and singable text, constraints are gradually applied to the replacements.
Human opinions confirmed that more balanced lyrics are obtained when there is a one-to-one mapping between original words and their replacement; only content words are replaced; and the replacement has the same part-of-speech and rhythm as the original word."

WeirdAnalogyAutomatic: Experimenting with Analogy for Lyrics Transformation

Generation and transformation of images and videos using artificial intelligence have flourished over the past few years. Yet, there are only a few works aiming to produce creative 3D shapes, such as sculptures. Here we show a novel 3D-to-3D topology transformation method using Generative Adversarial Networks (GAN). We use a modified pix2pix GAN, which we call Vox2Vox, to transform the volumetric style of a 3D object while retaining the original object shape. In particular, we show how to transform 3D models into two new volumetric topologies - the 3D Network and the Ghirigoro. We describe how to use our approach to construct customized 3D representations. We believe that the generated 3D shapes are novel and inspirational. Finally, we compare the results between our approach and a baseline algorithm that directly convert the 3D shapes, without using our GAN.

3D Topology Transformation with Generative Adversarial Networks

"Self-expression is essential to processing our thoughts and feelings and is central to successful mental health therapy. Art therapy provides a wider range of expressive mechanisms than offered through traditional approaches, allowing individuals to process their emotions when traditional therapies prove unsuccessful. Yet, effective expression through art therapy may call on a level of artistic experience that is not available to all.  As such, a lack of expertise or comfort with artistic expression may hinder one's ability to receive needed mental health support. 
 
Creative machines can offer novel therapeutic approaches by offloading the need for creative expertise and opening up creative self-expression to those who lack the corresponding experience. In this paper, we focus on bereavement, and explore a co-creative songwriting system, ALYSIA, as a new form of therapy for those who had recently suffered the loss of a loved one. We evaluate the utility of this creative system in aiding bereaved individuals through several case studies. In addition, we discuss the utility of co-creative systems to the therapeutic context with potential application to  a broad range of therapies."

Co-Creative Songwriting for Bereavement Support

We consider the advent of computational creativity systems that are potentially far superior to human creators, at least in some domains. To do so, we revisit the concepts of P- and H-creativity and propose a social refinement called group-relative creativity (G-creativity). Using this reconceptualization, we explore several critical questions regarding computational creativity’s effect on human creators.

What Happens When a Computer Joins the Group?

"The presentation describes an exploratory study involving humans and computational musical agents. In the study, the system's co-creative behavior
was varied by manipulating a single experimental parameter to engender different co-creative experiences in rhythm duets. In one experimental condition, the system induced a sense of producing divergent material, influencing creative outcomes, and negotiating musical outcomes, three aspects that were consistent with group music creativity. These preliminary findings provide supportive evidence that the system’s parameter can be varied to systematically engender co-creative experiences distinct from creative support."

Engendering co-creative experiences through parametric variation

“Hello” is an interactive cinematic generative artwork that gained traction within the Cultural Adventures research project, an international endeavor between Portugal and Thailand. The encompassing rationale assumed the need of a link connecting both countries, and thus “Hello” was born from the similarities that can be found in commuters that flock to and from both capital cities by train, often more engaged with their mobile phones than with their immediate surroundings – people, train or traversed landscapes –, which constitute more of a shapeless blurred audiovisual background than a sharp attention-grabbing focus. The artwork is also used as a demonstrator of the potential of generative art in producing complex cinematic narrative figurative artworks, yet permeated with interference, chaos and randomness, and how the audience perceives and makes sense of them through the ability to identify and create patterns, to conceptualize, to deal with information overload, and to imagine and build a narrative through abductive reasoning from incomplete yet suggestive information.

Hello, An Interactive Cinematic Generative Artwork

Many machine learning approaches are focused on defining artificial agents able to find solutions to a certain problem given fixed design tools or parameters to optimize. In order to do that, creators must have a certain knowledge of the solution space to define design parameters that ensure enough exploration allowing agents to find its best configuration. However, this approach may limit artificial agents since they are restricted by their initial conditions of a certain design problem. In addition, specific initial conditions also limit them to scale across multiple challenges. In this paper, we explore how the definition of more general design tools can allow artificial agents to better explore the solution space and generalize through multiple design problems. To do that, we compare design artifacts produced by an artificial agent that learns to construct 2D shapes with a fixed number of pieces to another artificial agent that also learns to add or remove pieces from its design proposal. We demonstrate how by allowing more freedom in design, an artificial system is able to produce more novel artifacts with higher performances in multiple scenarios.

Exploring the flexibility of a design tool through different artificial agents

Casual creators are a genre of creativity support tool that use generative systems to empower autotelic and enjoyable amateur creativity. They have been posited as a unique means of democratising creativity through the support of user exploration via system generativity, yet little is known about what casual creators are actually available to wider audiences. We conducted a qualitative analysis of currently available casual creators on the App Store. We found three categories of interaction techniques across 89 commercially available apps that qualified as casual creators, which we describe in their exploration potential, feedback speed, and user autonomy.

Creating a Typology of Casual Creators in the Wild

Being creative in movement-based improvisational environments, such as dance floors, poses a difficult challenge for computers. It is particularly challenging for computers to judge the creativity of inputs and responses in these open-ended domains. LuminAI is an improvisational dance installation where an artificially intelligent agent dances with a user. In this paper, we enable the agent to judge the creativity of dance gestures through the development of creativity metrics, including the novelty, value, and surprise of a gesture. We then use these metrics to implement a lead-and- follow dynamic: when the user is performing less creatively, the agent tries to lead by performing more creatively. A user study is performed to compare the original and lead-and-follow systems, with results showing users found the lead-and-follow system’s gestures lower in quality than the original, more surprising than the original, and found the original system more engaging and influential on their actions.

Creativity Metrics for a Lead-and-Follow Dynamic in an Improvisational Dance Agent

"This paper reports on the automatic selection of short-texts for amplifying the range of a given input's context, e.g. a news headline. Different methods are applied to select a corresponding expression from a list, which should be as semantically related to the input as possible. This study was developed for the Portuguese language, and considered expressions are proverbs, where figurative language is predominant.
The set of explored methods includes some based on word overlap, others on static word embeddings, and also on recent contextual embeddings. To compare the explored methods in this subjective scenario, a survey was answered by humans, who rated the value of the selected expressions in terms of relatedness with regard to the input, and the humoristic value that may arise from this selection. The main conclusion was that simpler approaches, which end up selecting expressions that share words with the headline, are more easily related and considered to be funnier than other more elaborate approaches, which are more focused on the context."

Lemotif: An Affective Visual Journal Using Deep Neural Networks

Downloads

Next from ICCC'20

WeirdAnalogyAutomatic: Experimenting with Analogy for Lyrics Transformation

Similar lecture

Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training

Lemotif: An Affective Visual Journal Using Deep Neural Networks

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICCC'20

WeirdAnalogyAutomatic: Experimenting with Analogy for Lyrics Transformation

Similar lecture

Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training

Downloads