Lemotif: An Affective Visual Journal Using Deep Neural Networks

Auteur-es

X. Alice Li, Devi Parikh

Nombre Auteurs

2

Titre

Lemotif: An Affective Visual Journal Using Deep Neural Networks

Année de publication

2020

Référence (APA)

Li, X. A., & Parikh, D. (2020). Lemotif : An Affective Visual Journal Using Deep Neural Networks.

résumé

We present Lemotif, an integrated natural language processing and image generation system that uses machine learning to (1) parse a text-based input journal entry describing the user’s day for salient themes and emotions and (2) visualize the detected themes and emotions in creative and appealing image motifs. Synthesizing approaches from artificial intelligence and psychology, Lemotif acts as an affective visual journal, encouraging users to regularly write and reflect on their daily experiences through visual reinforcement. By making patterns in emotions and their sources more apparent, Lemotif aims to help users better understand their emotional lives, identify opportunities for action, and track the effectiveness of behavioral changes over time. We verify via human studies that prospective users prefer motifs generated by Lemotif over corresponding baselines, find the motifs representative of their journal entries, and think they would be more likely to journal regularly using a Lemotif-based app.

URL

https://research.facebook.com/file/804280910247449/Lemotif-An-Affective-Visual-Journal-Using-Deep-Neural-Networks.pdf

doi

https://doi.org/10.48550/arXiv.1903.07766

Accessibilité de l'article

Libre

Champ

Artificial Intelligence, Machine Learning, Human Computer Interaction & UXNatural Language Processing & Speech

Type contenu (théorique Applicative méthodologique)

Applicatif

Méthode

The method involves using natural language processing and machine learning to analyze journal entries and generate visual motifs that represent salient themes and emotions. The effectiveness of these mappings and the overall system is evaluated through studies with AMT users

Cas d'usage

Lemotif

Objectifs de l'article

The objective of the article are : to develop and evaluate a system that can visually represent journal entries, with the aim of making associations between feelings and life events more apparent to users and
to present a system that helps users better understand their emotional lives and track the effectiveness of behavioral changes over time
to evaluate the effectiveness of the system through human studies.

Question(s) de recherche/Hypothèses/conclusion

The research question is whether Lemotif, the system presented in the article, can effectively encourage users to regularly write and reflect on their daily experiences.

The hypothesis is that the creative and appealing image motifs generated by Lemotif will make journaling more actionable and engaging for users.

The conclusions are that Lemotif is effective in encouraging users to journal regularly and that the generated motifs are meaningful and representative of users' journal entries.

Cadre théorique/Auteur.es

The theoretical framework of the article includes psychology, NLP, affective computing and machine learning, and the main authors cited include Pennebaker, Cowen and Keltner.

Concepts clés

Image generation, Color-emotion mapping, Journaling

Données collectées (type source)

The data collected are text/journal entries from 500 respondents on AMT. These entries were used for training and analysis.

Définition des émotions

Categorical emotions

Ampleur expérimentation (volume de comptes)

500 respondents

Technologies associées

Natural language processing, Machine learning, Image generation

Mention de l'éthique

Non

Finalité communicationnelle

"Lemotif aims to make associations between feelings and parts of a user’s life more apparent, presenting opportunities to take actions towards improved emotional well being.

We also find that subjects are interested in using an app like Lemotif and consider the generated motifs representative of their journal entries."