“That’s so cute!”: The CARE Dataset for Affective Response Detection

Auteur-es

Jane Dwivedi-Yu, Alon Y. Halevy

Nombre Auteurs

2

Titre

“That’s so cute!”: The CARE Dataset for Affective Response Detection

Année de publication

2022

Référence (APA)

Dwivedi-Yu, J., & Halevy, A. Y. (2022). “That’s so cute!” : The CARE Dataset for Affective Response Detection.

résumé

Social media plays an increasing role in our communication with friends and family, and in our consumption of entertainment and information. Hence, to design effective ranking functions for posts on social media, it would be useful to predict the affective responses of a post (e.g., whether it is likely to elicit feelings of entertainment, inspiration, or anger). Similar to work on emotion detection (which focuses on the affect of the publisher of the post), the traditional approach to recognizing affective response would involve an expensive investment in human annotation of training data.

We create and publicly release CAREdb, a dataset of 230k social media post annotations according to seven affective responses using the Common Affective Response Expression (CARE) method. The CARE method is a means of leveraging the signal that is present in comments that are posted in response to a post, providing high-precision evidence about the affective response to the post without human annotation. Unlike human annotation, the annotation process we describe here can be iterated upon to expand the coverage of the method, particularly for new affective responses. We present experiments that demonstrate that the CARE annotations compare favorably with crowdsourced annotations. Finally, we use CAREdb to train competitive BERT-based models for predicting affective response as well as emotion detection, demonstrating the utility of the dataset for related tasks.

Mots-clés

affective response, social media, emotion classification, CARE dataset, human annotation, machine learning

URL

https://research.facebook.com/file/393453116239728/Thats-so-cute--The-CARE-Dataset-for-Affective-Response-Detection.pdf

doi

https://doi.org/10.48550/arXiv.2201.11895

Accessibilité de l'article

Libre

Champ

Artificial Intelligence

Type contenu (théorique Applicative méthodologique)

Méthodologique, applicatif

Méthode

The method used in this article is the Common Affective Response Expression (CARE) method, which is a cost-effective alternative to human annotation for predicting the emotional impact of social media posts. The CARE method involves two steps: (1) identifying the CARE patterns in the text, and (2) using a lexicon to label the publisher affect of the comments. The authors also compare the performance of the CARE method to a state-of-the-art emotion classifier.

Annotations according to the CARE method, as well as a human annotated dataset for comparison.

Cas d'usage

Reddit

Objectifs de l'article

The objectives of the article are to introduce the CARE dataset, evaluate its accuracy compared to crowdsourced annotations, and explore its potential applications in social media ranking functions.

Question(s) de recherche/Hypothèses/conclusion

The research question is whether the CARE dataset can accurately predict affective response in social media posts.

The hypothesis is that the CARE dataset will perform comparably to crowdsourced annotations in predicting affective response in social media posts.

The authors conclude that the CARE dataset is a valuable resource for predicting affective response in social media posts, and that it performs comparably to crowdsourced annotations. They also suggest that the CARE dataset could be used to improve social media ranking functions.

Cadre théorique/Auteur.es

The theoretical framework of the article is based on previous work in affective computing and emotion classification. The main authors cited include Stark and Hoey (2021), Demszky et al. (2020), Lin et al. (2008), Ma et al. (2005), and Mohammad and Kiritchenko (2015).

Concepts clés

Affective response, Social Media, Emotion classification, Machine learning

Données collectées (type source)

Our experiments use a dataset that is created from Reddit posts and comments in the pushshift.io database that were created between 2011 and 2019.

Définition des émotions

Non

Ampleur expérimentation (volume de comptes)

34 million comments from
24 million distinct posts

TOTAL in CARE : 230k

Technologies associées

CARE method, machine learning algorithms, Emotion classifier

Mention de l'éthique

Yes, there is mention of ethics in this article. The authors note that the CARE method should not be used for ill-intended purposes such as purposefully recommending particular content to manipulate a user's perception or preferences.

Finalité communicationnelle

"We described a method for extracting training data for models that predict the affective responses of a post on social media"

Commentaires

Voir la bibliographie de l'article, plusieurs références qui pourraient entrer dans la RL.