The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes

Auteur-es

Kiela Douwe, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, manpreet Singh, Pratik Ringshia, Davide Testuggine

Nombre Auteurs

7

Titre

The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes

Année de publication

2020

Référence (APA)

Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., & Testuggine, D. (2020). The Hateful Memes Challenge : Detecting Hate Speech in Multimodal Memes.

résumé

This work proposes a new challenge set for multimodal classification, focusing on detecting hate speech in multimodal memes. It is constructed such that unimodal models struggle and only multimodal models can succeed: difficult examples (“benign confounders”) are added to the dataset to make it hard to rely on unimodal signals. The task requires subtle reasoning, yet is straightforward to evaluate as a binary classification problem. We provide baseline performance numbers for unimodal models, as well as for multimodal models with various degrees of sophistication. We find that state-of-the-art methods perform poorly compared to humans, illustrating the difficulty of the task and highlighting the challenge that this important problem poses to the community.

URL

https://research.facebook.com/file/4239227162870648/The-Hateful-Memes-Challenge-Detecting-Hate-Speech-in-Multimodal-Memes.pdf

doi

https://doi.org/10.48550/arXiv.2005.04790

Accessibilité de l'article

Libre

Champ

Artificial Intelligence, Computer Vision, Machine Learning, Natural Language Processing & Speech

Type contenu (théorique Applicative méthodologique)

Applicatif

Méthode

The method involves evaluating a variety of models, belonging to one of three classes: unimodal models, multimodal models that were unimodally pretrained, and multimodal models that were multimodally pretrained. Baseline scores are established for these models on the task of detecting hate speech in multimodal memes.

Cas d'usage

N/A

Objectifs de l'article

The objectives of the article are to propose a new challenge set for multimodal classification, to evaluate the performance of various models on this task, and to highlight the challenges and difficulties associated with detecting hate speech in multimodal memes.

Question(s) de recherche/Hypothèses/conclusion

The research question is how well can unimodal and multimodal models detect hate speech in multimodal memes?

The hypothesis is that performance relative to humans is still poor, indicating that there is a lot of room for improvement.

The conclusions are that the task of detecting hate speech in multimodal memes presents a challenge for both unimodal and multimodal models, and that there is a lot of room for improvement in this area.

Cadre théorique/Auteur.es

The theoretical framework of the article is not explicitly stated, but the main authors cited include Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Danna Gurari.

Concepts clés

Multimodal classification, Hate speech detection, Visual modality Natural language processing.

Données collectées (type source)

Hateful memes

Définition des émotions

Non

Ampleur expérimentation (volume de comptes)

Non renseigné

Technologies associées

Natural language processing, Computer vision, Machine learning.

Mention de l'éthique

Non