The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents

Auteur-es: Shuster, K., Ju, D., Roller, S., Dinan, E., Boureau, Y.-L., & Weston, J.
Nombre Auteurs: 6
Titre: The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
Année de publication: 2020
Référence (APA): Shuster, K., Ju, D., Roller, S., Dinan, E., Boureau, Y.-L., & Weston, J. (2020). The Dialogue Dodecathlon : Open-Domain Knowledge and Image Grounded Conversational Agents. https://doi.org/10.18653/v1/2020.acl-main.222
résumé: We introduce dodecaDialogue: a set of 12 tasks that measures if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, discuss topics and situations, and perceive and converse about images. By multi-tasking on such a broad large-scale set of data, we hope to both move towards and measure progress in producing a single unified agent that can perceive, reason and converse with humans in an open-domain setting. We show that such multi-tasking improves over a BERT pretrained baseline, largely due to multi-tasking with very large dialogue datasets in a similar domain, and that the multi-tasking in general provides gains to both text and image-based tasks using several metrics in both the finetune and task transfer settings. We obtain stateof-the-art results on many of the tasks, providing a strong baseline for this challenge.
URL: https://research.facebook.com/file/556482462439112/The-Dialogue-Dodecathlon-Open-Domain-Knowledge-and-Image-Grounded-Conversational-Agents.pdf
doi: https://doi.org/10.18653/v1/2020.acl-main.222
Accessibilité de l'article: Libre
Champ: Artificial Intelligence, Natural Language Processing & Speech
Type contenu (théorique Applicative méthodologique): Méthodologique, applicatif
Méthode: Dialogue Dodecathlon, which is a set of 12 tasks that measure a conversational agent's ability to communicate engagingly with personality and empathy, utilize knowledge resources, discuss topics and situations, and converse about images.
Using metrics, multi-tasking, single task fine-tuning, zero-shot Transfer and human evaluation.
Cas d'usage: N/A
Objectifs de l'article: The objectives of the article are to introduce the Dialogue Dodecathlon as a way to measure the performance of conversational agents and to produce a single unified agent that can perceive, reason, and converse with humans in an open-domain setting.
Question(s) de recherche/Hypothèses/conclusion: The conclusion are that "The goal of introducing this task is not just as another challenge dataset, but to further motivate building and evaluating conversational agents capable of multiple skills – one of the core goals of AI. We believe current systems are closer to that goal than ever before – but we also still have a long way to go."
Cadre théorique/Auteur.es: The theoretical framework of the article includes natural language processing, machine learning, and conversational agents. The main authors cited include Dinan et al., Rashkin et al., and See et al.
Concepts clés: Conversational agents, Empathy, Knowledge grounding, Situation grounding, Image grounding.
Données collectées (type source): Cornell Movie: 309,987 training, 38,974 validation, 38,636 test utterances
LIGHT: 110,877 training, 6,623 validation, 13,272 test utterances
ELI5: 231,410 training, 9,828 validation, 24,560 test utterances
Ubuntu: 1,000,000 training, 19,560 validation, 18,920 test utterances
Twitter: 2,580,428 training, 10,405 validation, 10,405 test utterances
pushshift.io Reddit: approximately 2,200 million training, 10,000 validation, 10,000 test utterances
Image Chat: 355,862 training, 15,000 validation, 29,991 test utterances
IGC: 4,353 training, 486 validation, 7,773 test utterances
Définition des émotions: Non
Technologies associées: Natural language processing, Machine learning, Conversational agents, BERT, Image+Seq2Seq
Mention de l'éthique: Non
Finalité communicationnelle: "Such an agent should be able to get to know you when you first talk to it (ConvAI2), discuss everyday topics (DailyDialog, pushshift.io Reddit, Twitter, Cornell Movie), speak knowledgeably at depth (Wizard of Wikipedia, Ubuntu) and answer questions on such topics (ELI5)."

Pages du site: Contenu

Fait partie de The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents