Multilingual Multi-class Sentiment Classification Using Convolutional Neural Networks

Auteur-es

Attia, Mohammed; Samih, Younes; Elkahky, Ali; Kallmeyer, Laura

Nombre Auteurs

4

Titre

Multilingual Multi-class Sentiment Classification Using Convolutional Neural Networks

Année de publication

2018

Référence (APA)

Attia, M., Samih, Y., Elkahky, A., & Kallmeyer, L. (2018, mai). Multilingual Multi-class Sentiment Classification Using Convolutional Neural Networks. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). LREC 2018, Miyazaki, Japan. https://aclanthology.org/L18-1101

Mots-clés

sentiment analysis, sentiment detection, multi-class

URL

https://aclanthology.org/L18-1101

Accessibilité de l'article

Open access

Champ

Natural Language Processing

Type contenu (théorique Applicative méthodologique)

Applicative

Méthode

This paper describes our system for multilingual, multiclass sentiment classification using Convolutional Neural Networks (CNNs). We evaluate our system on three datasets in three different languages, and we find that state-of-art results can be achieved without language-specific features or pre-trained word embeddings.

Cas d'usage

User comments on social networks

Objectifs de l'article

Users generally express a broad variety of sentiments with a wide range of degrees, but to simplify the task, sentiment analysis approaches have traditionally classified sentiments into either positive, negative or neutral.
This paper describes our system for multilingual, multiclass sentiment classification using Convolutional Neural Networks (CNNs).

Question(s) de recherche/Hypothèses/conclusion

Research question(s) : Users generally express a broad variety of sentiments with a wide range of degrees, but to simplify the task, sentiment analysis approaches have traditionally classified sentiments into either positive, negative or neutral. This paper describes our system for multilingual, multiclass sentiment classification using Convolutional Neural Networks (CNNs).

Hypothesis(es) : The advantage of the proposed model is that it does not rely on language-specific features such as ontologies, dictionaries, or morphological or syntactic pre-processing. Equally important, our system does not use pre-trained word2vec embeddings which can be costly to obtain and train for some languages

Conclusion(s) : In this paper we have presented our systems for multi-class sentiment classification and we found that the deep neural network model can outperform traditional methods that rely on language-specific feature engineering. We show that the class imbalance in the data can lead to degradation in the system performance, and point out that oversampling can be a helpful workaround for handling this imbalance.

Cadre théorique/Auteur.es

Sentiment analysis (Pang et Lee, 2008 ; Lin et He, 2009 ; Cambria, 2016)

Knowledge based sentiment analysis (Mullen et Collier, 2004 ; Boiy et Moens, 2009 ; Godbole et al., 2007 ; Gamon, 2004 ; Wilson et al., 2005 ; Esuli et Sebastiani, 2006 ; Baccianella et al, 2010 ; Cambria et al., 2016),

Statistic-based sentiment analysis (Neethu et Rajasree, 2013 ; Maas et al., 2011 ; Tripathy et al., 2016 ; Pang et al., 2002 ; Mikolov et al., 2013b ; Mikolov et al., 2013a ; Pennington et al., 2014)

Deep learning for sentiment analysis (Glorot et al., 2011 ; Poria et al., 2016 ; dos Santos et Gatti, 2014)

Convolutional Neural Networks (LeCun et al., 1995 ; Krizhevsky et al., 2012 ; Graves et al., 2013 ; Kim, 2014 ; Johnson et Zhang, 2014)

Concepts clés

Sentiment analysis

Données collectées (type source)

Three publicly available datasets for English, German and Arabic :
- The Sanders Twitter Sentiment dataset (EN) 1 (Sanders, 2011) consists of tweets related to the products of four companies (Apple, Google, Microsoft, and Twitter). Tweets have been manually tagged as either positive, negative, neutral, or irrelevant with respect to the topic
- GermEval shared task (GE) consists of messages from various social media and web sources intended for analyzing customer reviews about “Deutsche Bahn”. The data is split roughly into 90% for training and 10% for development as shown in Table 2. They also provide two test sets. The shared task’s Subtask-B is on multi-class documentlevel polarity, which is about identifying whether the customer’s opinion of “Deutsche Bahn” or travel is positive, negative or neutral. The data was annotated by a team of six annotators, and each document was annotated by two annotators using WebAnno’s curation interface. The documents were checked for consistency by a supervisor who decided on divergences and new issues.
- The Arabic Sentiment Tweets Dataset (AR) consists of tweets which are classified as ‘positive’, ‘negative’, ‘mixed’, and ‘objective’. The tweets were collected from EgyptTrends and were not related to any particular topic, but generally included comments on diverse political issues. The tweets were annotated through the Amazon Mechanical Turk by three annotators. Tweets that were assigned the same rating by at least two annotators were accepted, otherwise rejected.

Définition des émotions

Definition of emotion analysis

Remarks on emotions, a complex object to define

Negative, neutral, positive labeling

Ampleur expérimentation (volume de comptes)

The Sanders Twitter Sentiment dataset (EN) consists of 5,513 tweets related to the products of four companies: Apple, Google, Microsoft, and Twitter.

GermEval shared task (GE) consists of 21,824 messages from various social media and web sources

The Arabic Sentiment Tweets Dataset (AR) consists of 10,006 tweets which are classified as ‘positive’, ‘negative’, ‘mixed’, and ‘objective’.

Technologies associées

A simple neural network architecture of five layers (Embedding, Conv1D, GlobalMaxPooling and two Fully-Connected)

Convolutional Neural Networks (CNNs)

Mention de l'éthique

ND

Finalité communicationnelle

Negative reviews circulated online may cause critical problems for the reputation, competitive power, and survival chances of any business. This in turn has led to some fundamental changes in how businesses approach their customers, gauge satisfaction, provide support and manage risks. Sentiment analysis is formally defined as the task to identify and analyze subjective information of people’s opinions in social media sources (Pham and Le, 2016; Yang et al., 2017). This field of study has recently attracted a lot of attention due to its implications for businesses and governments.

Résumé

This paper describes a language-independent model for multi-class sentiment analysis using a simple neural network architecture of five layers (Embedding, Conv1D, GlobalMaxPooling and two Fully-Connected). The advantage of the proposed model is that it does not rely on language-specific features such as ontologies, dictionaries, or morphological or syntactic pre-processing. Equally important, our system does not use pre-trained word2vec embeddings which can be costly to obtain and train for some languages. In this research, we also demonstrate that oversampling can be an effective approach for correcting class imbalance in the data. We evaluate our methods on three publicly available datasets for English, German and Arabic, and the results show that our system’s performance is comparable to, or even better than, the state of the art for these datasets. We make our source-code publicly available.