Aesthetics and Emotions in Images

Auteur-es

Joshi, Dhiraj; Datta, Ritendra; Fedorovskaya, Elena; Luong, Quang-Tuan; Wang, James Z.; Li, Jia; Luo, Jiebo

Nombre Auteurs

7

Titre

Aesthetics and Emotions in Images

Année de publication

2011

Référence (APA)

Joshi, D., Datta, R., Fedorovskaya, E., Luong, Q.-T., Wang, J. Z., Li, J., & Luo, J. (2011). Aesthetics and Emotions in Images. IEEE Signal Processing Magazine, 28(5), 94‑115. https://doi.org/10.1109/MSP.2011.941851

Mots-clés

Emotion recognition, Photography, Semantics, Data visualization, Painting, Human factors

URL

https://ieeexplore.ieee.org/document/5999579

doi

https://doi.org/10.1109/MSP.2011.941851

Accessibilité de l'article

Open access

Champ

Machine Perception

Type contenu (théorique Applicative méthodologique)

Theoretical

Methodological

Méthode

Here we present plots of features of the data sets [photo resources on the web], in particular the nature of user ratings received in each case

Cas d'usage

ND

Objectifs de l'article

In this tutorial, we define and discuss key aspects of the problem of computational inference of aesthetics and emotion from images.
In this survey, we discuss research that attempts to explain the observed phenomena of aesthetics and emotions that arise from subjective judgments using known tools and knowledge about computer vision, machine learning, art, and photography.

Question(s) de recherche/Hypothèses/conclusion

Research question(s) : Aspects of the problem of computational inference of aesthetics and emotion from images. [...] key computational problems that the research community has been striving to solve and the computational framework required for solving them.

Hypothesis(es) : We strongly believe that computational models of aesthetics and emotions may be able to assist in this decision making and perhaps with time and feedback learn to adapt to expert opinion better
[...] While tutorials are typically written for relatively mature topics, we believe an early tutorial on this active topic will help summarize the existing attempts, conjure up future research directions, and ultimately lead to robust solutions.

Conclusion(s) : We also discuss future directions that researchers can pursue and make a strong case for seriously attempting to solve problems in this research domain.

Cadre théorique/Auteur.es

[aesthetic studies; aesthetics in photography; aesthetics in painting; aesthetics in other visual art forms; psychology of aesthetics; prediction of aesthetics; machine learning]

Emotion prediction (Yanulevskaya et al., 2008)

Detecting and categorizing emotion in art (Yanulevskaya et al., 2008 ; USA Today, 2006)

Understanding beauty, attractiveness (Valentine, 1962 ; Davis et Lazebnik, 2008 ; O'Doherty et al, 2003 ; Scheib, Gangestad, et Thornhill, 1999 ; Swaddle et Cuthill, 1995 ; Zaidel et Cohen, 2005)

Feedback, personalization, and emotions in image retrieval (Wang et He, 2008 ; Fang, Geman, et Boujemaa, 2005 ; Bianchi-Berthouze, 2003)

Feature extraction and image representation for semantics and image understanding (Data et al., 2008 ; Freeman, 2007 ; Axelsson, 2007 ; Datta et al., 2006 ; Peters, 2007 ; Ke, Tang, et Jing, 2006 ; Li et Chen, 2009)

Aesthetics and emotions in artwork characterization (Yanulevskaya et al., 2008 ; Bianchi-Berthouze, 2003 ; Bressan, Cifarelli, et Perronnin, 2008)

Recognizing emotions in images and artwork (Arapakis, Konstas, et Jose, 2009 : Davis et Lazebnik, 2008 ; Eisenthal, Dror, et Ruppin, 2006 ; Machajdik et Hanbury, 2010 ; Ramanathan et al., 2009 ; Valenti, Jaimes, et Sebe, 2010)

Concepts clés

Emotional response

Sentiment analysis

Données collectées (type source)

We performed a preliminary analysis of the above data sources [Web photo resources : Photo.net, DPChallenge, Terragalleria, ALIPR] to compare and contrast the different rating patterns. A collection of images was formed, drawing at random, to create real-world data sets (to be available at http://riemann.ist.psu.edu/).

Définition des émotions

Brief description of emotions and attractiveness

Ampleur expérimentation (volume de comptes)

14,839 images from Photo.net
16,509 images from DPChallenge
14,449 images from Terragalleria
13,010 emotion-tagged images from ALIPR

Technologies associées

Computational techniques

Machine learning

Mention de l'éthique

ND

Finalité communicationnelle

Despite the challenges, various research attempts have been made and are increasingly being made to address basic understanding and solve various subproblems under the umbrella of aesthetics, mood, and emotion inference in pictures. What motivates the multidisciplinary community to make such attempts is the fact that there is much to be gained from systems that can indeed reliably infer, at least for a section of the population, what the perceptual, cognitive, aesthetic, and emotional response to a photograph or a visual artwork will be. The potential beneficiaries of this research include general consumers, media management vendors, photographers, and people who work with art. Good shots or photo opportunities may be recommended to consumers; media personnel can be assisted with good images for illustration while interior and healthcare designers can be helped with more appropriate visual design items.

We hope that this tutorial will significantly increase the visibility of this research area and foster dialogue and collaboration among artists, photographers, and researchers in signal processing, computer vision, pattern recognition, and psychology.

Résumé

In this tutorial, we define and discuss key aspects of the problem of computational inference of aesthetics and emotion from images. We begin with a background discussion on philosophy, photography, paintings, visual arts, and psychology. This is followed by introduction of a set of key computational problems that the research community has been striving to solve and the computational framework required for solving them. We also describe data sets available for performing assessment and outline several real-world applications where research in this domain can be employed. A significant number of papers that have attempted to solve problems in aesthetics and emotion inference are surveyed in this tutorial. We also discuss future directions that researchers can pursue and make a strong case for seriously attempting to solve problems in this research domain.