← Back to Tools-Radar

ArtEmis: Affective Language for Visual Art

Categories: Research, Image Generation, Text & Writing  |  Pricing: Free  |  Official Website ↗

ArtEmis is a dataset and machine learning models for understanding the emotional effect of visual art and explaining it in language.

ArtEmis introduces a large-scale dataset and accompanying machine learning models designed to provide a detailed understanding of the interplay between visual content, its emotional effect, and linguistic explanations for these emotions. Unlike most existing computer vision annotation datasets, ArtEmis focuses on the affective experience triggered by visual artworks. Annotators indicate the dominant emotion they feel for an image and provide a grounded verbal explanation for their choice. The dataset, termed ArtEmis, contains 455K emotion attributions and explanations from humans across 80K artworks from WikiArt. Building on this data, the project trains and demonstrates a series of captioning systems capable of expressing and explaining emotions from visual stimuli. These systems produce captions that often reflect the semantic and abstract content of images, going beyond capabilities of systems trained on other datasets. The neural speaker can also be grounded with a desired emotion to adapt explanations accordingly.

Key Features

• Large-scale dataset of 455K emotion attributions and explanations for 80K artworks
• Machine learning models for affective image captioning
• Neural speaker capable of expressing and explaining emotions from visual stimuli
• Ability to ground neural speaker with desired emotions for tailored explanations
• Tools for dataset evaluation and speaker analysis

Pros

• Provides a unique dataset focused on emotional responses to visual art
• Generates explanations that go beyond objective content, including abstract concepts and metaphors
• Allows grounding of explanations with specific emotions
• Code and dataset are publicly available for research
• Offers a new direction in affective captioning research

Cons

• Neural generations can make basic object-recognition mistakes
• Generated explanations can sometimes be generic or 'vanilla'-like
• Generations are not yet as soulful and diverse as human-made counterparts
• Primarily a research project, not a commercial product
• Requires filling a form to download the dataset

Use Cases

• Training AI models to understand and explain emotions in art
• Developing advanced image captioning systems
• Researching the interplay between visual content, emotion, and language
• Exploring new methods for human-computer interaction in creative fields
• Creating emotionally intelligent AI assistants for art analysis

Best For

• AI researchers
• Computer vision scientists
• Natural language processing researchers
• Academics studying art and emotion
• Developers interested in affective computing

Platforms: Web

Watch demo on YouTube ↗

View full ArtEmis: Affective Language for Visual Art profile on Tools-Radar | Browse Research tools | Alternatives to ArtEmis: Affective Language for Visual Art

Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com