Denotation Graph

Categories: Research, Data Analysis | Pricing: Free | Official Website ↗

A research project defining visual denotations of linguistic expressions and constructing a subsumption hierarchy over them.

The Denotation Graph project proposes using visual denotations of linguistic expressions (the set of images they describe) to define novel denotational similarity metrics. These metrics are shown to be beneficial for tasks requiring semantic inference. The core of the project is the construction of a denotation graph, which is a subsumption hierarchy over linguistic constituents and their visual denotations. This graph is built upon a large corpus of 30,000 images and 150,000 descriptive captions. The project defines the visual denotation of an expression 's' as the set of images that depict what 's' describes. The graph structures these expressions such that a parent expression is more generic than its child, and the visual denotation of the parent subsumes that of the child. The project also computes denotational similarity metrics like pointwise mutual information (PMI) and conditional probabilities based on the graph.

Key Features

Visual denotation definition for linguistic expressions
Construction of a denotation graph (subsumption hierarchy)
Flickr 30k Image Caption Corpus (31,783 images, 158,915 captions)
Computation of denotational similarity metrics (PMI, conditional probabilities)
Application to approximate textual entailment tasks
Downloadable data files for the denotation graph

Pros

Provides novel similarity metrics for semantic inference
Based on a large, crowd-sourced image caption corpus
Offers a structured subsumption hierarchy for linguistic expressions
Data files are available for research and educational use
Demonstrates utility in tasks like approximate textual entailment

Cons

Primarily a research project, not a commercial tool
Requires filling a form to access the dataset
Use of images is subject to Flickr Terms of Use
No direct API for programmatic access
Focuses on specific research tasks rather than broad application

Use Cases

Developing new semantic inference models
Analyzing relationships between language and visual content
Creating datasets for textual entailment research
Studying linguistic subsumption hierarchies
Training and evaluating image captioning systems

Best For

Computational linguistics researchers
Computer vision researchers
Academics studying semantic inference
Educators in NLP and AI fields

Platforms: Web

Watch demo on YouTube ↗

View full Denotation Graph profile on Tools-Radar | Browse Research tools | Alternatives to Denotation Graph

Tools-Radar is a free directory of 10,000+ AI tools — discover, compare, and choose the right AI software for your needs. Visit tools-radar.com