Intentonomy Dataset Overview

Dataset Introduction

Dataset Download

• Name: Intentonomy
• Content: 14 K images manually labeled with 28 intent categories, organized in a hierarchical structure by psychology experts.
• Download: See DATA.md.

Annotation Method

• Method: “Purpose Game” approach, gathering intent annotations via Amazon Mechanical Turk.
• Details: See Appendix C of the paper.

Research Content

Relationship Between Image Content and Human Intent

• Goal: Explore the subtle link between visual content and intent.
• Findings:
  1. Different intent categories rely on distinct objects and scenes for recognition.
  2. For categories with large intra‑class variation, visual content offers limited performance gains.
  3. Focusing on relevant objects and scene categories positively influences intent recognition.

Intent Recognition Baselines

• Framework: Introduces weakly‑supervised localization and auxiliary label modeling to narrow the gap between human and machine image understanding.
• Implementation: Provides the localization loss in loc_loss.py; download image masks and set MASK_ROOT accordingly.
• Dependencies: Requires cv2 and pycocotools.

Intent Category Sub‑division

• Basis:
  1. Content Dependency: Object‑dependent (O‑classes), Context‑dependent (C‑classes), and Others.
  2. Difficulty: Classified as “Easy”, “Medium”, and “Hard” based on the performance gap between visual models and random chance.
• Details: See Appendix A of the paper.

Baseline Results

Validation Set

Test Set

Validation Sub‑division (by Content Dependency)

Validation Sub‑division (by Difficulty)

Citation

@inproceedings{jia2021intentonomy,
  title={Intentonomy: a Dataset and Study towards Human Intent Understanding},
  author={Jia, Menglin and Wu, Zuxuan and Reiter, Austin and Cardie, Claire and Belongie, Serge and Lim, Ser‑Nam},
  booktitle={CVPR},
  year={2021}
}