JUHE API Marketplace
DATASET
Open Source Community

MC-Bench

MC‑Bench is a multi‑context visual grounding benchmark created by the Cognitive Computing and Learning Lab at Zhejiang University. It evaluates multimodal large language models (MLLMs) on visual grounding tasks across multiple images. The benchmark comprises 2,000 high‑quality, manually annotated samples covering diverse domains and subjects. Each sample includes a pair of images, instance‑level annotations, and a textual prompt in one of three styles: referring, comparison, or reasoning. The dataset was built by gathering varied images from multiple sources and meticulously annotating them. MC‑Bench targets instance‑level visual grounding in multi‑image scenarios, aiming to address the limitations of current MLLMs in handling complex textual descriptions and cross‑image context understanding.

Updated 10/16/2024
arXiv

Description

MC‑Bench: A Benchmark for Multi‑Context Visual Grounding in the Era of MLLMs

Overview

  • Name: MC‑Bench
  • Type: Multi‑context visual grounding benchmark
  • Domain: Vision‑Language Understanding
  • Goal: Evaluate multimodal large language models (MLLMs) on instance‑level visual grounding across multiple images

Details

  • Samples: 2,000 high‑quality manually annotated instances
  • Composition: Image pairs with instance annotations and associated textual prompts
  • Prompt Types:
    • Referring
    • Comparison
    • Reasoning
  • Covered Skills: Over 10 practical abilities, including multi‑hop reasoning, commonsense reasoning, multi‑view reasoning, temporal understanding, etc.

Download

Related Paper

Authors

  • Authors:
    • Yunqiu Xu
    • Linchao Zhu
    • Yi Yang
  • Institution: ReLER Lab, CCAI, Zhejiang University
  • Status: Submitted

AI studio

Generate PPTs instantly with Nano Banana Pro.

Generate PPT Now

Access Dataset

Login to Access

Please login to view download links and access full dataset details.

Topics

Visual Grounding
Multimodal Language Models

Source

Organization: arXiv

Created: 10/16/2024

Power Your Data Analysis with Premium AI Models

Supporting GPT-5, Claude-4, DeepSeek v3, Gemini and more.

Enjoy a free trial and save 20%+ compared to official pricing.