M4‑Instruct Dataset Overview

Dataset Details

Dataset Type: M4‑Instruct is a collection of multi‑image data gathered from public datasets or generated via the GPT‑4V API. It aims to train large multimodal models with interleaved multi‑image capabilities, such as LLaVA‑NeXT‑Interleave.

Dataset Date: Collected in April 2024, released in June 2024.

Data Statistics: The release includes multi‑image, multi‑frame (video), and multi‑view (3D) data for M4‑Instruct.

Data Content:

• JSON files: m4_instruct_annotations.json and m4_instruct_video.json
• Images: *.zip
• For dreamsim_split.z01 and dreamsim_split.zip, run zip -s 0 dreamsim_split.zip --out dreamsim.zip

License: Creative Commons Attribution 4.0 International; compliance with OpenAI policy required: https://openai.com/policies/terms-of-use

Contact for Issues:

• fliay@connect.ust.hk
• 1700012927@pku.edu.cn

Intended Use

Primary Intended Use: Research on large multimodal models and chatbots.

Primary Intended Users: Researchers and enthusiasts in computer vision, natural language processing, machine learning, and artificial intelligence.