opc‑fineweb‑code‑corpus

Dataset Overview

• Dataset Name: opc‑fineweb‑code‑corpus
• Source: Fineweb
• Purpose: Used for OpenCoder pre‑training
• Size: 55 B code and math‑related tokens

Features

• url: string
• tag: string
• text: string
• file_path: string
• dump: string
• file_size_in_byte: 64‑bit integer
• line_count: 64‑bit integer

Splits

• train: 100,920,235 samples, total size 254,927,419,643 bytes

Configuration

• config_name: default
• data_files:
  • split: train
  • path: data/train-*

Citation

• Paper: OpenCoder: The Open Cookbook for Top‑Tier Code Large Language Models
• Authors: Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J. Yang, J. H. Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Zhaoxiang Zhang, Jie Fu, Qian Liu, Ge Zhang, Zili Wang, Yuan Qi, Yinghui Xu, Wei Chu
• Year: 2024
• URL: https://arxiv.org/pdf/2411.04905