WikiBio (wikipedia biography dataset)

Dataset Overview

Dataset Name

WikiBio (wikipedia biography dataset)

Dataset Content

• Contains 728,321 Wikipedia biography articles.
• Each article provides the opening paragraph text and infobox (both tokenized).

Intended Use

For evaluating text generation algorithms, especially in the biography domain.

Data Processing

• Preprocessed using Stanford CoreNLP, including sentence segmentation and tokenization.
• Randomly split into three subsets: training (80%), validation (10%), test (10%).

Dataset Structure

The dataset is divided into three subdirectories: train, valid, test. Each subdirectory contains seven files:

• SET.id: list of Wikipedia article IDs.
• SET.url: list of Wikipedia article URLs.
• SET.box: infobox data.
• SET.nb: number of sentences per article.
• SET.sent: list of sentences.
• SET.title: article title.
• SET.contributors: list of article contributors.

Infobox Data Format

• Each line represents an infobox.
• Infoboxes are encoded as lists, with tab‑separated tokens.
• Each token format: fieldname_position:wordtype.
• Empty fields or unreadable tokens are represented as: fieldname:.

Citation

When using this dataset, cite the following papers:

• Neural Text Generation from Structured Data with Application to the Biography Domain
• Rémi Lebret, David Grangier and Michael Auli, EMNLP 16
• Paper link: http://arxiv.org/abs/1603.07771