TEDtalk-en-ja Dataset Overview

Dataset Summary

This dataset consists of Japanese‑English bilingual pairs extracted from MTTT (Multitarget TED Talks), a multi‑target bilingual text collection based on TED Talks. The source is WIT³, and the data have also been used in the IWSLT machine translation evaluation activities.

Dataset Information

• Languages: English (en) and Japanese (ja)
• License: CC BY‑NC‑ND 4.0
• Task Category: Translation

Features

• Translation:
  • en: string type
  • ja: string type

Data Splits

• Training Set:
  • File Size: 35279668 bytes
  • Number of Samples: 158535

Download and Size

• Download Size: 20322391 bytes
• Dataset Size: 35279668 bytes

Configuration

• Default Config:
  • Data Files:
    • Split: training
    • Path: data/train-*

Usage

from datasets import load_dataset
dataset = load_dataset("Hoshikuzu/TEDtalk-en-ja")

If loading takes too long, streaming can be used:

from datasets import load_dataset
dataset = load_dataset("Hoshikuzu/TEDtalk-en-ja", streaming=True)

Data Example

{
  "en": "(Applause) David Gallo: This is Bill Lange. Im Dave Gallo. ",
  "ja": "(拍手)、デイビッド：彼はビル・ラング、私はデイブ・ガロです。"
}

Data Splits

Only a train split is provided.

License Information

The dataset is released under CC BY‑NC‑ND 4.0. TED states this license on its website and requires acknowledgment of TED's copyright when using the data.

Citation

@misc{duh18multitarget,
        author = {Kevin Duh},
        title = {The Multitarget TED Talks Task},
        howpublished = {url{http://www.cs.jhu.edu/~kevinduh/a/multitarget-tedtalks/}},
        year = {2018},
}