IndoPref : A Multi-Domain Pairwise Preference Dataset for Indonesian

Vanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, Genta Indra Winata · 2025

Read IndoPref : A Multi-Domain Pairwise Preference Dataset for Indonesian by Vanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, Genta Indra Winata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-22159.txt.

Description

Vanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, and Genta Indra Winata build the first preference dataset for Indonesian that is written by humans from scratch. More than two hundred million people speak the language, yet research on which model answers people prefer relies mostly on translated English material, which loses local flavor. Their dataset, IndoPref, is meant to judge how natural and good model-written Indonesian feels.

The set holds 522 prompts and 4,099 pairwise preferences, gathered by comparing answers from five instruction-tuned language models. Everything was written and judged natively in Indonesian, with strong agreement among annotators. The prompts span ten categories, from everyday requests to specialized topics. That spread lets developers see fine-grained strengths and weaknesses of each model in this language rather than a single overall score.

Catalog record

CREATORVanessa Rebecca Wiyono, David Anugraha, Ayu Purwarianti, Genta Indra Winata
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-22159

Browse the full library · About the Tower of Babel