WAON: Large-Scale and High-Quality Japanese Image-Text Pair Dataset for Vision-Language Models

2025

Read WAON: Large-Scale and High-Quality Japanese Image-Text Pair Dataset for Vision-Language Models (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22276.txt.

Introduction

Issa Sugiura leads a six-author team asking whether global training data is enough for one culture. Vision-language models keep improving at multicultural tasks. But native fluency in a single culture's objects and customs may need native material. Japan is the test case here.

The team built WAON, the largest public Japanese image-text dataset. It was drawn from native Japanese web pages in Common Crawl. It holds roughly one hundred fifty-five million examples. A hand-curated benchmark, WAON-Bench, spans three hundred seventy-four cultural classes. Fine-tuning comparisons used several Japanese datasets. Models trained on WAON did better on Japanese cultural benchmarks. Global pretraining alone did not match them. The dataset is offered as evidence and as infrastructure for culture-aware adaptation.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22276

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WAON: Large-Scale and High-Quality Japanese Image-Text Pair Dataset for Vision-Language Models Abstract Large-scale and high-quality image-text pair datasets play an important role in developing high-performing Vision-Language Models (VLMs). In this work, we introduce WAON , a large-scale and high-quality Japanese image-text pair dataset containing approximately 155 million examples, collected from Common Crawl. Our dataset construction pipeline employs various techniques, including filtering and deduplication, which have been shown to be effective in previous studies. To evaluate its effectiveness, we also construct WAON-Bench , a manually curated benchmark for Japanese cultural image classification, consisting of 374 classes. To assess the effectiveness of our dataset, we conduct experiments using both WAON and the Japanese subset of ReLAION, one of the most widely used vision-language datasets. We fine-tune SigLIP2, a strong multilingual model, on both datasets. The results demonstrate that WAON enhances model performance on WAON-Bench more efficiently than ReLAION and achieves higher accuracy across all evaluated benchmarks. Furthermore, the model fine-tuned on WAON achieves state-of-the-art performance on several Japanese cultural benchmarks. We release our dataset, model, and code at https://speed1313.github.io/WAON . Keywords: Vision-Language Models, Image-Text Pair Dataset, Japanese WAON: Large-Scale and High-Quality Japanese Image-Text Pair Dataset for Vision-Language Models Issa Sugiura ♣,♢ , Shuhei Kurita ‡,♢ , Yusuke Oda ♢ , Daisuke Kawahara ♡,♢ , Yasuo Okabe ♣ , Naoaki Okazaki †,♢ ♣ Kyoto University, ♢ NII LLMC, ‡ NII, ♡ Waseda University, † Institute of Science Tokyo Abstract content 1. Introduction The scale and quality of image-text pair datasets, which consist of images and their corresponding textual descriptions, are critical factors in developing high-performing Vision-Language Models (VLMs) ( Schuhmann et al. 2022 ; Cherti et al. 2023 ; Gadre et al. 2023 ) . However, most existing dataset construction efforts have focused on English and Chinese, and large-scale, high-quality datasets for other languages remain scarce. In this study, we aim to construct a large-scale and high-quality dataset for Japanese language and cultural understanding. Although several Japanese image-text pair datasets have been developed ( Schuhmann et al. 2022 ; Sugiura et al. 2025 ; Sasagawa et al. 2025 ) , they still face notable limitations. One of the most widely used open resources is the Japanese subset of ReLAION ( Schuhmann et al. 2022 ) , which contains around 120 million Japanese image-text pairs. However, this dataset has several limitations: its snapshot is outdated, with 20-30% of image URLs no longer accessible as of June 2025. Furthermore, its filtering relies on mCLIP ( Chen et al. 2023 ) , a model with considerably lower performance than recent state-of-the-art CLIP variants, leading to reduced data quality. Another line of work translates the captions from the English subset of ReLAION into Japanese using LLMs, but translation errors often introduce noise and mixed-language captions ( Sugiura et al. 2025 ) . Moreover, since the English subset of ReLAION primarily consists of Western cultural image-text pairs, such data contribute little to improving performance on Japanese cultural tasks. Figure 1: WAON-Bench Top-1 accuracy over training. Fine-tuning google/siglip2-base-patch16-256 on WAON improves performance on Japanese cultural benchmarks more efficiently than fine-tuning on the ReLAION (Japanese subset). Figure 2: Overview of WAON-Bench. WAON-Bench consists of 8 categories with a total of 374 classes, each containing 5 images, resulting in a total of 1,870 images. Dataset # Examples Source WIT (ja subset) ( Srinivasan et al. 2021 ) 1M Wikipedia ReLAION (ja subset) ( Schuhmann et al. 2022 ) 120M Common Crawl ReLAION-ja ( Sugiura et al. 2025 ) 2.1B Common Crawl (translation) llm-jp-japanese-image-text-pairs ( Sasagawa et al. 2025 ) 6.6M Common Crawl WAON (Ours) 155M Common Crawl Table 1: Comparison of Japanese image-text pair datasets. To address these issues, we present WAON (Web-scale image text Aligned Open Nihongo), a large-scale, high-quality dataset of 155M Japanese image-text pairs sourced from Common Crawl. Our dataset construction pipeline leverages recent advances in data curation and filtering, including deduplication and CLIP score based filtering ( Schuhmann et al. 2022 ) using a strong multilingual model ( Tschannen et al. 2025 ) . WAON is designed to efficiently improve model performance in Japanese, particularly on Japanese cultural content. Since the proposed dataset construction pipeline uses Japanese only for the language identification step, it can be readily adapted to other languages. To evaluate the effectiveness of WAON, we also introduce WAON-Bench , a manually curated benchmark for Japanese cultural image classification. WAON-Bench consists of 1,870 images across 374 classes related to Japanese culture (e.g., Shiba Inu, Jōmon pottery). We fine-tune SigLIP2 ( Tschannen et al. 2025 ) on WAON and the Japanese subset of ReLAION, and demonstrate that WAON improves model performance more efficiently than ReLAION on WAON-Bench and consistently outperforms it across all tasks in the Japanese cultural benchmark. The fine-tuned model on WAON achieves the best performance on Japanese cultural benchmarks compared to existing models. To foster the development of VLMs, we release our dataset, model, and code. 2. Related Work Image-text pair datasets have played a central role in advancing Vision-Language Models (VLMs). Early efforts relied on small-scale datasets annotated by human experts ( Deng et al. 2009 ; Lin et al. 2015 ) . With the introduction of CLIP ( Radford et al. 2021 ) , which leveraged web-scale data, the construction of large-scale datasets from sources such as Common Crawl became a major research focus. Among these, LAION-5B ( Schuhmann et al. 2022 ) has been the most influential, serving as the first public billion-scale dataset that has powered a wide range of models, including OpenCLIP ( Cherti et al. 2023 ) , diffusion models ( Rombach et al. 2022 ) , and Visual Language Models ( Lin et al. 2024 ) . Alongside scaling datasets, recent research has prioritized data quality. DataComp ( Gadre et al. 2023 ) introduced a systematic benchmark for evaluating dataset curation methods. Data Filtering Network (DFN) ( Fang et al. 2024 ) developed a data filtering model and provides a dataset curated with this model that enables training high-performance CLIP models. MetaCLIP ( Xu et al. 2024 ) reverse-engineered OpenAI’s CLIP data curation strategy and proposed a metadata-based approach to curate and balance datasets, demonstrating that this approach yields higher-quality data than the original CLIP dataset. No-Filter ( Pouget et al. 2024 ) experimentally demonstrated that pre-training with global data before fine-tuning on English content could improve cultural understanding without sacrificing performance on Western-centric benchmarks. For Japanese, several datasets exist as shown in Table 1 . WIT ( Srinivasan et al. 2021 ) is constructed from Wikipedia articles and their corresponding image captions, but its Japanese subset contains only 1M image-text pairs, which is too small to train VLMs. The Japanese subset of ReLAION ( Schuhmann et al. 2022 ) is large-scale, containing about 120M examples. However, it has several limitations, such as insufficient deduplication and quality filtering performed using relatively low-performing multilingual CLIP models trained on smaller datasets compared to the latest state-of-the-art models ( Chen et al. 2023 ) . ReLAION-ja ( Sugiura et al. 2025 ) is a large-scale dataset by translating ReLAION English captions to Japanese with Gemma 2 ( Gemma Team 2024 ) , but the dataset introduced translation errors, and cultural deviations derived from the English data. As shown in

Read the complete text