Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia
Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasut · 2025
Read Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia by Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasut (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07920.txt.
Introduction
Samuel Cahyawijaya and a large team of co-authors build in 2025 SEA-VL, an open vision language data set for Southeast Asia. That region holds rich tongue and culture mix, yet image text research scarce covers it. Models then miss the culture cues of the region's daily life.
Three build paths are set side by side: crowd work by folk in the region, web crawl, and made images. Crawl shots hit near eighty five per cent culture fit at low cost and time. Made shots, by contrast, oft missed true culture marks. The set aims to pull more of the region's tongues into sight text work.
Catalog record
| CREATOR | Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasut |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-07920
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia Samuel Cahyawijaya, Holy Lovenia, Joel Ruben Antony Moniz, Tack Hwa Wong, Mohammad Rifqi Farhansyah, Thant Thiri Maung, Frederikus Hudi, David Anugraha, Muhammad Ravi Shulthan Habibi, Muhammad Reza Qorib, Amit Agarwal, Joseph Marvin Imperial, Hitesh Laxmichand Patel, Vicky Feliren, Bahrul Ilmi Nasution, Manuel Antonio Rufino, Genta Indra Winata, Rian Adam Rajagede, Carlos Rafael Catalan, Mohamed Fazli Imam, Priyaranjan Pattnayak, Salsabila Zahirah Pranida, Kevin Pratama, Yeshil Bangera, Adisai Na-Thalang, Patricia Nicole Monderin, Yueqi Song, Christian Simon, Lynnette Hui Xian Ng, Richardy Lobo’ Sapan, Taki Hasan Rafi, Bin Wang, Supryadi, Kanyakorn Veerakanjana, Piyalitt Ittichaiwong, Matthew Theodore Roque, Karissa Vincentio, Takdanai Kreangphet, Phakphum Artkaew, Kadek Hendrawan Palgunadi, Yanzhi Yu, Rochana Prih Hastuti, William Nixon, Mithil Bangera, Adrian Xuan Wei Lim, Aye Hninn Khine, Hanif Muhammad Zhafran, Teddy Ferdinan, Audra Aurora Izzani, Ayushman Singh, Evan, Jauza Akbar Krito, Michael Anugraha, Fenal Ashokbhai Ilasariya, Haochen Li, John Amadeo Daniswara, Filbert Aurelian Tjiaranata, Eryawan Presma Yulianrifat, Can Udomcharoenchaikit, Fadil Risdian Ansori, Mahardika Krisna Ihsani, Giang Nguyen, Anab Maulana Barik, Dan John Velasco, Rifo Ahmad Genadi, Saptarshi Saha, Chengwei Wei, Isaiah Flores, Kenneth Ko Han Chen, Anjela Gail Santos, Wan Shen Lim, Kaung Si Phyo, Tim Santos, Meisyarah Dwiastuti, Jiayun Luo, Jan Christian Blaise Cruz, Ming Shan Hee, Ikhlasul Akmal Hanif, M.Alif Al Hakim, Muhammad Rizky Sya’ban, Kun Kerdthaisong, Lester James V. Miranda, Fajri Koto, Tirana Noor Fatyanosa, Alham Fikri Aji, Jostin Jerico Rosal, Jun Kevin, Robert Wijaya, Onno P. Kampman, Ruochen Zhang, Börje F. Karlsson, Peerat Limkonchotiwat, Cohere SEACrowd IndoNLP Mila - Quebec AI Institute Polytechnique Montreal Independent Bandung Institute of Technology Ton Duc Thang University Nara Institute of Science and Technology Works Applications University of Toronto University of Indonesia National University of Singapore Oracle University of Bath National University Philippines Monash University, Indonesia The University of Manchester Samsung R&D Institute Philippines Capital One Universitas Islam Indonesia MBZUAI Meta University of New Haven SCB 10X Carnegie Mellon University Sony Group Corporation Hanyang University Institute for Infocomm Research, Singapore Tianjin University Faculty of Medicine Siriraj Hospital, Mahidol University Binus University Srinakharinwirot University New York University Institut Teknologi Sepuluh Nopember Macau University of Science and Technology Universitas Gadjah Mada King Mongkut’s University of Technology Thonburi Wrocław Tech University of Illiinois, Urbana-Champaign Vidyasirimedhi Institute of Science and Technology Auburn University Indian Statistical Institute, Kolkata Ateneo de Manila University Singapore Polytechnic University of the Philippines Graphcore Dataxet:Sonar Singapore University of Technology and Design Thammasat University Allen AI Brawijaya University Seoul National University of Science and Technology Universitas Pelita Harapan MOH Office for Healthcare Transformation Brown University Beijing Academy of Artificial Intelligence (BAAI) AI Singapore Chulalongkorn University Abstract Southeast Asia (SEA) is a region of extraordinary linguistic and cultural diversity, yet it remains significantly underrepresented in vision-language (VL) research. This often results in artificial intelligence (AI) models that fail to capture SEA cultural nuances. To fill this gap, we present SEA-VL , an open-source initiative dedicated to developing high-quality, culturally relevant data for SEA languages. By involving contributors from SEA countries, SEA-VL aims to ensure better cultural relevance and diversity, fostering greater inclusivity of underrepresented languages in VL research. Beyond crowdsourcing, our initiative goes one step further in the exploration of the automatic collection of culturally relevant images through crawling and image generation. First, we find that image crawling achieves approximately ∼ \sim 85% cultural relevance while being more cost- and time-efficient than crowdsourcing. Second, despite the substantial progress in generative vision models, synthetic images remain unreliable in accurately reflecting SEA cultures. The generated images often fail to reflect the nuanced traditions and cultural contexts of the region. Collectively, we gather 1.28M SEA culturally-relevant images, more than 50 times larger than other existing datasets. Through SEA-VL, we aim to bridge the representation gap in SEA, fostering the development of more inclusive AI systems that authentically represent diverse cultures across SEA. 1 Introduction The rapid evolution of artificial intelligence (AI) and machine learning (ML) has produced increasingly sophisticated models capable of integrating textual and visual information. However, these advancements often disproportionately benefit certain languages and cultures Yong et al. 2023 ; Pham et al. 2023 ; Cahyawijaya et al. 2023b ; Tao et al. 2024 ; Cahyawijaya et al. 2024a ; Myung et al. 2024 ; Li et al. 2024 ; Winata et al. 2024 , leaving underrepresented cultures—particularly those of Southeast Asia (SEA)—largely overlooked Aji et al. 2022b ; Winata et al. 2023 ; Purwarianti et al. 2025 ; Cahyawijaya et al. 2024b ; Urailertprasert et al. 2024 . This disparity creates a significant challenge in developing AI technologies that effectively cater to the diverse cultural contexts of underrepresented regions. Home to over 1,300 languages and rich cultural diversity, SEA is among the world’s most linguistically vibrant regions Enfield 2011 ; Aji et al. 2022a ; Lovenia et al. 2024 . However, the lack of SEA-relevant datasets, particularly in the vision-language (VL) domain Lovenia et al. 2024 , limits AI accessibility and risks cultural irrelevance or bias against SEA populations Winata et al. 2024 ; Urailertprasert et al. 2024 ; Cahyawijaya 2024 . Addressing this disparity by creating datasets that authentically capture SEA’s linguistic and cultural nuances requires large-scale collaborative efforts Bell and Kampman 2021 . Building on crowdsourcing initiatives like NusaCrowd Cahyawijaya et al. 2023a , SEACrowd Lovenia et al. 2024 , and Aya Dataset Singh et al. 2024 , SEA-VL takes a holistic approach to bridging the resource gap for SEA cultural representation in VL research. Unlike existing efforts, which primarily focus on text-based tasks or limited subsets of visual data, SEA-VL aims to develop comprehensive, high-quality VL datasets that reflect SEA’s cultural heritage and linguistic diversity. SEA-VL seeks to address linguistic underrepresentation, trust, and dignity in AI, ensuring technological advancements benefit the diverse communities of SEA. Figure 1: SEA-VL addresses the underrepresentation of SEA languages in vision-language research through a multipronged strategy for collecting culturally relevant images, incorporating image crowdsourcing, crawling, and synthetic generation. SEA-VL 1 1 1 SEA-VL dataset: https://huggingface.co/collections/SEACrowd/sea-vl-multicultural-vl-dataset-for-southeast-asia-67cf223d0c341d4ba2b236e7 . distinguishes itself from other grassroots community-driven initiatives by going beyond relying on manual data collection only, through the participation of local contributors. With the recent popularity of various strong AI models, SEA-VL explores diverse methodologies to collect culturally relevant images in SEA. Specifically, SEA-VL collects culturally relevant image data using three different methods: (1) manual human collection, (2) a validated data filtering and deduplication pipeline on crawled images, and (3) image generation through diffusion models. To ensure that the data collected authentically represent the lived experiences and cultural contexts of the region, an extensive human evaluation by native participants is performed using different image collection methodologies. This evaluation not only enhances the quality and relevance of the datasets, but also provides a better understanding of the feasibility, efficiency, and quality of using AI-based data collection solutions to produce culturally relevant VL datasets, specifically for the SEA region. We also compare manual vs. automatic metadata collection to assess how well AI-based solutions generate valid, relevant metadata, which is beneficial for creating culturally relevant VL datasets. Dataset Name #Images %SEA Images Cultural Coverage † ID TH PH SG MY MM BN KH LA VN TL SEA-VQA Urailertprasert et al. 2024 488 100% Tradition & Art Landmark