COSMMIC: Comment-Sensitive Multimodal Multilingual Indian Corpus for Summarization and Headline Generation

Raghvendra Kumar, S. A. Mohammed Salman, Aryan Sahu, Tridib Nandi, Pragathi Y. P., Sriparna Saha, Jose G. Moreno · 2025

Read COSMMIC: Comment-Sensitive Multimodal Multilingual Indian Corpus for Summarization and Headline Generation by Raghvendra Kumar, S. A. Mohammed Salman, Aryan Sahu, Tridib Nandi, Pragathi Y. P., Sriparna Saha, Jose G. Moreno (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15372.txt.

Introduction

Raghvendra Kumar and six co-authors build a data set for Indian languages in twenty twenty-five. Summaries that use reader comments exist for English and Chinese, but Indian languages lag behind. Their corpus, COSMMIC, is sensitive to comments and works across media and languages. It covers nine major Indian languages.

The corpus holds close to five thousand article-image pairs and about twenty-four thousand reader comments. Reference summaries exist in every language included. The team tested four setups: text alone, text with comments, images, and all three together. They ran top models such as Llama3 and GPT-4 on summary and headline tasks. Readers get a base for research on how public reaction can enrich news summaries.

Catalog record

CREATORRaghvendra Kumar, S. A. Mohammed Salman, Aryan Sahu, Tridib Nandi, Pragathi Y. P., Sriparna Saha, Jose G. Moreno
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15372

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

COSMMIC: Comment-Sensitive Multimodal Multilingual Indian Corpus for Summarization and Headline Generation Raghvendra Kumar, S. A. Mohammed Salman, Aryan Sahu, Tridib Nandi, Pragathi Y. P., Sriparna Saha, Jose G. Moreno Abstract Despite progress in comment-aware multimodal and multilingual summarization for English and Chinese, research in Indian languages remains limited. This study addresses this gap by introducing COSMMIC, a pioneering comment-sensitive multimodal, multilingual dataset featuring nine major Indian languages. COSMMIC comprises 4,959 article-image pairs and 24,484 reader comments, with ground-truth summaries available in all included languages. Our approach enhances summaries by integrating reader insights and feedback. We explore summarization and headline generation across four configurations: (1) using article text alone, (2) incorporating user comments, (3) utilizing images, and (4) combining text, comments, and images. To assess the dataset’s effectiveness, we employ state-of-the-art language models such as LLama3 and GPT-4. We conduct a comprehensive study to evaluate different component combinations, including identifying supportive comments, filtering out noise using a dedicated comment classifier using IndicBERT, and extracting valuable insights from images with a multilingual CLIP-based classifier. This helps determine the most effective configurations for natural language generation (NLG) tasks. Unlike many existing datasets that are either text-only or lack user comments in multimodal settings, COSMMIC uniquely integrates text, images, and user feedback. This holistic approach bridges gaps in Indian language resources, advancing NLP research and fostering inclusivity. The resources are available at: https://github.com/AaryanSahu/COSMMIC . 1 Introduction In today’s digital era, online content seamlessly integrates text and visuals, with user comments enriching discussions and providing diverse perspectives. Reader-Aware Comment-Based Summarization (RACBS) harnesses these interactions to generate more relevant and comprehensive summaries by incorporating user feedback ( Kumar et al. 2023 ) . However, the development of multilingual, comment-based, multimodal datasets for Indian languages remains largely unexplored. For instance, a news article on the Indian general elections 1 1 1 https://en.wikipedia.org/wiki/2024_Indian_general_election often features images alongside a flood of reader comments in multiple languages. While the text delivers factual information and images offer visual context, the comments reflect public sentiment, debates, and interpretations, making them a valuable yet underutilized resource for enhancing summarization. Motivation for the dataset: The challenge lies in analyzing such multifaceted content comprehensively. Current datasets often fail to integrate all these elements—text, images, and comments—particularly in the context of Indian languages. A dataset limited to text or images alone cannot capture the full spectrum of user engagement and content richness. Our dataset COSMMIC addresses this shortcoming by combining article text, images, and user comments in nine major Indian languages: Bengali, Hindi, Gujarati, Marathi, Malayalam, Odia, Tamil, Telugu, and Kannada. This integrated approach offers a more complete view of online content. It is essential for developing advanced NLP models that can process and understand the full scope of information in today’s digital media landscape. Datasets: MMSUM M2DS Varta XLHT INLG MSMO M3LS RAMDS RASG COSMMIC Multilingual ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✗ ✓ Multimodal ✓ ✗ ✗ ✓ ✗ ✓ ✓ ✗ ✗ ✓ Comments ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✓ ✓ Summaries ✓ ✓ ✗ ✗ ✓ ✓ ✓ ✓ ✓ ✓ Headlines ✗ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ # Indian Lang. 8 1 14 8 11 0 8 0 0 9 Table 1: Comparison of Datasets Based on Multilingualism, Multimodality, Comment Inclusion, Presence of Summaries and Headlines, and Coverage of Indian Languages. NLG tasks driven by the dataset presented: Summarization and headline generation are key NLP challenges with applications in news aggregation, content curation, and information retrieval ( Tan et al. 2017 ; Singh et al. 2021 ; Verma et al. 2023a ; Hamborg et al. 2017 ; Khatter and Ahlawat 2022 ; Gaci et al. 2023 ; Mahalakshmi and Fatima 2022 ; Esteva et al. 2021 ) . Summarization condenses text into concise, coherent outputs, while headline generation crafts engaging titles, both leveraging advanced language models ( Zhang et al. 2024 ) and multimodal data ( Krubiński and Pecina 2024b ) for improved accuracy and relevance. Integrating images and user comments enriches these tasks, adding context to summaries and enhancing headline relevance ( Li et al. 2017 ; Gao et al. 2019 ) . Multimodal approaches ( He et al. 2023 ; Qiao et al. 2022 ) combining text, visuals, and user interactions produce more informative outputs. Building on these advancements, we assess the impact of filtered comments and categorized images on NLG tasks. By integrating only informative inputs, our approach enhances coherence and relevance, with evaluations confirming its effectiveness in multilingual and multimodal RACBS. Our contributions enhance context-aware summaries and headlines, as detailed below. (1) We introduce “COSMMIC", an innovative comment-sensitive multimodal, multilingual dataset having article-image pairs with reader comments covering nine major Indian languages: Bengali, Hindi, Gujarati, Marathi, Malayalam, Odia, Tamil, Telugu, and Kannada. This dataset represents a pioneering effort in this field. (2) Human-written ground truth summaries are provided for all multilingual news articles within the dataset. (3) We establish benchmarks for our dataset by evaluating two NLG tasks—headline generation and summarization—using state-of-the-art language models such as GPT-4 and LLama3. (4) An extensive study is conducted to explore various configurations: article text alone, article text with comments, article text with relevant comments, article text with images, article text with images and comments, and article text with images and relevant comments. (5) To advance this task, we propose methods for filtering noisy comments and classifying images in the Hindi dataset. Using IndicBERT ( Kakwani et al. 2020 ) , we develop a classifier to retain only informative comments and filter out noisy comments. Additionally, a threshold-based multilingual CLIP ( Carlsson et al. 2022 ; Reimers and Gurevych 2019 ) classifier categorizes images as supplementary (reinforcing the text) or complementary (providing additional context). To assess linguistic preservation in RACBS, we perform a back-translation study on the Hindi dataset, highlighting quality degradation and the need to retain the original language. 2 Related Works We explore multilingual, multimodal datasets with reader comments for NLG tasks, as Table 1 details their linguistic, modal, and structural differences, excluding English from Indian languages. Multilingual Datasets: Scialom et al. 2020 introduced MLSUM, a large-scale dataset containing 1.5 million article-summary pairs in five languages, emphasizing cross-lingual biases. Hasan et al. 2021 developed XL-Sum, featuring 1 million pairs across 44 languages for abstractive summarization. Ladhak et al. 2020 presented WikiLingua, which includes 18 languages and image-based alignments for cross-lingual summarization. Hewapathirana et al. 2024 introduced M2DS, the first multilingual dataset for multi-document summarization (MDS) using BBC articles. Aralikatte et al. 2023 developed Vārta, a dataset designed for headline generation in 14 Indic languages and English. Kumar et al. 2022 introduced the IndicNLG (abbreviated as INLG in Table 1 ) benchmark for NLG tasks across 11 Indic languages. Readers are also referred to notable works such as Urlana et al. 2023 , Cao et al. 2020 , Varab and Schluter 2021 , Feng et al. 2022 , and Gliwa et al. 2019 . Multimodal Datasets: In the realm of multimodal datasets, Zhu et al. 2018 introduced MSMO, a dataset for multimodal summarization that includes both text and image outputs and features a multimodal attention model. Verma et al. 2023b developed M3LS, the largest multilingual multimodal summarization dataset, comprising over a

Read the complete text