Less is More: Multimodal Region Representation via Pairwise Inter-view Learning Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY CCS: Computing methodologies Knowledge representation and reasoning
Min Namgung, Yijun Lin, JangHyeon Lee, Yao-Yi Chiang · 2025
Read Less is More: Multimodal Region Representation via Pairwise Inter-view Learning Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY CCS: Computing methodologies Knowledge representation and reasoning by Min Namgung, Yijun Lin, JangHyeon Lee, Yao-Yi Chiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18178.txt.
Introduction
Min Namgung, Yijun Lin, JangHyeon Lee, and Yao-Yi Chiang present a method to learn compact descriptions of geographic regions from many data types, in work from 2025. Region learning helps analysts read complex traits of a place. Past methods use contrast between two data types and keep only what they share.
That shared only view drops details unique to each data type, which can matter for a task. The authors factor the data into shared and unique parts. Extending that split past two data types explodes the number of goals. Their method injects cross modal knowledge to keep the split workable. Paired views of a region teach the model.
Catalog record
| CREATOR | Min Namgung, Yijun Lin, JangHyeon Lee, Yao-Yi Chiang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18178
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Less is More: Multimodal Region Representation via Pairwise Inter-view Learning Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY CCS: Computing methodologies Knowledge representation and reasoning Min Namgung, Yijun Lin, JangHyeon Lee, Yao-Yi Chiang Abstract . With the increasing availability of geospatial datasets, researchers have explored region representation learning (RRL) to analyze complex region characteristics. Recent RRL methods use contrastive learning (CL) to capture shared information between two modalities but often overlook task-relevant unique information specific to each modality. Such modality-specific details can explain region characteristics that shared information alone cannot capture. Bringing information factorization to RRL can address this by factorizing multimodal data into shared and unique information. However, existing factorization approaches focus on two modalities, whereas RRL can benefit from various geospatial data. Extending factorization beyond two modalities is non-trivial because modeling high-order relationships introduces a combinatorial number of learning objectives, increasing model complexity. We introduce C r o ss m o dal K nowledge I njected E mbedding (CooKIE ), an information factorization approach for RRL that captures both shared and unique representations. CooKIE uses a pairwise inter-view learning approach that captures high-order information without modeling high-order dependency, avoiding exhaustive combinations. We evaluate CooKIE on three regression tasks and a land use classification task in New York City and Delhi, India. Results show that CooKIE outperforms existing RRL methods and a factorized RRL model, capturing multimodal information with fewer training parameters and floating-point operations per second (FLOPs). We release the code: https://github.com/MinNamgung/CooKIE. 1. Introduction The global urban population has grown from 52% to 57% over the past decade ( UN 2023 ) , increasing the demand for efficient urban planning. Socioeconomic indicators like population density and land classification support decision-making by providing insights into regional dynamics ( Naess 2001 ; Hahn and Truman 2015 ; Dustmann and Fasani 2016 ) . However, these indicators often rely on costly and infrequent surveys ( Gebru et al. 2017 ; Pissourios 2019 ) . As a scalable alternative, region representation learning (RRL) uses multimodal geospatial data, such as remote sensing and point-of-interest (POI) data, to predict socioeconomic indicators by transforming various geospatial data into region representations ( Jenkins et al. 2019 ; Yao et al. 2018 ; Wang and Li 2017 ; Zhang et al. 2021 ; Liu et al. 2023 ; Li et al. 2022 ; Yan et al. 2024 ; Jin et al. 2024 ; Xiao et al. 2024 ; Zou et al. 2025 ; Mai et al. 2025 ) . Recent RRL methods use contrastive learning (CL) to model relationships between regions and between two datasets (e.g., human mobility and POI data) via intra- and inter-view learning ( Zhang et al. 2023 ; Li et al. 2024 ) . Intra-view learning makes representations distinctive between regions within a single modality and inter-view learning aligns information across multiple datasets (i.e., modalities) for a region by focusing on shared information. However, inter-view CL’s focus on shared information discards modality-specific (i.e., unique) information during data alignment, degrading downstream performance in various domains ( Liang et al. 2023 ) . Since CL-based RRL methods ( Liu et al. 2023 ; Zhang et al. 2023 ; Li et al. 2024 ) follow the same learning paradigm, they also risk overlooking task-relevant unique information. This limits RRL’s ability to fully capture the diverse characteristics of regions, potentially decreasing downstream task performance. FactorCL ( Liang et al. 2023 ) addresses the limitation of inter-view CL discarding unique information by introducing an information factorization approach. Specifically, FactorCL decomposes multimodal data to learn unique representations from each modality and shared information across modalities. While this approach is promising, FactorCL is designed for two modalities, limiting its applicability to RRL that benefits from multiple geospatial datasets. To overcome this, we introduce GFactorCL, a generalized form of FactorCL that handles multiple modalities for effective use in RRL. Figure 1 . Comparison of shared information captured during inter-view learning between CooKIE and the direct extension of FactorCL - GFactorCL - for three modalities. CooKIE captures three shared information ( S_{z_{pa}},S_{z_{ps}},S_{z_{as}} ) without considering conditional mutual information (CMI). GFactorCL captures three conditional pairwise information ( S_{z_{pa|s}},S_{z_{ps|a}},S_{z_{as|p}} ) and one high-order information ( S_{z_{pas}} ). Notably, the combined pairwise embeddings in CooKIE are larger than those in GFactorCL, as each shared information term captures high-order dependencies. Finally, the machine learning (ML)-based predictor utilizes all shared embeddings while handling multiple representations in predicting socioeconomic indicators. GFactorCL can capture both shared and unique information in inter-view CL, making it a promising approach for RRL. However, the explicit factorization of high-order shared information (i.e., the shared information among \geq 3 modalities) in GFactorCL increases computational cost. Approximating these high-order dependencies involves decomposing task-relevant information into a complex chain of conditional mutual information terms. As the number of modalities grows, GFactorCL’s objectives amplify, leading to an increase in training parameters and model complexity. Instead, capturing high-order dependencies does not necessarily require conditional mutual information terms. According to interaction information theory ( McGill 1954 ) , high-order dependencies among three modalities can be expressed without explicitly modeling conditional mutual information terms. Specifically, the shared information between any two modalities consists of both high-order information involving all three modalities and the conditional mutual information between the two given the third. Therefore, we propose CooKIE , 1 1 1 CooKIE refers to C r o ss m o dal K nowledge I njected E mbedding a novel RRL method that applies an information factorization approach to efficiently learn task-relevant information from real-world multimodal geospatial datasets. CooKIE addresses the computational challenge of high-order modeling by fusing modalities via a pairwise inter-view learning approach, which can learn (1) task-relevant unique information within each modality and (2) pairwise shared information between modalities without the need for exhaustive conditional dependencies. As shown in Figure 1 , CooKIE ’s pairwise shared embeddings can capture high-order information multiple times. Even if high-order information appears multiple times, it does not increase computational cost. Moreover, the machine learning model can leverage learned representations to manage redundant information, reducing the risk of negative impact on downstream performance. To show the effectiveness of CooKIE , we use multiple geospatial datasets (e.g., POI data, area-of-interest (AOI) data, satellite imagery, and building footprints) to learn region representations. We evaluate these representations on regression tasks (e.g., population density prediction) and land use classification in NYC, United States, and Delhi, India. The contributions of our paper are as follows: • We bring an information factorization approach to RRL, extending FactorCL to multiple geospatial modalities via generalized FactorCL (GFactorCL). This is the first work to explicitly factorize multimodal geospatial data into shared and unique information across more than two modalities in inter-view CL, enabling effective RRL. • We propose CooKIE , a pairwise inter-view learning approach that reduces complexity in factorized multimodal RRL (i.e., GFactorCL) without explicitly modeling high-order dependencies. This approach allows CooKIE to scale beyond two geospatial data while preserving both shared and unique information. • We evaluate CooKIE on real-world downstream tasks, including population density estimation, crime rate prediction, greenness assessment, and land use classification across heterogeneous geographical regions (NYC, USA, and Delhi, India). Results show that CooKIE outperforms state-of-the-art