MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning

Haochen You, Baojing Liu · 2025

Read MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning by Haochen You, Baojing Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06219.txt.

Introduction

Learning new classes on a graph without keeping old examples is the setting of MCIGLE, a 2025 framework from Haochen You and Baojing Liu. Multimodal graph data keep growing, and incremental learners face catastrophic forgetting, distribution bias, and tight memory. MCIGLE extracts and aligns features across data types, then retains knowledge through concatenated recursive least squares.

Multi-channel processing balances fresh accuracy against preserved memory. Tests on public datasets back up how well it works, by the authors' report. The framework targets systems, from recommendation to monitoring, where graph classes arrive over time and storing the past is not an option.

Catalog record

CREATORHaochen You, Baojing Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE12 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06219

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning Haochen You, Baojing Liu Abstract Exemplar-free class-incremental learning enables models to learn new classes over time without storing data from old ones. As multimodal graph-structured data becomes increasingly prevalent, existing methods struggle with challenges like catastrophic forgetting, distribution bias, memory limits, and weak generalization. We propose MCIGLE , a novel framework that addresses these issues by extracting and aligning multimodal graph features and applying Concatenated Recursive Least Squares for effective knowledge retention. Through multi-channel processing, MCIGLE balances accuracy and memory preservation. Experiments on public datasets validate its effectiveness and generalizability. 1 Introduction Class-Incremental Learning (CIL) addresses the challenge of incrementally learning new classes without forgetting previously learned ones, especially without access to historical data [ 13 , 25 ] . It has broad applications, including dynamic classification, continual learning, and privacy-sensitive scenarios [ 12 ] . Among CIL variants, exemplar-free learning is particularly challenging due to its strict constraint of not retaining any old class samples [ 15 ] . Without historical samples, exemplar-free CIL cannot rely on replay mechanisms and becomes more susceptible to catastrophic forgetting [ 14 ] . Parameter updates for new classes may interfere with learned representations, degrading performance on old classes [ 20 ] . As a result, the model must rely on parameter regularization, knowledge distillation, or generated features-requiring strong generalization. With the growth of large-scale multimodal data, CIL has increasingly focused on learning from diverse modalities (e.g., text, images, audio) while retaining prior knowledge [ 6 , 16 ] . In many real-world scenarios, such data form structured relationships-like user interactions or item co-occurrence—that are naturally represented as graphs [ 22 ] . Graphs support dynamic topology, making them well-suited for continual learning. However, integrating graph-structured multimodal data remains underexplored. Existing approaches often rely on fixed GNN architectures, which lack scalability and adaptability, exacerbating forgetting in evolving tasks [ 2 , 10 ] . To address the existing challenges, we integrate multiple key modules in this paper and propose a novel framework. Our main contributions are given: (i) We propose a novel multimodal exemplar-free class-incremental graph learning framework, MCIGLE , illustrated in Figure 1 . (ii) To capture and align multimodal graph-structured features, we design a Multimodal Feature Processing Module and a Periodic Feature Extraction Module. Additionally, a Non-Forgetting Mainstream Module, based on Concatenated Recursive Least Squares, and a Residual Fitting Enhancement Module help mitigate forgetting and compensate for historical knowledge. (iii) Extensive comparative and ablation experiments on four public datasets validate the effectiveness, robustness, and generalization capability of MCIGLE. 2 Multimodal Class-Incremental Learning 2.1 Multimodal Feature Processing Module This module processes sequential multimodal graph structures with textual and visual modalities [ 3 ] , aligning information across modalities to support node classification in evolving graphs under continual learning. Figure 1 : Framework of MCIGLE . Each modality m\in\{v,t\} is represented as a graph G_{m}=(V_{m},E) , where V_{m}=\{X^{m}\mid X^{m}\in I_{m}\} and E=\{(i,j)\} . For the l -th layer, the feature update for node u in modality m is: h_{u,m}^{(l)}=\sigma\left(W_{m}^{(l)}\cdot\Phi_{l,m}\left(\left\{h_{v,m}^{(l-1)}:v\in N(u)\right\}\right)\right), (1) Here, \Phi_{l,m} aggregates neighbor features using correlation-based weighting: \Phi_{l,m}\left(\left\{h_{v,m}^{(l-1)}\right\}\right)=\operatorname{Agg}\left(\left\{e_{u,v,m}^{(l)}h_{v,m}^{(l-1)}\right\}\right), (2) where e_{u,v,m}^{(l)} is the correlation coefficient. We align and fuse features via optimal transport. Specifically, visual features h_{u,v} are projected to the textual feature space using the transport plan P^{*} : h_{u}^{\prime}=h_{u,v\rightarrow t}\|h_{u,t},\quad\text{where }h_{u,v\rightarrow t}=P^{*}h_{u,v}, (3) and P^{*}=\underset{\mathbf{P}\in\mathcal{P}}{\arg\min}\sum_{i,i^{\prime},j,j^{\prime}}\mathbf{P}_{ij}\left(\lambda_{1}\mathbf{C}(x_{i},y_{j})+(1-\lambda_{1})\mathbf{L}(x_{i},y_{j},x_{i^{\prime}},y_{j^{\prime}})\mathbf{P}_{i^{\prime}j^{\prime}}\right)+\epsilon W(\mathbf{P}). (4) where \mathbf{C} is cost matrix, \mathbf{L} the node similarity, and W the regularization term. Finally, node labels are predicted by: \hat{y}_{u}=\operatorname{softmax}\left(\tanh\left(Wh_{u}^{(l)}+b\right)\right), (5) with W and b being trainable parameters. 2.2 Periodic Feature Extraction Module After obtaining node-level embeddings, we flatten them into a global feature matrix and normalize the input. The data is then passed through neural layers based on Fourier analysis [ 7 ] , with each layer defined as: \phi^{(l)}(x)=\left[\cos\left(W_{p}^{(l)}x\right)\left\|\sin\left(W_{p}^{(l)}x\right)\right\|\sigma\left(B_{\bar{p}}^{(l)}+W_{\bar{p}}^{(l)}x\right)\right], where x is the input, \sigma is a nonlinear activation (e.g., GELU or ReLU), and W_{p} , W_{\bar{p}} , B_{\bar{p}} are trainable parameters. The full output is: X_{\text{FAN}}=\phi^{(L)}\circ\cdots\circ\phi^{(1)}(X_{\text{norm}}). (6) We train the network using backpropagation with cross-entropy loss and optimizers such as AdamW. The output X_{\text{FAN}} is passed to a classifier head for final predictions: \hat{Y}=\operatorname{softmax}(X_{\text{FAN}}\cdot W_{FCN}), (7) with W_{FCN} optimized jointly. After training, all parameters are frozen for later incremental phases. Fourier-based layers effectively capture periodic patterns with fewer parameters than CNNs or MLPs, improving efficiency and representation quality. This enhances inputs for both the mainstream and compensation stages. 2.3 Non-Forgetting Mainstream Module This module addresses class-incremental learning via analytical linear mapping using Concatenated Recursive Least Squares (C-RLS), enabling recursive weight updates without storing historical data and achieving performance comparable to joint training. Let X_{M,k} and Y_{train}^{k} denote the input features and target labels at phase k , respectively. The optimization objective is: \underset{W_{M}^{(k)}}{\arg\min}\left\|Y_{train}^{k}-X_{M,k}W_{M}^{(k)}\right\|_{F}^{2}+\gamma\left\|W_{M}^{(k)}\right\|_{F}^{2}, (8) where \gamma is a regularization parameter. The closed-form ridge regression solution is omitted for brevity. To support efficient updates, we define an autocorrelation matrix \Phi_{k} and a cross-correlation matrix Z_{k} , both updated recursively with forgetting factor \beta . The inverse \Phi_{k}^{-1} is maintained using the Sherman–Morrison formula without direct matrix inversion [ 8 ] . The final recursive weight update is given by: \hat{W}_{M}^{(k)}=\hat{W}_{M}^{(k-1)}+k_{k}\left(Y_{train}^{k}-X_{M,k}\hat{W}_{M}^{(k-1)}\right), (9) where the gain coefficient is: k_{k}=\frac{\Phi_{k-1}^{-1}X_{M,k}^{T}}{\beta+X_{M,k}\Phi_{k-1}^{-1}X_{M,k}^{T}}. (10) 2.4 Residual Fitting Enhancement Module Due to the linear nature of the Non-Forgetting Mainstream Module, it may underfit complex samples [ 23 ] . To address this, we introduce a Residual Fitting Enhancement Module that captures the residual information missed by the main stream via independent feature projection and nonlinear transformations. After training, the residual matrix is defined as: \widetilde{Y}_{k}=\left[0_{N_{0:k-1}\times d_{y,k-1}},Y_{k}^{\mathrm{train}}\right]-X_{M,k}W_{M}^{(k)}, (11) representing prediction errors, where the zero matrix enforces phase-wise label exclusivity [ 27 ] . The compensation stream generates embeddings via: X_{C,k}=\sigma_{C}\left(B\left(\operatorname{flat}\left(\operatorname{CNN}\left(X_{k}^{\text{train}},W_{\mathrm{CNN}}\right)\right)\right)\right), (12) with \sigma_{C} (e.g., Tanh or Mish) different from the main stream’s activation to capture complementary features [ 9 ] . To avoid error propagation from earlier phases, only the current-phase component of \widetilde{Y}_{k} is retained: \left\{\widetilde{Y}_{k}\right\}_{\mathrm{PLC}}=\left[0_{N_{0:k-1}\times d_{y,k-1}},\left(\widetilde{Y}_{k}\right)_{\mathrm{new}}\right]. (13) The compensation weights are updated using C-RLS: W_{C}^{(k)}=W_{C}^{(k-1)^{\prime}}+R_{C,k}X_{C,k}^{T}\left(\left\{\widetilde{Y}_{k}\right\}_{\mathrm{PLC}}-X_{C,k}W_{C}^{(k-1)^{\prime}}\right), where R_{C,k} is the inverse correlation matrix of the compensation stream. The final prediction combines both streams: \hat{Y}_{k}^{(\mathrm{all})}=\lambda_{2}X_{M,k}W_{M}^{(k)}+(1-\lambda_{2})X_{C,k}W_{C}^{(k)}, (14) with \lambda_{2} controlling the compensation contribution. 3 Experiments We evaluate MCIGLE on four public datasets- COCO-QA 1 1 1 https://www.cs.toronto.edu/~mren/research/imageqa/data/cocoqa/ , VoxCeleb 2 2 2 https://www.robots.ox.ac.uk/~vgg/data/voxceleb/ , SNLI-VE 3 3 3 https://github.com/necla-ml/SNLI-VE , and AudioSet-MI 4 4 4 https://research.google.com/audioset/index.html —and compare it with state-of-the-art methods including CavRL [ 26 ] , TAM-CL [ 4 ] , GMM [ 5 ] , DS-AL [ 27 ] , and MedCoSS [ 24 ] . As shown in Table 1 , MCIGLE consistently outperforms baselines across most metrics and datasets, achieving high accuracy and low forgetting. The only exception is the Acc metric on AudioSet-MI, where CavRL performs better, which is expected given its specialization in audio-visual incremental learning. Figures 2(a) and 2(b) further illustrate the accuracy trends over cumulative classes on AudioSet-MI and VoxCeleb. Table 1 : Overall Performance of our model MCIGLE and the baselines . Method COCO-QA VoxCeleb Acc \uparrow F \downarrow BwF \downarrow \mathbb{T}_{\mathbf{F}}(j\leftarrow i) \downarrow Acc \uparrow F \downarrow BwF \downarrow \mathbb{T}_{\mathbf{F}}(j\leftarrow i) \downarrow CavRL 0.739 0.175 0.297 0.250 0.915 0.237 0.249 0.273 TAM-CL 0.741 0.199 0.363 0.267 0.860 0.213 0.291 0.204 GMM 0.698 0.154 0.413 0.312 0.832 0.305 0.314 0.357 DS-AL 0.749

Read the complete text