AFT: An Exemplar-Free Class Incremental Learning Method for Environmental Sound Classification
Xinyi Chen, Xi Chen, Zhenyu Weng, Yang Xiao · 2025
Read AFT: An Exemplar-Free Class Incremental Learning Method for Environmental Sound Classification by Xinyi Chen, Xi Chen, Zhenyu Weng, Yang Xiao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15523.txt.
Introduction
Xinyi Chen leads a team that teaches sound classifiers new sounds without erasing old ones. Sorting sounds from the world around us helps tasks like spotting rare wild animals. As the world changes, the model must learn new classes over time. The hard part is forgetting: new lessons can wipe out old skills. Saving old samples helps, but privacy can forbid that.
Their method, AFT, keeps no old samples. Instead it shifts the stored traits of old classes into the new model's space, in a squeezed form. Tests on two sound data sets show steady gains over base models. Accuracy rose by a bit under four points. The work suits settings where old data cannot be kept.
Catalog record
| CREATOR | Xinyi Chen, Xi Chen, Zhenyu Weng, Yang Xiao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-15523
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AFT: An Exemplar-Free Class Incremental Learning Method for Environmental Sound Classification Xinyi Chen, Xi Chen, Zhenyu Weng, Yang Xiao Abstract As sounds carry rich information, environmental sound classification (ESC) is crucial for numerous applications such as rare wild animals detection. However, our world constantly changes, asking ESC models to adapt to new sounds periodically. The major challenge here is catastrophic forgetting, where models lose the ability to recognize old sounds when learning new ones. Many methods address this using replay-based continual learning. This could be impractical in scenarios such as data privacy concerns. Exemplar-free methods are commonly used but can distort old features, leading to worse performance. To overcome such limitations, we propose an Acoustic Feature Transformation (AFT) technique that aligns the temporal features of old classes to the new space, including a selectively compressed feature space. AFT mitigates the forgetting of old knowledge without retaining past data. We conducted experiments on two datasets, showing consistent improvements over baseline models with accuracy gains of 3.7% to 3.9%. † † address: Author Affiliation(s) † † address: ⋆ South China University of Technology, China † The Chinese University of Hong Kong, Shenzhen, China ‡ The University of Melbourne, Australia 1 Introduction Environmental sound classification (ESC) [ 1 ] is an important task in audio signal processing. Its goal is to help computers recognize and label sounds in daily life, such as animal calls or crowd chatter. Compared with speech-related tasks like speech recognition or speaker identification, environmental sounds do not carry clear meaning or speaker identity cues [ 2 ] . As a result, classification mainly depends on acoustic features. At the same time, the sound environment is not fixed and changes constantly, which makes ESC models face new and unseen classes of sounds [ 3 , 4 , 5 ] . The need to adapt to completely new acoustic classes introduces the main challenge of catastrophic forgetting, where models lose the ability to recognize previous sounds once they learn new ones. To solve the problem of catastrophic forgetting, recent work has focused on class incremental learning (CIL) [ 6 ] , which is the most practical setting in continual learning [ 7 , 8 , 9 , 10 , 11 ] . In CIL, models learn new classes step by step while retaining old classes’ knowledge. Existing CIL methods for sound classification can be roughly divided into two categories based on their approach to handling historical data. The first category is replay-based methods [ 12 , 13 , 14 , 15 , 16 , 17 , 18 ] , which retain knowledge of old classes by replaying a portion of historical data, but accessing this data poses challenges due to privacy and security considerations. To address this limitation, the second category consists of exemplar-free methods like regularization [ 19 ] , knowledge distillation [ 20 ] , analytic learning [ 21 ] , or dynamic network expansion [ 22 ] , which eliminate the need to store any historical data. [ 23 ] keeps old knowledge by minimizing the differences in feature representations and logits between learners, while [ 19 ] enables the model to adaptively optimize gradient directions to capture the balance between new and old knowledge. Such methods are better suited to memory-constrained edge devices and mitigate data privacy concerns, as no past audio samples need to be stored. However, current exemplar-free methods still face performance degradation when novel and existing sound classes share high acoustic similarity (e.g., in frequency patterns or temporal stability). Without historical data as a reference, learning new classes gives rise to a critical issue: the temporal features of old classes become severely misaligned within the updated feature space. This problem is particularly noticeable for acoustically similar sounds. For example, “drilling” and “jackhammer” both have short, sharp high-frequency components, whereas “street music” has a more slow-paced temporal profile. As the model updates its weights to learn new acoustic characteristics, the features of old classes drift away from their previously learned feature space. This eventually leads to the loss of key cues that help recognize sounds the model learned before, as shown in the Figure 1 : “jackhammer” is 32% misclassified as “drilling” , while “street music” , which has clearly different temporal features, maintains relatively high recognition accuracy. (a) (b) Figure 1 : (a) presents t-SNE feature clusters of street music and jackhammer (trained in the previous phase) with clear separation; (b) shows feature distribution after fine-tuning only on drilling data: jackhammer is 32 % mis-classified as drilling, whereas street music maintains a relatively clear feature boundary. Figure 2 : The diagram shows the AFT framework’s key modules: red for the frozen previous-phase (Task \tau_{t-1} ) components, green for the elements being trained in the current phase (Task \tau_{t} ), and the purple box indicates the Acoustic Feature Transformation Network (AFT Network). The core AFT Network bridges old and new knowledge by aligning old acoustic features with the new model’s feature space, enabling the model to retain prior knowledge while adapting to new environmental sound categories. Additionally, the diagram’s plots show the feature space; the AFT Network updates old classes’ space after each task and incorporates new classes’ prototypes into it. To address this issue, we propose an Acoustic Feature Transformation (AFT) mechanism designed to mitigate acoustic feature variation and reduce catastrophic forgetting without relying on stored past data, making it ideal for privacy-sensitive scenarios. Inspired by the human brain’s memory mechanism, AFT builds connections between old and new tasks by aligning old feature representations into the new model’s feature space. Its implementation involves three core steps: first, applying feature-level knowledge distillation to transfer key old-model information to the new one; second, transforming the old feature space into the new space for direct comparison to prevent feature variation; third, constructing a selectively compressed acoustic feature space via high-quality samples to maintain clear sound class boundaries. Our contributions are as follows: • We propose AFT, a novel mechanism that mitigates acoustic feature variation and reduces catastrophic forgetting without stored past data. • Inspired by human memory, AFT resolves dynamic learning’s feature inconsistency by cross-task alignment, strengthens sound boundaries with a selectively compressed acoustic feature space, and enhances model adaptability in real-world acoustic tasks. • Extensive experiments on two public datasets show AFT outperforms baseline models, achieving absolute accuracy gains of 3.7% to 3.9% and lower forgetting, validating its effectiveness. 2 OUR METHOD 2.1 Problem Formulation In this study, we aim to classify environmental sounds in stages and reduce forgetting. We assume the model learns a series of tasks, denoted as { \tau_{1} , \tau_{2} , …, \tau_{t} }, in separate stages, where data from one task is not reused in others. For each task \tau_{t} , we are given a pair of inputs { x_{t} , y_{t} }, where x_{t} is the environmental sound data and y_{t} are the ground truth labels following the distribution D_{t} . Our goal is to minimize the cross-entropy loss [ 6 ] across all tasks after the entire sequence is learned. \displaystyle\mathcal{L}_{\text{ce}}=\sum_{(x_{i},y_{i})\in D_{t}}y_{i}\text{log}\frac{\text{exp}(o_{i})}{\sum_{j\in D_{t}}\text{exp}(o_{j})} (1) where o denotes the output logits of the model in the task \tau_{t} . 2.2 AFT Structure We present an overview of our proposed AFT