Gradient-free Continual Learning

Grzegorz Rypeść · 2025

Read Gradient-free Continual Learning by Grzegorz Rypeść (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01219.txt.

Introduction

Grzegorz Rypeść (2025) proposes EvoCL, a way to teach neural networks new tasks without erasing old ones. Networks are notorious for catastrophic forgetting. The usual fixes either store old data or lean on gradient tricks. Both are awkward when memory is tight or past data cannot be kept. EvoCL instead updates the network with evolutionary algorithms, which need neither old data nor gradients. A small adapter module translates saved representations from earlier tasks into the model's current space.

Across several benchmarks, EvoCL matched or beat standard continual learning methods while obeying strict memory limits. Rypeść presents it as a simple, flexible new direction for learning that continues over time. He releases the code so others can reproduce the results.

Catalog record

CREATORGrzegorz Rypeść
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE7 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01219

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Gradient-free Continual Learning
Grzegorz Rypeść

Abstract




1
Introduction

Figure 1
:
Gradient-based optimizers cannot find a minimum suitable for solving both tasks A and B (joint) in CL as the gradient
\nabla_{\theta}L_{A}
cannot be computed due to the lack of data for A when training on B. However, provided a good approximation of loss function
L_{A}
(
\hat{L}_{A}
), a gradient-free optimizer can find such satisfactory minimum.

Continual learning (CL) presents a fundamental challenge in training neural networks on sequential tasks without experiencing catastrophic forgetting
[
2
]
. Traditionally, the dominant approach in CL has been gradient-based optimization, where updates to the network parameters are performed using stochastic gradient descent (SGD) or its variants
[
9
,
15
]
. However, a major limitation arises when previous data is no longer accessible, as is often assumed in CL settings
[
7
,
12
,
3
,
11
,
13
]
. In such cases, there is no gradient information available for past data, leading to uncontrolled parameter changes and consequently severe forgetting of previously learned tasks what is depicted in Fig.
1
.

What if the root cause of forgetting is not the absence of old data, but rather the absence of the gradients for old data? If the inability to compute gradients on past tasks is the primary reason for performance degradation in continual learning, then gradient-free optimization methods offer a promising alternative. Unlike traditional gradient-based methods, these techniques do not rely on backpropagation through stored data, enabling a fundamentally different mechanisms for preserving past knowledge.

By shifting focus from data availability to gradient availability, this work opens up new avenues for addressing forgetting in CL. We explore the hypothesis that gradient-free optimization methods can provide a robust alternative to conventional gradient-based continual learning approaches. We discuss the theoretical underpinnings of such method, analyze their potential advantages and limitations, and present empirical evidence supporting their effectiveness. By reconsidering the fundamental cause of forgetting, this work aims to contribute a fresh perspective to the field of continual learning and inspire novel research directions.

2
Method

We consider the well-established Exemplar-Free Class-Incremental Learning (EFCIL) scenario
[
11
,
9
]
, where a dataset is split into
T
tasks, each consisting of the non-overlapping set of classes. We utilize a task-agnostic evaluation, where the method does not know the task id during the inference. For the purpose of our method, we memorize
N
latent space features of size
S
per class similarly to
[
5
]
.

At each task
t>1
, our objective is to minimize
L_{<t}+L_{t}
, where
L_{<t}
ensures retention of previous tasks and
L_{t}
represents the classification loss for the new task. Since direct computation of
L_{<t}
is infeasible without past data, we approximate it as
\hat{L}_{<t}
using an auxiliary adapter network, e.g. MLP. This adapter transforms embeddings of past classes from the latent space of frozen model
F_{t-1}
to the latent space of the current model
F_{t}
. During naive SGD training, parameters of
F_{t}
would be updated via gradient descent as
\theta_{t}\leftarrow\theta_{t}-\nabla_{\theta}(L_{t}+\hat{L}_{<t})
. However, since
\hat{L}_{<t}
depends on transformed features outside the computational graph of
F_{t}
, gradient-based optimizers cannot update
\theta_{t}
effectively. To overcome this, we employ a gradient-free evolution strategy to update
\theta_{t}
. The classification losses
L_{t}
and
L_{<t}
are computed using cross-entropy, where
L_{t}
is based on task
t
data and
L_{<t}
on adapter-transformed features. A linear classification head, reinitialized at each task, is trained jointly with
F_{t}
. The adapter is optimized via mean squared error (
L_{MSE}
) loss by forwarding task
t
data through
F_{t-1}
and the adapter, with the target being
F_{t}
-processed data. The final loss to optimize is equal to:
L_{t}+\hat{L}_{<t}+\alpha*L_{MSE}
, where
\alpha
is the trade-off between the quality of classification of features and the adapter.

3
Experiments

We perform experiments well-established EFCIL benchmark datasets. MNIST
[
1
]
and FashionMNIST
[
14
]
consists of 60k training and 10k test images belonging to 10 classes. More challenging - CIFAR100
[
6
]
- consists of 50k training and 10k testing images in resolution 32x32. We split these datasets into
T
equal tasks. As the feature extractor
F
we utilize MLP with two hidden layers for MNIST and Fashion MNIST where we train all the parameters. On the other hand, for CIFAR100 we train only a subset of parameters attached to the 4th block using LORA
[
4
]
. For the evaluation metric, we utilize commonly used average accuracy
A_{last}
, which is the accuracy after the last task, and average incremental accuracy
A_{inc}
, which is the average of accuracies after each task
[
9
,
8
,
3
]
. As the feature extractor
F
we utilize MLP with two hidden layers for MNIST and Fashion MNIST where we train all the parameters. On the other hand, for CIFAR100 we train only a subset of parameters attached to the 4th block using LORA
[
4
]
.

The results are provided in Tab.
1
. Our approach (dubbed EvoCL) performs much better on MNIST and FashionMNIST datasets than baseline methods. We can see an improvement
over the most recent state-of-the-art method - AdaGauss
[
12
]
by 11.0% and 24.1% points in terms of average accuracy on MNIST split into 3 and 5 tasks, respectively. This improvement is also consistent in terms of average incremental accuracy - 6.3% and 12.4% points. However, EvoCL performs worse than AdaGauss on CIFAR100 - 8.7% and 5.9% lower average accuracy on 10 and 20 tasks respectively. Further investigation is required to explain why the methods performs poorly - is it because of the frozen part of the feature extractor or more complex dataset?

Table 1
:
Average incremental and last accuracy in EFCIL for different datasets, baselines and number of tasks
T
. Gradient-free appproach (EvoCL) yields very promising results.

MNIST

FashionMNIST

CIFAR100

Method

T
=3

T
=5

T
=3

T
=5

T
=10

T
=20

A_{last}

A_{inc}

A_{last}

A_{inc}

A_{last}

A_{inc}

A_{last}

A_{inc}

A_{last}

A_{inc}

A_{last}

A_{inc}

Upper bound

99.7

92.1

85.8

Finetune

42.4

58.2

26.6

46.2

42.2

51.9

14.7

42.3

22.6

31.8

12.7

24.1

PASS
[
16
]

57.4

66.3

36.7

57.9

49.2

58.2

31.5

59.2

30.5

47.9

17.4

32.9

LwF
[
7
]

61.5

78.3

43.3

68.1

63.7

67.4

53.7

66.2

32.8

53.9

17.4

38.4

FeTrIL
[
10
]

60.4

76.1

41.6

60.8

61.4

66.7

50.6

62.4

34.9

51.2

23.3

38.5

FeCAM
[
3
]

62.2

80.7

46.1

66.9

63.6

69.1

53.2

66.6

32.4

48.3

20.6

34.1

AdaGauss
[
12
]

67.7

82.4

50.4

73.2

66.2

71.9

55.4

67.3

46.1

60.2

37.8

52.4

EvoCL

78.7

88.7

74.5

85.6

72.6

78.3

66.1

71.4

37.4

54.8

31.9

44.7

4
Conclusions and limitations

Read the complete text