Hybrid Active-Online Learning Framework for Label-Efficient Concept Drift Adaptation in Optical Network Failure Detection

Yousuf Moiz Ali (1), Jaroslaw E. Prilepsky (1), João Pedro (2), Sasipim Srivallapanondh (3), Antonio Napoli (3), Sergei K. Turitsyn (1), Pedro Freire (1) · 2026

Read Hybrid Active-Online Learning Framework for Label-Efficient Concept Drift Adaptation in Optical Network Failure Detection by Yousuf Moiz Ali (1), Jaroslaw E. Prilepsky (1), João Pedro (2), Sasipim Srivallapanondh (3), Antonio Napoli (3), Sergei K. Turitsyn (1), Pedro Freire (1) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30322.txt.

Introduction

Yousuf Ali, Jaroslaw Prilepsky, and five colleagues combine active and online learning to detect failures in optical networks with few labels. Optical networks carry huge traffic, and failures must be caught fast. Machine learning models for detection go stale as networks age. This drift in data patterns, called concept drift, breaks static models. Hard failure events are rare and poorly documented.

Online learning updates the model sample by sample as data stream in. Their earlier online detector adapted to new hard failures with low latency. It assumed expert labels were always available during training. Real installations rarely enjoy that constant label supply. Active learning asks experts to label only the most informative samples. The hybrid framework joins the two strategies for label efficiency. It targets drift adaptation where annotation budgets are tight. The testbed generates data with a wavelength selective switch simulating failures.

Catalog record

CREATORYousuf Moiz Ali (1), Jaroslaw E. Prilepsky (1), João Pedro (2), Sasipim Srivallapanondh (3), Antonio Napoli (3), Sergei K. Turitsyn (1), Pedro Freire (1)
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30322

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hybrid Active-Online Learning Framework for Label-Efficient Concept Drift Adaptation in Optical Network Failure Detection Yousuf Moiz Ali (1), Jaroslaw E. Prilepsky (1), João Pedro (2), Sasipim Srivallapanondh (3), Antonio Napoli (3), Sergei K. Turitsyn (1), Pedro Freire (1) Abstract 1 Introduction Effective and timely failure management is of utmost importance in optical networks to ensure continuous service to end users [ 14 ] . For this task, different machine learning (ML) based methods have been proposed to automate the entire failure management pipeline and reduce the risk of human error [ 14 , 16 , 15 ] . However, these ML models often struggle to adapt when the data distribution changes [ 19 ] . In an optical network, the distribution of the data points can change due to various factors such as equipment aging or varying channel properties [ 2 , 7 ] . This shift in the data’s statistical properties is referred to as Concept Drift (CD) [ 13 ] . Static models, which remain fixed after training, generally fail to adapt to CD in the dataset, thereby leading to misclassifications. Figure 1: Experimental testbed setup used to generate the dataset. The Wavelength Selective Switch (WSS) was used to introduce attenuation at OA1 to simulate normal and failure conditions. One prominent example of CD in optical telemetry occurs when a model trained on soft failure data encounters hard failure events. Hard failures are rare and difficult to simulate, given the limited data from network operators and vendors. Online learning has emerged as one of the tools for adapting to CD. Online learning differs from traditional batch learning in that data arrives sequentially, and the model is updated once per sample. Training online models is prequential: the model first makes a prediction on a new sample, then learns from it [ 11 ] . In our recent work [ 1 ] , we proposed an online learning method for failure detection in optical networks. This approach was adapted to new hard failures and maintained low latency. However, because the models were trained using fully supervised learning, we assumed labels were constantly available. In real-world installations, this assumption rarely holds. Labels are often missing during training because experts or automated oracles are not always available to provide them. Active learning addresses this challenge by selectively querying only the most informative samples, making it suitable for label-scarce settings [ 18 ] . Active learning has received some attention in optical networks literature, but all works focused on Quality of Transmission (QoT) estimation [ 12 , 4 , 5 , 3 , 6 ] . To this end, we propose a novel hybrid active-online learning approach for failure detection in optical networks that leverages the power of online learning to adapt to CD in the dataset while using only the most informative samples, rather than the entire dataset, via active learning. The novelty of this work lies, to the best of our knowledge, in the first ever joint usage of active and online learning approaches, thus providing the best of both worlds. Our results indicate that the hybrid approach achieves the same accuracy and Area-Under-the-Curve (AUC) score as the supervised model while only using 3.4% of the available samples. The latency overhead of the hybrid approach is also very minimal, with only a 0.0026 ms increase in per-event latency. 2 Methodology and Dataset In this work, we used the dataset from [ 20 ] . Fig. 1 shows the testbed used to generate the data. The dataset is split into separate soft-failure (SFD) and hard-failure (HFD) datasets. It includes four features: Bit-Error-Rate (BER) and Optical Signal-to-Noise Ratio (OSNR) measured at both the transmitter ( BER_Tx , OSNR_Tx ) and receiver ( BER_Rx , OSNR_Rx ). Fig. 2 a shows the system design for the static, supervised online, and the hybrid system. The static model is used to predict only on the new sample. The supervised online model is trained in a prequential manner: it first predicts on the new sample, then learns from it. The hybrid model first predicts on the new sample, and then we use Uncertainty Sampling [ 18 ] to calculate the difference between the probabilities of the sample being a failure or normal. If the difference exceeds the Margin Threshold (tunable parameter) and there is budget to query the sample, we query it and obtain the label; otherwise, we skip the sample entirely. If the sample was queried, the hybrid model is updated with that sample, and one token is deducted from the budget counter. The budget increases by one token after a set number of steps, until it reaches the maximum allowed value. Uncertainty Sampling ensures budget is reserved only for the most informative samples and filter out those that might not add value to its parameters. To simulate real-world telemetry, we pre-trained all three models on the SFD and used the HFD as streaming data. To identify CD in the dataset, a two-stage procedure was deployed: the Page-Hinkley Test [ 17 , 10 ] (PHT) as the primary check, followed by the Kolmogorov-Smirnov (KS) test [ 8 ] as the second validation layer. PHT computes the running minimum and compares it with the cumulative sum of the data points; a drift is signaled if the value exceeds a set threshold. The KS test compares the cumulative distributions of two windows of data points. Currently, the drift detection mechanism is solely used for detecting drifts in the dataset and for visualization purposes and has no effect on the budget and the Margin Threshold . Figure 2: a) System design for the static, supervised online, and hybrid (active + online) systems. All three models were pre-trained on the SFD, and the HFD was used as streaming data. b) Data distribution of the OSNR_Rx feature, separated by the SFD (left) and HFD (right) boundary. Confirmed drifts are marked green (normal) and red (failure). Figure 3: Margin threshold impact on hybrid performance. Values show the final rolling window (500 samples) with query counts in parentheses. The red dashed box indicates the optimal threshold. Fig. 2 b shows data distribution for the OSNR_Rx feature, and the confirmed drifts in the SFD and the HFD. We are only considering drifts in the OSNR_Rx feature; it exhibited the highest correlation with the failure label as it captures cumulative signal degradation across the transmission span. The difference between the drifts in the two datasets is that the failure class drifts in the HFD occur suddenly without any warning, while they build up in the SFD region. To observe the effect of extra failures, we added synthetic failure samples to the end of the HFD stream. These were generated by adding small amounts of noise to randomly selected existing failure samples. We used the Adaptive Random Forest (ARF) [ 9 ] algorithm for the static, supervised online, and hybrid systems. We selected this ensemble model because it performs well across most datasets and is specifically designed for online learning tasks. In the static model, the ARF behaves like a standard Random Forest algorithm. The metrics used in this work are the rolling accuracy and the AUC scores, both calculated over windows of 500 data points.

Read the complete text