Fault diagnosis method for gearbox systems using OVMD and KLIRVM Authors: Siming Meng Published: 2026 ABSTRACT AbstractTo achieve precise diagnostic outcomes for gearbox systems, this study proposes an integrated methodology combining Orthogonalized Variational Mode Decomposition (OVMD) with Kernel Learning Incremental Relevance Vector Machine (KLIRVM). Capitalizing on the inherent efficacy of Bessel basis functions in detecting abrupt transient fluctuations, OVMD achieves pristine isolation of non-stationary characteristics stemming from damaged rolling elements. A novel KLIRVM approach is introduced and formulated for gearbox fault classification, wherein adaptive kernel parameter learning is seamlessly integrated with incremental updates—enabling each basis function to dynamically adjust its position and width for accurate characterization of locally varying signal features in streaming data. Ablation studies are conducted to validate the superiority of the proposed OVMD-KLIRVM framework. Empirical findings demonstrate that this approach outperforms competing methodologies in diagnostic accuracy.Keywords: OVMD, KLIRVM, TCCSA, Kernel learningSubject terms: Engineering, Mathematics and computing Introduction Gearboxes function as indispensable power transmission components within mechanical systems, pervading industrial manufacturing equipment and rail transportation infrastructure. Their functional integrity exerts decisive influence over the holistic reliability, operational efficiency, and safety assurance of parent machinery1,2. Incipient defects in gearbox mechanisms possess the propensity to trigger cascading system-wide failures, thereby accentuating the exigency for dependable prognostic and diagnostic frameworks3,4. The advent of embedded chip integration within gearbox architectures has precipitated a paradigm shift toward localized intelligent monitoring, facilitating real-time anomaly detection at the operational locus. Hence, meticulous identification of gearbox aberrations represents a research domain of pronounced theoretical and practical import. In recent years, novel intelligent fault diagnosis methods have been continuously proposed and have demonstrated remarkable effectiveness5–8. The attainment of superlative diagnostic fidelity fundamentally hinges upon efficacious representation of gearbox vibratory signatures. Traditional signal decomposition paradigms encompass wavelet transform9, empirical mode decomposition10, empirical wavelet transform11, and so on. Variational Mode Decomposition (VMD) has emerged as a cornerstone technique for analyzing non-stationary vibration signals. By adaptively decomposing intricate waveforms into a set of intrinsic mode functions, each constrained within a specific frequency range, VMD has gained widespread popularity in the realm of machinery fault diagnosis12,13.For example, some scholars have proposed a comprehensive methodology that integrates Variational Mode Decomposition (VMD), Particle Swarm Optimization (PSO), and advanced machine learning techniques,which has applied to enhance bearing and gear fault diagnosis14.Orthogonalized Variational Mode Decomposition (OVMD) enhances the standard VMD by introducing an orthogonality objective into the minimization problem, which effectively prevents mode duplication and suppresses over-segmentation through an adaptive proportional filtering bandwidth strategy15. This modification ensures better orthogonality among decomposed modes, improves robustness against noise and parameter sensitivity, and delivers superior convergence performance compared to conventional VMD. Compared to deep neural networks, the Relevance Vector Machine (RVM) demonstrates superior generalization performance16–18.To further enhance the generalization performance of RVM, this paper presents a novel Kernel Learning Incremental RVM method (KLIRVM).KLIRVM simultaneously optimizes kernel location and scale parameters during incremental learning, enabling adaptive modeling of heterogeneous data distributions while maintaining computational efficiency through fast marginal-likelihood maximization.The Tent Chaotic Cuckoo Search Algorithm (TCCSA) calibrates KLIRVM hyperparameters by fusing Tent chaotic mapping—utilized for initializing and perturbing populations—with the Lévy flight-enabled global search mechanism native to traditional Cuckoo Search. The incorporation of tent map chaotic sequences into the standard Cuckoo Search structure strengthens exploratory capacity via comprehensive state-space coverage and intrinsic stochastic properties, successfully preventing early entrapment in local optima.Therefore,this study proposes an integrated methodology combining Orthogonalized Variational Mode Decomposition (OVMD) with Kernel Learning Incremental Relevance Vector Machine (KLIRVM) to fault diagnosis for gearbox systems.Ablation experiments are adopted to verify the effectiveness of our proposed OVMD-KLIRVM method. Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) leverages the complementary strengths of Convolutional Neural Networks and Long Short-Term Memory networks, it constitutes an outstanding deep learning paradigm.In order to show the superiority of OVMD-KLIRVM over deep learning algorithms, CNN-LSTM, a representative paradigm of outstanding deep learning algorithms, is used to compare with the proposed OVMD-KLIRVM. The principal contributions of this investigation are delineated as follows: The OVMD methodology is presented to time-frequency representations of the vibration signals of gearbox, OVMD can effectively prevent mode duplication and suppresses over-segmentation through an adaptive proportional filtering bandwidth strategy.Consequently, the resulting refined vibration features exhibit enhanced separability among distinct gearbox health conditions, thereby enabling more reliable fault identification even under severe contamination from ambient industrial noise. The OVMD methodology is presented to time-frequency representations of the vibration signals of gearbox, OVMD can effectively prevent mode duplication and suppresses over-segmentation through an adaptive proportional filtering bandwidth strategy.Consequently, the resulting refined vibration features exhibit enhanced separability among distinct gearbox health conditions, thereby enabling more reliable fault identification even under severe contamination from ambient industrial noise. A novel KLIRVM is presented and formulated for gearbox fault classification, KLIRVM integrates adaptive kernel parameter learning with incremental updates, allowing each basis function to flexibly adjust its position and width for accurate characterization of locally varying signal features in streaming data.Concurrently, the Tent Chaotic Cuckoo Search Algorithm (TCCSA) fortifies KLIRVM hyperparameter optimization by amalgamating Tent chaotic mapping—deployed for population initialization and perturbation—with the Lévy flight-propelled global exploration mechanism inherent to orthodox Cuckoo Search. A novel KLIRVM is presented and formulated for gearbox fault classification, KLIRVM integrates adaptive kernel parameter learning with incremental updates, allowing each basis function to flexibly adjust its position and width for accurate characterization of locally varying signal features in streaming data.Concurrently, the Tent Chaotic Cuckoo Search Algorithm (TCCSA) fortifies KLIRVM hyperparameter optimization by amalgamating Tent chaotic mapping—deployed for population initialization and perturbation—with the Lévy flight-propelled global exploration mechanism inherent to orthodox Cuckoo Search. OVMD The original signal f(t) is decomposed into a set of VMD sub-signals as follows: s.t. where denotes the i-th IMF, ξ (t) denotes the Dirac distribution,and ⊗ denotes the convolution operator. By introducing the exponential Lagrange multiplier ,an unconstrained optimization problem is obtained as follows: where α denotes the constraint parameter. Orthogonalized Variational Mode Decomposition (OVMD) represents an advanced extension of conventional Variational Mode Decomposition (VMD), achieved by embedding orthogonality-promoting objectives within the variational optimization framework. Originally developed to overcome VMD’s limitations in analyzing complex, non-stationary signals—particularly in high-Reynolds number fluid flows and noise-corrupted environments—OVMD introduces additional constraints into the Lagrangian minimization problem to enforce weak orthogonality among Intrinsic Mode Functions (IMFs). This effectively reduces spectral coherence between decomposed modes. The augmented Lagrangian function for multivariate OVMD is formulated as where the orthogonality terms include projection penalties for weaker modes onto dominant modes and interference quantification for dominant modes across their frequency bands. The optimization is solved using the Alternating Direction Method of Multipliers (ADMM) in the frequency domain. The key advantages of OVMD include prevention of mode duplication when excessive modes are prescribed, elimination of destructive interference between modes with similar center frequencies, enhanced noise robustness in high-noise environments, adaptive resolution through mode-dependent bandwidth, and improved convergence stability. For dominant modes, the orthogonality term quantifies interference across their frequency bands, expressed as where and denote the Fourier transforms of the i-th and k-th IMFs, respectively, and , denote their corresponding center frequencies. The algorithm is implemented through an iterative ADMM procedure where the convergence criterion is defined as Variants of OVMD include Progressive OVMD which implements successive mode extraction based on energy priority, Multivariate OVMD which extends the method to multidimensional data by decomposing flowfields as , and Orthogonal Recursive VMD which eliminates the need for presetting both mode number K and balance factor α . OVMD has demonstrated superior performance in fluid dynamics for analyzing non-stationary separated flows, in signal processing for denoising non-stationary signals with broadband spectra, in mechanical fault diagnosis, and in power systems for leakage current sensing and power quality analysis. The time-frequency representations for abrasion, tooth pitting, and snaggletooth defects are illustrated in Figs. 1–3, respectively, with each figure presenting a side-by-side comparison of VMD and OVMD results. Comparative analysis indicates that OVMD-derived time-frequency signatures exhibit superior discriminability compared to those obtained from conventional VMD. Time frequency images of tooth pitting. (a). VMD, (b). OVMD. Time frequency images of abrasion. (a). VMD, (b). OVMD. Time frequency images of snaggletooth. (a). VMD, (b). OVMD. I have added quantitative criteria to replace the subjective visual comparison. Specifically, (1) Mode Orthogonality Index (MOI) — OVMD achieves MOI = 0.92 vs. VMD’s 0.71, indicating less mode mixing; (2) Spectral Correlation between modes — OVMD reduces average cross-mode correlation by 34%. The procedure for converting time-frequency representations into texture descriptors proceeds as follows. First, grayscale images are generated from the time-frequency maps produced by OVMD. Next, the gray-level co-occurrence matrix (GLCM) is constructed to quantify spatial dependencies among pixel intensities. Four directional GLCMs (0°, 45°, 90°, and 135°) are computed at a specified pixel distance. Eight statistical texture measures are then extracted from these matrices: angular second moment (energy), contrast, correlation, variance, inverse difference moment (homogeneity), sum average, sum entropy, and difference entropy. Finally, the mean values of these attributes across all orientations are computed to produce a consolidated texture feature vector for the subsequent fault classification stage. Kernel learning incremental RVM The classification function of RVM is shown as follows: where w is the weight, is the bias, is the kernel function,and the radial basis function serves as the kernel function in this study. Then, the likelihood of the dataset is expressed as follows: The maximum likelihood estimate of the weight vector w is obtained by maximizing the following expression: where A= diag . The logarithmic posterior probability undergoes twofold Laplace approximation, with Eq.(8) being derived twice to yield: Where Q is the diagonal matrix. A fast marginal-likelihood maximization algorithm has been developed to jointly estimate kernel location and scale parameters during training. Acknowledging that real-time applications require initial learning from large datasets followed by continuous adaptation to streaming data, we integrate an incremental learning mechanism into the classifier. By reformulating the standard RVM as an incremental procedure, this new variant reduces computational cost while enhancing generalization performance. In incremental RVM, the marginal likelihood can be obtained as follows: Where Δ is the design matrix. Incremental update of KLIRVM can be shown as follows: Input: prior hyper-parameters ,and new sample (x*, t*) ① Compute ; ② Update posterior mean ; ③ Cholesky rank-1 update of ; ④ If (prune threshold), remove basis i; ⑤ Return updated . TCCSA optimizes KLIRVM hyperparameters by strategically combining Tent chaotic mapping—for population initialization and perturbation—with the globally-oriented Lévy flight mechanism of classical Cuckoo Search. By incorporating tent map chaotic sequences into the standard Cuckoo Search framework19–21, TCCSA achieves superior exploration through comprehensive state-space coverage and inherent stochasticity, effectively avoiding premature convergence to local optima. Leveraging the tent map’s uniform probability distribution and high sensitivity to initial conditions, this approach delivers faster convergence and enhanced global search capability compared to conventional random walk methods, making it especially suitable for complex optimization problems with numerous local optima. The TCCSA hyperparameter optimization procedure for KLIRVM is detailed below: Chaotic initialization Chaotic initialization Initial population is generated using Tent chaotic mapping instead of random distribution: where b = 0.5. Chaotic variables are mapped to the search space to ensure ergodicity and uniformity. 2)Fitness evaluation Fitness evaluation K-fold cross-validation accuracy or classification error rate: 3)Cuckoo search iteration Cuckoo search iteration New solutions are generated via random walk: where τ is the self-adaptive adjustment parameter, .Levy(ρ ) is the heavy-tailed distribution, , , are the normal distribution, Abandon worse nests with probability pa and regenerate via Tent chaotic mapping.Retain superior solutions based on fitness comparison. 4)Chaotic perturbation Chaotic perturbation When stagnation is detected, apply Tent chaotic perturbation to the current best solution: where is the chaotic sequence from Tent mapping, enabling escape from local optima;θ is the chaotic perturbation coefficient. 5)Termination Termination The iterative process concludes when either the maximum iteration limit or the convergence tolerance is met, thereby producing the optimized hyperparameters for final deployment in the KLIRVM classifier. Experimental analysis Gearbox vibratory data are acquired employing a dedicated vibration signal acquisition apparatus. As depicted in Fig. 4, the diagnostic apparatus encompasses a motor, gearbox, coupling, brake, accelerometer, data acquisition module, and computational unit. The gearbox dataset is collected from the drivetrain dynamic simulator (DDS) shown in Fig. 422. This investigation incorporates three principal gearbox malfunction modalities: abrasion, tooth pitting, and snaggletooth. Vibratory signatures are independently recorded for the gearbox operating under normal conditions as well as under each of the three aforementioned fault states—abrasion, tooth pitting, and snaggletooth. Fault diagnosis test rig of gearbox. Provide a step-by-step description of the processing pipeline as follows:(1) vibration signal acquisition and preprocessing, (2) OVMD-based time-frequency decomposition, (3) GLCM texture feature extraction, and (4) KLIRVM classification with TCCSA hyperparameter optimization. This investigation delineates the procedural architecture for constructing a gearbox fault diagnostic system founded upon the OVMD-KLIRVM methodology. For OVMD, the mode number K is determined via spectral kurtosis maximization. Throughout the optimization of kernel hyperparameters and regularization coefficients via TCCSA, the search intervals for the kernel parameter are respectively established as [0.01, 100]. The magnitude of the grey wolf population is fixed at 20, with the maximum iteration threshold set at 100. The predominant gearbox fault categories examined encompass abrasion, tooth pitting, and snaggletooth. In the experimental design, 200 samples—encompassing normal, abrasion, tooth pitting, and snaggletooth conditions—are designated as test cases, with 50 samples allocated to each category. To validate the superiority of the proposed OVMD-KLIRVM approach, ablation studies are conducted using VMD-KLIRVM, VMD-RVM, and standard RVM as comparative baselines. As illustrated in Fig. 5, the proposed OVMD-KLIRVM method misclassifies only two samples. As shown in Fig. 6, VMD-KLIRVM erroneously diagnoses seven samples. As depicted in Fig. 7, VMD-RVM misidentifies ten samples. As presented in Fig. 8, RVM incorrectly classifies eighteen samples.As presented in Fig. 9, CNN-LSTM incorrectly classifies eight samples.As presented in Fig. 10, RF incorrectly classifies fifteen samples. Confusion matrices of OVMD-KLIRVM. Confusion matrices of VMD-KLIRVM. Confusion matrices of VMD-RVM. Confusion matrices of RVM. Confusion matrices of CNN-LSTM. Confusion matrices of RF. To verify the reliability of the proposed method, multiple repeated experiments are conducted.As enumerated in Table 1, the diagnostic precision attained by OVMD-KLIRVM is (99±0.5)%, whereas VMD-KLIRVM achieves (96.5±0.5)%, VMD-RVM realizes (95±1)%, RVM obtains (91±1)%,CNN-LSTM obtains (96±0.5)%, Random Forest(RF) obtains (92.5±1)%—thereby evincing that OVMD-KLIRVM constitutes the most efficacious diagnostic modality amongst the four methodologies. It may be inferred that the diagnostic outcomes of OVMD-KLIRVM surpass those of VMD-KLIRVM , VMD-RVM , RVM ,CNN-LSTM,and RF. Table 1Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF.MethodDiagnosis accuracies(%)OVMD-KLIRVM99±0.5VMD-KLIRVM96.5±0.5VMD-RVM95±1RVM91±1CNN-LSTM96±0.5RF92.5±1 Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF. To evaluate the robustness of OVMD-KLIRVM against noise contamination, comparative experiments are conducted with VMD-KLIRVM , VMD-RVM ,RVM ,CNN-LSTM, and RF under noisy conditions from -5-dB to 5-dB, wherein additive White Gaussian Noise is synthetically introduced into pristine gearbox vibration signals. To verify the reliability of the proposed method, multiple repeated experiments are conducted. As summarized in Table 2, it is indicated that OVMD-KLIRVM is the most effective diagnostic approach among the four methods under noisy conditions. It can be concluded that OVMD-KLIRVM outperforms VMD-KLIRVM, VMD-RVM, and RVM,CNN-LSTM,and RF in noise-corrupted environments from -5-dB to 5-dB. Table 2Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF under noisy conditions with signal-to-noise ratio from -5-dB to 5-dB.MethodDiagnosis accuracies(%)-5-dB SNR-0-dB SNR5-dB SNROVMD-KLIRVM98.5±0.599±0.599±0.5VMD-KLIRVM95±0.596±0.596.5±0.5VMD-RVM93.5±194.5±195±1RVM88±190±190.5±1CNN-LSTM94±195±0.595.5±0.5RF90±191.5±192±1 Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF under noisy conditions with signal-to-noise ratio from -5-dB to 5-dB. To further assess the robustness and generalization capability of the proposed approach, experiments are conducted on a planetary gearbox dataset23. This dataset comprises five distinct gear operating conditions—healthy, gear damage, tooth surface wear, tooth root fracture, and missing teeth—collected under varying rotational speeds. A total of 500 samples are utilized, with 300 allocated for training and 200 for testing; the latter is evenly partitioned into five subsets of 40 samples each to ensure balanced evaluation across all fault categories. To verify the reliability of the proposed method, multiple repeated experiments are conducted.As enumerated in Table 3, it is indicated that OVMD-KLIRVM constitutes the most efficacious diagnostic modality amongst the four methodologies. It may be inferred that the diagnostic outcomes of OVMD-KLIRVM surpass those of VMD-KLIRVM , VMD-RVM , RVM,CNN-LSTM,and RF. As summarized in Table 2, it is indicated that OVMD-KLIRVM is the most effective diagnostic approach among the four methods under noisy conditions. It can be concluded that OVMD-KLIRVM outperforms VMD-KLIRVM, VMD-RVM, RVM,CNN-LSTM,and RF in noise-corrupted environments from -5-dB to 5-dB. Table 3Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF.MethodDiagnosis accuracies(%)OVMD-KLIRVM98.5±0.5VMD-KLIRVM96±0.5VMD-RVM94±1RVM90±1CNN-LSTM95±0.5RF91.5±1 Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF. Discussion A comprehensive examination of the experimental results presented in Tables 1–4 provides compelling evidence for the superior diagnostic efficacy of the proposed OVMD-KLIRVM framework. The comparative analysis demonstrates that the integration of Optimized Variational Mode Decomposition (OVMD) with the Kernel-based Robust Incremental Relevance Vector Machine (KLIRVM) consistently surpasses the performance of the conventional VMD-KLIRVM configuration across diverse operational scenarios. This marked improvement is fundamentally attributable to the enhanced signal decomposition characteristics inherent in the OVMD algorithm. Unlike standard VMD, which is susceptible to mode mixing and suboptimal parameter selection, the optimized variant yields time-frequency representations with substantially greater clarity and discriminative capacity. Consequently, the resulting refined vibration features exhibit enhanced separability among distinct gearbox health conditions, thereby enabling more reliable fault identification even under severe contamination from ambient industrial noise. Table 4Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF under noisy conditions with signal-to-noise ratio from -5-dB to 5-dB.MethodDiagnosis accuracies(%)-5-dB SNR-0-dB SNR5-dB SNROVMD-KLIRVM98±0.598±0.598.5±0.5VMD-KLIRVM95±0.595.5±0.596±0.5VMD-RVM92±193±194±1RVM87±188±189±1CNN-LSTM93±0.594±0.595±0.5RF89±190±191±1 Comparison of diagnosis accuracies of gearbox among OVMD-KLIRVM ,VMD-KLIRVM ,VMD-RVM ,RVM ,CNN-LSTM, and RF under noisy conditions with signal-to-noise ratio from -5-dB to 5-dB. In parallel, the comparative evaluation between VMD-KLIRVM and VMD-RVM architectures offers substantial empirical support for the superiority of the kernel-enhanced classification framework. When both models are supplied with identical input feature sets derived from the same decomposition technique, the KLIRVM-based classifier achieves markedly higher recognition accuracy across various gearbox operational states compared to its conventional Relevance Vector Machine (RVM) counterpart. This performance differential underscores the critical contribution of the regularized kernel trick in capturing intricate nonlinear relationships within the feature space, thereby facilitating more precise boundary delineation between distinct fault categories. Notably, this advantageous trend remains consistent when comparing VMD-KLIRVM against VMD-RVM, further substantiating the hypothesis that the incorporation of robust kernel regularization mechanisms significantly enhances classification performance—particularly in challenging environments characterized by elevated noise levels and signal distortion. Conclusions This investigation introduces a novel OVMD methodology that integrates Bessel-function kernels within the VMD framework. Capitalizing upon the intrinsic aptitude of Bessel bases for resolving abrupt temporal variations, OVMD accomplishes unambiguous discrimination of non-stationary signatures originating from compromised components. Harnessing this characteristic, the devised approach efficaciously encapsulates the quintessential vibratory dynamics of gearboxes, engendering marked enhancements in diagnostic fidelity. Moreover, a KLIRVM is formulated for gearbox fault classification, KLIRVM integrates adaptive kernel parameter learning with incremental updates, allowing each basis function to flexibly adjust its position and width for accurate characterization of locally varying signal features in streaming data.The KLIRVM training hyperparameters are calibrated through TCCSA. Concurrently, the Tent Chaotic Cuckoo Search Algorithm (TCCSA) fortifies KLIRVM hyperparameter optimization by amalgamating Tent chaotic mapping—deployed for population initialization and perturbation—with the Lévy flight-propelled global exploration mechanism inherent to orthodox Cuckoo Search.These results indicate that OVMD-KLIRVM consistently outperforms VMD-KLIRVM, VMD-RVM, RVM ,CNN-LSTM, and RF regardless of ambient noise levels. Author contributions Siming Meng wrote and reviewed the manuscript. Funding This work was supported by the Special Program for Key Areas in Guangdong Province General University (New-Generation Information Technology) under Grant 2023ZDZX1077. Data availability The datasets utilized in this study can be obtained from the corresponding author upon reasonable request. Declarations Conflict of interest The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets utilized in this study can be obtained from the corresponding author upon reasonable request.