Multimodal semantic learning for early fault diagnosis in warp knitting machines via masked fault modeling Authors: Wei Gao, Han Shi, Dazhou Li, Yang Liu Published: 2026 ABSTRACT Warp knitting machines operate under high-speed and continuous working conditions, where minor abnormalities in yarn tension or vibration may quickly evolve into severe defects. Conventional vision-based inspection methods usually detect faults only after visible defects appear, resulting in substantial latency and limited capability for early warning and predictive maintenance. To address this issue, we propose KnitLLM, a multimodal semantic learning framework for early fault diagnosis in warp knitting machines, which integrates sensor sequences and fabric images for proactive anomaly detection. KnitLLM first employs a vector-quantized variational autoencoder to map heterogeneous temporal signals and image features into a unified discrete semantic space, enabling aligned fault primitive representation across modalities. A Transformer-based masked fault modeling strategy is then introduced to learn long-range dependencies and structural priors of the knitting process by reconstructing masked process states. In addition, a diffusion model is used to augment rare fault samples, alleviating class imbalance and improving recognition performance for minority fault categories. Theoretical analysis from the perspectives of manifold representation, quantization error, and attention structure further supports the semantic abstraction and information preservation capabilities of the proposed framework. Experiments on a real-world industrial dataset containing 10,000 h of warp knitting production data demonstrate that KnitLLM consistently outperforms competitive baselines across multiple metrics, achieving an F1-score of 0.961 and a mean detection latency of 124 ms. The results indicate that the proposed method can identify early faults before visible defects emerge, offering a practical solution for intelligent maintenance and fault warning in high-speed textile manufacturing. Introduction With the rapid development of intelligent manufacturing, fault diagnosis for industrial equipment is shifting from post-event inspection toward predictive maintenance1,2. In the textile industry, warp knitting machines are high-speed precision devices that typically operate at rotational speeds of 2000 to 2500 revolutions per minute. Under such conditions, even minor mechanical abnormalities may rapidly evolve into serious defects such as yarn breakage, bars, or fabric damage, leading to substantial and often irreversible production losses3,4. Conventional fault detection methods rely mainly on manual inspection or offline quality control. Such passive, result-oriented approaches are inherently delayed, as alarms are usually triggered only after visible defects appear on the fabric surface, by which point losses have already occurred5,6. In recent years, the adoption of Industrial Internet of Things (IIoT) technologies has enabled warp knitting machines to be equipped with high-frequency sensors7,8, such as tension sensors, speed encoders, and vibration accelerometers, together with machine vision systems, thereby providing the data foundation for early fault warning9. However, effectively integrating temporal dynamic features from sensor signals with spatial texture information from fabric images to achieve end-to-end fault diagnosis from physical causes to visual manifestations remains a major challenge10,11. Existing studies on fault diagnosis for warp knitting machines mainly follow two technical routes. The first focuses on fabric defect recognition based on machine vision12,13. These methods generally use convolutional neural networks to extract texture features from fabric surfaces and perform automatic classification of visually evident defects such as holes, oil stains, and needle marks14,15. However, vision-based methods suffer from inherent latency: image sensors can only capture defects after they become visible on the fabric surface and are therefore unable to perceive the dynamic causes that induce them16,17. For example, when yarn tension fluctuates abnormally, such disturbances may take hundreds of milliseconds or even several seconds to develop into visible bars or density irregularities, meaning that the optimal detection window has already passed. The second route focuses on temporal anomaly detection using sensor signals, where deep learning models monitor physical variables such as tension, rotational speed, and current18,19. Although these sensor-based methods offer better timeliness, they face two important limitations. First, high-frequency industrial signals often contain strong non-stationary noise and periodic vibrations, making threshold-based decisions on raw signals prone to false alarms20. Second, information from a single modality is limited, making it difficult to reliably distinguish transient disturbances under normal operation from early signs of real faults21. From a broader perspective, multimodal learning and temporal modeling have shown clear advantages in industrial fault diagnosis22,23. By integrating information from different physical domains, multimodal methods can provide a more comprehensive description of equipment conditions24,25. Meanwhile, Transformer-based architectures have made significant progress in temporal anomaly detection26,27, owing to their ability to capture long-range dependencies through self-attention28,29. For instance, Anomaly Transformer achieves unsupervised anomaly detection through association discrepancy modeling27. Variable Temporal Transformer considers both temporal dependencies and inter-variable relationships30, improving performance on multivariate time series31,32. TimesNet transforms time series into two-dimensional tensors and applies convolution to extract periodic patterns33, achieving state-of-the-art results on several benchmark datasets. PatchTST improves the modeling of local patterns through patch-based strategies34, while Informer reduces the computational cost of long-sequence modeling by using sparse attention mechanisms35. Nevertheless, these approaches are mainly designed for single-modality temporal tasks and do not provide semantic alignment mechanisms for heterogeneous multimodal data such as sensor signals and images36. In addition, recent research has explored intelligent optimization and reasoning methods to improve model performance across different applications. For example, particle swarm optimization has been used for automatic prompt design in large language model-based medical information extraction37, demonstrating the value of optimization-driven prompt engineering. Relation-based node quality evaluation has also been incorporated into multi-hop reasoning for sparse medical knowledge graph38, improving reasoning reliability under limited connectivity. Neural architecture search based on the Brain Storm Optimization Algorithm has likewise shown promise in designing efficient face detection models39. However, when directly applied to warp knitting fault diagnosis, two fundamental difficulties remain. First, sensor signals are high-frequency one-dimensional streams, whereas fabric images are discrete two-dimensional snapshots, and the difference in their temporal and spatial scales makes precise modality alignment difficult. Second, industrial fault data are often highly imbalanced, and the scarcity of rare fault samples prevents models from learning effective decision boundaries40,41. To address these issues, this study proposes KnitLLM, a multimodal semantic learning framework for early fault diagnosis in warp knitting machines. In we, LLM refers to a language-model-inspired masked semantic reasoning architecture operating over discrete multimodal fault primitives, rather than a conventional text-only large language model. Unlike approaches that simply concatenate multimodal inputs, KnitLLM is designed to achieve semantic alignment across heterogeneous data sources. The core idea is to abstract continuous mechanical signals into discrete fault primitives and build a Transformer backbone on this basis to learn the long-range structural priors of the weaving process. Specifically, a Vector Quantized Variational Autoencoder (VQ-VAE) is used as a context-aware tokenizer to jointly encode sensor time series and fabric images into a unified discrete semantic space42, enabling cross-modal alignment at the semantic level. Through masked modeling, the framework is trained to infer masked mechanical states from context, thereby learning the underlying structural rules of warp knitting operation. In addition, diffusion models are introduced to augment rare fault samples by synthesizing boundary cases in latent space43, alleviating the class imbalance problem. The proposed framework is not a simple combination of existing modules; rather, each component serves a distinct function within a unified semantic early fault diagnosis pipeline: VQ-VAE performs fault primitive discretization, masked modeling learns process grammar, and diffusion augmentation improves rare-event coverage. As illustrated in Fig. 1, the main contributions of we are threefold. First, at the methodological level, we propose a VQ-VAE-based multimodal semantic alignment framework that maps heterogeneous sensor signals and fabric images into a shared discrete fault primitive space, offering a new direction for industrial multimodal learning. Second, at the theoretical level, we analyze how masked fault modeling captures long-range structural priors in warp knitting operations through gradient flow analysis, and we establish the convergence of quantization error using manifold coverage theory, providing a mathematical basis for semantic fault detection. Finally, at the application level, we construct a real-world warp knitting dataset containing multiple fault types and conduct systematic experiments to validate the effectiveness of each module, offering empirical support for applying this approach in intelligent textile equipment.Fig. 1KnitLLM framework for semantic fault detection in warp knitting machines. KnitLLM framework for semantic fault detection in warp knitting machines. The main innovation lies in a semantic fault representation paradigm for industrial multimodal early fault diagnosis. Unlike existing multimodal fault diagnosis methods that rely on feature concatenation or decision-level fusion, KnitLLM first abstracts heterogeneous sensor signals and images into shared discrete fault primitives, and then uses masked fault modeling to infer latent process states and detect early deviations before visible defects emerge. Diffusion-based augmentation is used as a complementary strategy to improve the diversity of rare fault samples. The remainder of this paper is organized as follows: Sect. 2 describes the framework in detail, Sect. 3 gives the theoretical analysis, Sect. 4 presents the experimental results, Sect. 5 concludes the paper. The KnitLLM framework design Mathematical formalization of multimodal data in warp knitting machines In an industrial warp knitting scenario, the dataset consists of two synchronized modalities: sensor time series and fabric surface images. Let the raw observations be denoted as:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{X}} = \left\{ {({\mathbf{S}}_{i} ,{\mathbf{I}}_{i} )} \right\}_{i = 1}^{N}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}} \in {\mathbb{R}}^{{L \times d_{s} }}$$\end{document} denotes a multivariate time series of length L with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_{s}$$\end{document}-dimensional measurements, such as yarn tension, main shaft rotational speed, and motor current, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}} \in {\mathbb{R}}^{H \times W \times C}$$\end{document} denotes the corresponding fabric images acquired at the same operating time. Because warp knitting machines run at high speed, accurate alignment between sensor signals and images is essential for reliable diagnosis. The main challenge lies in the mismatch between the two modalities: sensor signals \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}$$\end{document} are high-frequency one-dimensional streams, whereas fabric images \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}$$\end{document} are two-dimensional spatial snapshots. To address this issue, we use a shaft-phase-triggered sampling mechanism so that each image frame \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}_{i}$$\end{document} is paired with a sensor sequence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}_{i}$$\end{document} of the same operating phase. The synchronization operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{T}}$$\end{document} can be written as:2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left\{ {{\boldsymbol{S}}_{i} ,{\boldsymbol{I}}_{i} } \right\} = {\mathcal{T}}\left( {t,\theta_{phase} } \right)$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\theta_{phase}$$\end{document} denotes the physical phase of the main shaft. This ensures that image acquisition occurs at consistent stages of the loop formation cycle, enabling precise alignment across modalities. Under this mechanism, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}} \in {\mathbb{R}}^{{L \times d_{s} }}$$\end{document} captures dynamic tension fluctuations during the knitting process, while \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}} \in {\mathbb{R}}^{H \times W \times C}$$\end{document} records the static spatial results generated by this dynamic process. To obtain a unified representation for downstream learning, we define a joint feature extraction operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{F}}_{\theta }$$\end{document}:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e} = {\mathcal{F}}_{\theta } \left( {{\boldsymbol{S}},{\boldsymbol{I}}} \right),{\boldsymbol{Z}}_{e} = \left\{ {{\boldsymbol{z}}_{e}^{\left( 1 \right)} , \ldots ,{\boldsymbol{z}}_{e}^{\left( T \right)} } \right\} \in {\mathbb{R}}^{{T \times d_{model} }}$$\end{document} The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document} serves as the baseline input for subsequent semantic operations, aligning physical signals of different dimensions to a unified representation space of dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_{model}$$\end{document}. This shared representation provides the basis for subsequent semantic modeling of machine states and fault patterns. Compared with conventional visual inspection methods, which can only identify defects after they become visible on the fabric surface, the proposed framework aims to detect abnormal process deviations before visible damage appears. This is possible because sensor signals typically reflect the underlying mechanical disturbance earlier than images capture the final defect. As a result, the model can identify early warning signs from temporal fluctuations in tension, rotational speed, and related variables. Unlike conventional LLMs designed for natural language generation, KnitLLM processes discrete fault tokens derived from multimodal industrial observations. Its purpose is semantic alignment and fault-state inference, not open-ended text generation. Context-aware tokenizer based on VQ-VAE To achieve semantic alignment of multimodal information, we design a dual-stream feature fusion operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{F}}_{\theta }$$\end{document} that projects heterogeneous data into a unified continuous feature space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{Z}}_{e}$$\end{document}. Specifically, we employ 1D-CNN to extract temporal features from sensors and 2D-ResNet to extract spatial features from images. As illustrated in Fig. 2, the context-aware tokenizer based on VQ-VAE generates multimodal fault primitives. The tokenizer integrates sensor logs (tension, rotational speed) with fabric images through dual-stream encoding, then discretizes continuous signals into a fault codebook \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{C}} = \{ {\boldsymbol{e}}_{k} \}_{k = 1}^{K}$$\end{document} via vector quantization. Each continuous feature vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{z}}_{e}^{\left( t \right)}$$\end{document} is mapped to its nearest codebook entry \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{e}}_{k}$$\end{document}, producing discrete fault primitives \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{z}}_{q}^{\left( t \right)}$$\end{document} that encode both mechanical dynamics and visual patterns. The tokenizer loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{vq}$$\end{document} uses stop-gradient operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{sg[}} \cdot {]}$$\end{document} to stabilize codebook learning, ensuring the learned vocabulary accurately captures structural characteristics of knitting machine operations.Fig. 2Context-aware tokenizer based on VQ-VAE for multimodal fault primitive generation. Context-aware tokenizer based on VQ-VAE for multimodal fault primitive generation. For each aligned sample pair \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left( {{\boldsymbol{S}}_{i} ,{\boldsymbol{I}}_{i} } \right)$$\end{document}, the local feature extraction process is:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{h}}_{s} = {\mathrm{Encoder}}_{1D} \left( {{\boldsymbol{S}}_{i} } \right);{ h}_{i} = {\mathrm{Encoder}}_{2D} \left( {{\boldsymbol{I}}_{i} } \right)$$\end{document} To achieve cross-modal semantic alignment, we fuse dual-stream features through concatenation and linear projection layers, generating joint feature vectors \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document}:5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e} = {\mathrm{Linear}}\left( {\left[ {{\boldsymbol{h}}_{s} ;{\boldsymbol{h}}_{i} } \right]} \right) \in {\mathbb{R}}^{{T \times d_{model} }}$$\end{document} Its physical significance lies in compelling the model to capture the intrinsic connection between physical fluctuations and visual outcomes within a unified \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_{model}$$\end{document}-dimensional space. For instance, when sensors display abnormal increase in tension signal \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{2}$$\end{document} while images present local texture thinning, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document} can capture such associative semantics, forming unique discrete primitive representations. To abstract continuous mechanical dynamics into discrete semantic tokens, we introduce a fault codebook \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{C}} = \{ {\boldsymbol{e}}_{k} \}_{k = 1}^{K}$$\end{document}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{e}}_{k} \in {\mathbb{R}}^{{d_{model} }}$$\end{document}. Through vector quantization, the fused feature \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{z}}_{e}^{\left( t \right)}$$\end{document} is mapped to discrete fault primitives:6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$z_{q}^{\left( t \right)} = quantize\left( {z_{e}^{\left( t \right)} } \right) = e_{k} ,where_{{}} k = \arg \mathop {\min }\limits_{j} \left\| {z_{e}^{\left( t \right)} - e_{j} } \right\|_{2}$$\end{document} This process embeds both mechanical state and fabric appearance into each fault primitive, moving beyond single-point signal representations. We define the tokenizer loss to guide the codebook \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{C}}$$\end{document} in learning the true structural properties of the warp knitting machine:7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{vq} = \left\| {sg\left[ {Z_{e} } \right] - Z_{q} } \right\|_{2}^{2} + \beta \left\| {Z_{e} - sg\left[ {Z_{q} } \right]} \right\|_{2}^{2}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{sg[}} \cdot {]}$$\end{document} denotes the stop-gradient operator used to stabilize the codebook update process. Masked fault modeling and gradient analysis of long-range structural priors As shown in Fig. 3, after obtaining discrete sequence, a masking matrix constructs masked sequence, which is processed by Transformer backbone to generate hidden representations. To capture long-range dependencies, the gradient flow of reconstruction loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{recon}$$\end{document} with respect to unmasked timestep j is analyzed, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\partial A_{tj} {/}\partial {\boldsymbol{z}}_{q}^{\left( j \right)}$$\end{document} quantifies how physical states at step j influence fault prediction at step t. Large \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} values indicate learned physical coupling between timesteps. For interpretability, attention weights are aggregated by phase bins: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{Profile}}\left( {{\Phi}_{k} } \right) = \sum {A_{tj} }$$\end{document}, compressing temporal attention into physical stages. Deviation metrics detect shifts in attention patterns before fault occurrence. The classification head combines discrete fault primitives from codebook lookup with attention gradients to produce diagnostic outputs mapping abstract signals to technician-readable terms like lateral comb shift hysteresis.Fig. 3Masked Fault Modeling (MFM) with gradient-based analysis of long-range structural priors. Masked Fault Modeling (MFM) with gradient-based analysis of long-range structural priors. The specific computational process is as follows: after obtaining discrete sequence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{q}$$\end{document}, we utilize masking matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{M}} \in \{ 0,1\}^{T}$$\end{document} to construct masked sequence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{{Z}}_{q}$$\end{document}:8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{{z}}_{q}^{\left( t \right)} = m^{\left( t \right)} \cdot {\boldsymbol{z}}_{q}^{\left( t \right)} + \left( {1 - m^{\left( t \right)} } \right) \cdot {\boldsymbol{e}}_{mask}$$\end{document} Fed into Transformer backbone \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{G}}_{\phi }$$\end{document} to obtain hidden representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{H}} = \left\{ {{\boldsymbol{h}}^{\left( 1 \right)} , \ldots ,{\boldsymbol{h}}^{\left( T \right)} } \right\}$$\end{document}. Through random masked reconstruction, the model is forced to infer mechanical states at masked moments from context, thereby learning semantic dependencies between sequences. To demonstrate how this mechanism captures long-range structural priors, we analyze the partial derivative of reconstruction loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{recon}$$\end{document} with respect to distant input tokens. Let the attention weight in the l-th layer self-attention mechanism be \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{ij} = {\mathrm{Softmax(}}{\boldsymbol{q}}_{i} {\boldsymbol{k}}_{j}^{T} {/}\sqrt {d_{k} } {)}$$\end{document}. For masked timestep t, the gradient flow of reconstruction loss with respect to unmasked timestep j (where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$m^{\left( j \right)} = 1$$\end{document}) can be expressed as:9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{{\partial {\mathcal{L}}_{recon} }}{{\partial {\boldsymbol{z}}_{q}^{\left( j \right)} }} = \mathop \sum \limits_{{t \in \left\{ {m = 0} \right\}}} \frac{{\partial {\mathcal{L}}_{recon} }}{{\partial {\boldsymbol{h}}^{\left( t \right)} }} \cdot \frac{{\partial {\boldsymbol{h}}^{\left( t \right)} }}{{\partial A_{tj} }} \cdot \frac{{\partial A_{tj} }}{{\partial {\boldsymbol{z}}_{q}^{\left( j \right)} }}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\partial A_{tj} /\partial {\boldsymbol{z}}_{q}^{\left( j \right)}$$\end{document} embodies how the physical state at step j (such as yarn fluctuations in preceding cycles) influences fault prediction at step t. When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} is large, it indicates the model has learned the physical coupling relationship between j and t in warp knitting machine operation, which constitutes structural priors. This cross-spatiotemporal gradient association forces the model to comprehensively consider the structured performance of the machine throughout the entire process cycle when reconstructing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{z}}_{q}^{\left( t \right)}$$\end{document}. Finally, the masked-recovered sequence is classified through the classification head:10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P({\boldsymbol{z}}_{q}^{\left( t \right)} |\hat{{Z}}_{q} ) = {\mathrm{Softmax}}\left( {{\boldsymbol{W}}_{h} {\boldsymbol{h}}^{\left( t \right)} } \right)$$\end{document} The attention weight \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} essentially serves as the mathematical foundation for fault causality tracing. In warp knitting machine scenarios, this dependency exhibits clear periodic structure: if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} presents periodic peaks at specific time spans \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\Delta } = t - j$$\end{document}, it typically corresponds to harmonic features of main shaft rotational speed, reflecting inherent patterns of mechanical loop formation. When fault judgment occurs at moment t, the region with maximum weight j represents the originating cause of the fault. To convert this mechanism into interpretable diagnostic evidence, we design a phase aggregation-based attention analysis method. The bare \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} matrix is excessively high-dimensional for intuitive interpretation. Through phase binning, attention can be compressed from the timestep dimension to the physical stage dimension, with mathematical form:11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{Profile}}\left( {{\Phi}_{k} } \right) = \mathop \sum \limits_{{j \in {\mathrm{bin}}\left( {{\Phi}_{k} } \right)}} A_{tj}$$\end{document} Equation (11) enables the model to quantify the explanatory power of historical data from specific process stages (such as needle pressing stage \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\Phi}_{k}$$\end{document}) for the current state. Furthermore, by computing the deviation of attention distribution, the system can capture changes in the model’s reasoning mode before classification probability anomalies: if an attention head shifts from yarn laying phase to fabric take-up phase, even without triggering fault alarms, it indicates misalignment of internal physical coupling in the machine. This closed-loop diagnostic logic combines fault primitives from Eq. (6) with attention gradients from Eq. (9), mapping abstract numerical signals to concrete process terminology through codebook lookup (such as mapping specific \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{e}}_{k}$$\end{document} to lateral comb shift hysteresis), thereby achieving effective transformation from deep learning models to industrial diagnostic systems. Attention weights \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tj}$$\end{document} from the Transformer’s multi-head attention mechanism (corresponding to Eq. (9) are extracted and mapped from sequence indices to phase space (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0^{ \circ } \sim 360^{ \circ }$$\end{document}) using physical parameters of the warp knitting machine (main shaft RPM and sampling frequency). The left panel in Fig. 4 shows the global correlation matrix with diagonal stripe patterns indicating periodic structural priors of mechanical cycles. The right panel in Fig. 4 highlights attention distribution at fault occurrence (t = 450), revealing which historical phases the model depends on for fault prediction. Elevated weights at specific angles (such as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$180^{ \circ } \sim 210^{ \circ }$$\end{document}) directly indicate problematic mechanical actions including needle pressing or guide bar shifting, enabling technicians to trace fault origins to specific process stages.Fig. 4Attention weight extraction and phase-space visualization for fault diagnosis. Attention weight extraction and phase-space visualization for fault diagnosis. Diffusion-based continuous feature augmentation and fault determination via joint objective function In industrial production, severe fault samples are often scarce, especially for rare events such as warp breakage or needle collision. To alleviate this imbalance, a diffusion-based generative module is introduced to synthesize additional fault-related feature samples in latent space, as shown in Fig. 5. The forward diffusion process is defined, injecting Gaussian noise into \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document} over \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{diff}$$\end{document} steps to obtain \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}^{\left( \tau \right)}$$\end{document}:12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q({\boldsymbol{Z}}_{e}^{\left( \tau \right)} |{\boldsymbol{Z}}_{e}^{{\left( {\tau - 1} \right)}} ) = {\mathcal{N}}\left( {\sqrt {1 - \alpha_{\tau } } {\boldsymbol{Z}}_{e}^{{\left( {\tau - 1} \right)}} ,\alpha_{\tau } {\boldsymbol{I}}} \right)$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha_{\tau }$$\end{document} represents the predefined noise schedule. We train a reverse denoiser \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon_{\theta }$$\end{document} conditioned on Transformer hidden states \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{H}}$$\end{document}, whose input comes from structured features extracted in Eq. (5), ensuring generated fault features conform to physical logic:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{diff} = {\mathbb{E}}_{{{\boldsymbol{Z}}_{e} ,\epsilon ,\tau }} \left[ {\left\| {\epsilon - \epsilon_{\theta } \left( {{\boldsymbol{Z}}_{e}^{\left( \tau \right)} ,\tau ,{\boldsymbol{H}}} \right)} \right\|_{2}^{2} } \right]$$\end{document}Fig. 5Diffusion-based feature augmentation and joint optimization framework for rare fault synthesis. Diffusion-based feature augmentation and joint optimization framework for rare fault synthesis. Equation (13) generates physically coherent virtual fault feature points within the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document} space through the diffusion model. These augmented features \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}^{aug}$$\end{document} are reintroduced into the processes of Eq. (6) and Eq. (7), substantially enriching the model’s understanding of extreme operating conditions. We integrate structural priors from self-supervised learning, sample richness from diffusion augmentation, and the final classification objective. For the augmented global feature representation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\overline{{\mathbf{h}}}$$\end{document}, the final fault probability \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{y}$$\end{document} is computed as:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{y} = \sigma \left( {{\boldsymbol{W}}_{y} \overline{{\mathbf{h}}} + b_{y} } \right)$$\end{document} The complete optimization objective \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{total}$$\end{document} of KnitLLM is defined as:15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{L}}_{total} = \lambda_{1} {\mathcal{L}}_{vq} + \lambda_{2} {\mathcal{L}}_{recon} + \lambda_{3} {\mathcal{L}}_{diff} + \lambda_{4} {\mathcal{L}}_{cls}$$\end{document} Equation (15) achieves a complete logical closed loop from low-level multimodal sampling to high-level semantic reasoning and generative data completion. Figure 6 displays sensor feature trajectories during a complete loop formation cycle of the warp knitting machine. Normal operation exhibits a periodic limit cycle (dense blue spiral region) representing stable mechanical dynamics, where features remain constrained within a low-dimensional manifold \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{M}}$$\end{document} corresponding to Lipschitz continuity. Fault onset (red marker) disrupts dynamic equilibrium, introducing velocity components perpendicular to the current tangent plane due to physical disturbances (such as tension drop from warp breakage or needle collision). Color gradient from blue to green, orange, and deep red traces the manifold deviation process, where increasing redness indicates larger residual δ between observed \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{z}}_{q}^{\left( t \right)}$$\end{document} and Transformer-estimated \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{h}}^{\left( t \right)}$$\end{document}. The post-fault trajectory diverges from periodic patterns, exhibiting semantic incoherence that violates learned structural priors of knitting syntax. Deviation direction along specific feature components enables root cause diagnosis: deviation along Feature Component 1 indicates tension system anomalies, while high-frequency oscillations suggest bearing wear or mechanical impact.Fig. 6Manifold deviation visualization of feature trajectories in latent space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{Z}}_{e}$$\end{document} during normal operation and fault occurrence. Manifold deviation visualization of feature trajectories in latent space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{Z}}_{e}$$\end{document} during normal operation and fault occurrence. Theoretical analysis Manifold modeling in warp knitting machine The operating process of a warp knitting machine is governed by periodic mechanical motions, yarn tension variation, and coupled electromechanical responses. These factors jointly determine the evolution of the machine state over time. To describe such dynamics, we regard the underlying operating condition as a low-dimensional manifold embedded in a higher-dimensional observation space. Under mild regularity conditions, this mapping is locally Lipschitz continuous:16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left| {{\boldsymbol{z}}e^{\left( t \right)} - {\boldsymbol{z}}e^{\left( j \right)} } \right|{*}2 \le K \cdot d{*}{\mathcal{M}}\left( {{\boldsymbol{s}}\left( t \right),{\boldsymbol{s}}\left( j \right)} \right)$$\end{document} Is a constant determined by feature extractor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{F}}\theta$$\end{document}, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d{\mathcal{M}}$$\end{document} represents the geodesic distance on the manifold. This demonstrates that mathematically preserves the spatiotemporal topological properties of mechanical state evolution, establishing the foundation for subsequent semantic abstraction. In industrial environments, sensor signals exhibit non-stationarity due to load fluctuations and environmental drift. To ensure the reliability of semantic tokens, we investigate the representation learning consistency by analyzing the mutual information between discrete fault primitives and continuous physical states. Given the Lipschitz constraint and the quantization radius of codebook, the mapping preserves the underlying manifold structure. We prove the mutual information satisfies the following lower bound:17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I\left( {{\boldsymbol{Z}}{*}q;{\boldsymbol{S}}} \right) \ge H\left( {\boldsymbol{S}} \right) - \frac{d}{2}{\mathrm{log}}\left( {2\pi e \cdot \left( {K^{2} \cdot \delta {*}{\mathcal{M}}^{2} + {\mathcal{E}}{*}q} \right)} \right)$$\end{document}where denotes the source entropy of mechanical dynamics and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta {*}{\mathcal{M}}$$\end{document} is the maximum geodesic deviation under non-stationary perturbations. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I\left( {{\boldsymbol{Z}}{*}q;{\boldsymbol{S}}} \right)$$\end{document} establishes that as long as the mapping constant and quantization error are minimized relative to the manifold scale, the discrete sequence remains a sufficient statistic for the mechanical state. This theoretical guarantee ensures that KnitLLM maintains cross-condition consistency, filtering stochastic noise while retaining the causal information necessary for early fault diagnosis. Quantization error and information preservation To prove this semantic abstraction does not lose critical fault information, we analyze the quantization error. For the continuous embedding at any timestep, the error bound induced by the discretization process is:18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{E}}_{q} = \left| {{\boldsymbol{z}}_{e}^{\left( t \right)} - {\boldsymbol{Q}}\left( {{\boldsymbol{z}}_{e}^{\left( t \right)} } \right)|_{2}^{2} = } \right|{\boldsymbol{z}}_{e}^{\left( t \right)} - {\boldsymbol{e}}_{k} |2^{2}$$\end{document} According to vector quantization theory, when the codebook capacity is sufficiently large and uniformly distributed, the upper bound of error term is constrained by the cell radius of codebook entries. Since warp knitting faults (such as warp breakage or bars) typically manifest as singular perturbations on manifold, as long as remains smaller than the fault-induced manifold deviation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\Delta }{\mathcal{M}}fault$$\end{document}, the discretized can fully characterize fault states at the semantic level. Structural prior analysis of self-attention To understand how attention weight in Eq. (9) captures long-range structural priors, we treat the Transformer’s self-attention mechanism as a dynamic kernel mapping. For the input discrete sequence (output of Eq. (6)), the correlation strength generated in the hidden layer can be expressed as:19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{K}}\left( {{\boldsymbol{z}}q^{\left( t \right)} ,{\boldsymbol{z}}q^{\left( j \right)} } \right) = {\mathrm{exp}}\left( {\frac{{ < {\boldsymbol{W}}q{\boldsymbol{z}}q^{\left( t \right)} ,{\boldsymbol{W}}k{\boldsymbol{z}}q^{\left( j \right)}> }}{{\sqrt {d_{k} } }}} \right)$$\end{document} The output directly determines Atj. In warp knitting processes, the correlation between states and is not random but determined by the mechanical cycle Tcycle. We define a structural prior operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}str$$\end{document} that describes the state transition probability of the machine during stable operation. Theoretically, after sufficient training, the attention weight Atj tends to approximate the empirical distribution of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}str$$\end{document}:20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathrm{lim}}N \to \infty A_{tj} \approx {\mathcal{P}}_{str} ({\boldsymbol{s}}\left( t \right)|{\boldsymbol{s}}\left( j \right))$$\end{document} It reveals the deep structural priors of KnitLLM: the model mathematically reconstructs the grammatical rules of warp knitting machine operation by learning from extensive normal production sequences. Theoretical interpretation of masked fault modeling We demonstrate the effectiveness of MFM in Eq. (6). Let the masked ground-truth state be. The hidden state of the converter is actually an estimate of the conditional expected value of the missing state based on the context:21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{h}}^{\left( t \right)} \approx {\mathbb{E}}[{\boldsymbol{z}}q^{\left( t \right)} |{\boldsymbol{z}}{*}q^{\left( j \right)} \cdot m^{\left( j \right)} {*}j \ne t]$$\end{document} Equation (21) indicates that when the machine experiences a fault, the actual observation deviates from the expected trajectory predicted by Eq. (21). This deviation magnitude directly constitutes the fault discrimination criterion in Eq. (6) and Eq. (14). Since the computation of Eq. (21) heavily depends on the structural prior \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}str$$\end{document} learned in Eq. (20), this means KnitLLM is not performing simple feature matching but rather utilizing mechanical operational logic to examine current sensor inputs. This semantic-level reconstruction capability constitutes the theoretical foundation for why KnitLLM outperforms traditional CNN/RNN methods. Results and analysis Experimental environment and dataset description To train the Transformer backbone and support the diffusion-based feature generation module, we used a high-performance computing cluster. The hardware and software environments were selected to ensure stable optimization and efficient processing of the large-scale dataset. At the hardware level, the system was equipped with one NVIDIA A100 GPU and an Intel(R) Xeon(R) Gold 6348 CPU running at 2.60 GHz. At the software level, experiments were conducted on Ubuntu 22.04.3 LTS with PyTorch 2.1.0 (https://pytorch.org) and CUDA 12.1. The vector-quantized variational autoencoder was built upon the vector-quantize-pytorch library 0.14.0 (https://github.com/lucidrains/vector-quantize-pytorch). The diffusion model component was developed using the Hugging Face diffusers library 0.21.0 (https://github.com/huggingface/diffusers). Data preprocessing and analysis were implemented using Pandas 2.1.1, NumPy 1.24.3 (https://numpy.org), and Scikit-learn 1.3.0 (https://scikit-learn.org). The Transformer backbone was configured with an embedding dimension of 256, 4 encoder layers, and 8 attention heads in each layer. The hidden dimension of the feed-forward network was set to 1024. Model optimization was performed using the AdamW optimizer with an initial learning rate of 1e-4, a weight decay of 1e-2, and a batch size of 32. The model was trained for 50 epochs, and the learning rate followed a cosine decay schedule. A dropout rate of 0.1 was applied to reduce overfitting. Early stopping was adopted according to validation performance, and all hyperparameters were determined empirically on the validation set. The dataset used in this study was collected from 10,000 h of proprietary production logs from a large textile enterprise. The data were recorded continuously from high-speed warp knitting machines operating under 24/7 production conditions. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}$$\end{document} denotes the sensor time-series data, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}$$\end{document} denotes the corresponding fabric images captured at the same operating time. The dataset contains two storage formats. The CSV files record numerical sensor values sampled from the machines under different operating conditions, including normal production, warp breakage, needle-bed collision, excessive oil temperature, and other fault states. Each record contains six core physical channels (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1} \sim s_{6}$$\end{document}) sampled at one-second intervals. The image repository stores fabric photographs aligned with the timestamps of the CSV records, as illustrated in Fig. 7. Together, the two modalities \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{M}}$$\end{document} preserve the operational dynamics of the warp knitting process and provide the basis for learning structural patterns in the data. Table 1–2 summarizes the main characteristics of the dataset.Fig. 7Representative normal and faulty fabric images from the warp knitting dataset.Table 1Multimodal sensor signals of the warp knitting machine.SymbolPhysical MeaningNormal RangeFault CharacteristicsSampling/Fault Semantics\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document}Main shaft RPM2000–2500 RPMFluctuation rate > 5% (transmission fault)Core frequency source\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{2}$$\end{document}Yarn tension15–25 cN < 5 (warp breakage) or > 40 (entanglement)Highly coupled with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{3}$$\end{document}Machine vibration0.5–1.5 G > 3.5 G (collision/wear)High-frequency sampling feature\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{4}$$\end{document}Motor current30–50 AAbnormal spikes (stalling)Power load indicator\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{5}$$\end{document}Yarn delivery1000–4000 mm/rackProportional imbalance (bars)Quality control parameter\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{6}$$\end{document}Oil temperature35–55℃ > 65℃ (cooling failure)Thermal slow variableTable 2Representative fabric image categories and semantic descriptions.Image typeFault typeCorresponding figureVisual patternSemantic meaningNormal imageNormalFigure 7(a)Interlaced wale-course structureNormal manifoldFault image 1Warp breakFigure 7(b)Longitudinal yarn interruptionSemantic chain breakageFault image 2Oil stainFigure 7(c)Localized dark patchesVisual supplementary detectionFault image 3Needle markFigure 7(d)Texture geometric distortionGeneralization challengeFault image 4HoleFigure 7(e)Severe physical damageLatency stress testFault image 5Over-densityFigure 7(f)Horizontal density bandsCorresponds to yarn delivery anomalyFault image 6Low lightFigure 7(g)Edge light attenuationEnvironmental degradation testFault image 7Loop overlapFigure 7(h)High-magnification loop structurePrimitive precision validation Representative normal and faulty fabric images from the warp knitting dataset. Table 1Multimodal sensor signals of the warp knitting machine.SymbolPhysical MeaningNormal RangeFault CharacteristicsSampling/Fault Semantics\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document}Main shaft RPM2000–2500 RPMFluctuation rate > 5% (transmission fault)Core frequency source\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{2}$$\end{document}Yarn tension15–25 cN < 5 (warp breakage) or > 40 (entanglement)Highly coupled with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{3}$$\end{document}Machine vibration0.5–1.5 G > 3.5 G (collision/wear)High-frequency sampling feature\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{4}$$\end{document}Motor current30–50 AAbnormal spikes (stalling)Power load indicator\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{5}$$\end{document}Yarn delivery1000–4000 mm/rackProportional imbalance (bars)Quality control parameter\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{6}$$\end{document}Oil temperature35–55℃ > 65℃ (cooling failure)Thermal slow variable Multimodal sensor signals of the warp knitting machine. Table 2Representative fabric image categories and semantic descriptions.Image typeFault typeCorresponding figureVisual patternSemantic meaningNormal imageNormalFigure 7(a)Interlaced wale-course structureNormal manifoldFault image 1Warp breakFigure 7(b)Longitudinal yarn interruptionSemantic chain breakageFault image 2Oil stainFigure 7(c)Localized dark patchesVisual supplementary detectionFault image 3Needle markFigure 7(d)Texture geometric distortionGeneralization challengeFault image 4HoleFigure 7(e)Severe physical damageLatency stress testFault image 5Over-densityFigure 7(f)Horizontal density bandsCorresponds to yarn delivery anomalyFault image 6Low lightFigure 7(g)Edge light attenuationEnvironmental degradation testFault image 7Loop overlapFigure 7(h)High-magnification loop structurePrimitive precision validation Representative fabric image categories and semantic descriptions. To prevent data leakage, we adopted a machine-level split strategy, in which all samples from the same machine or production run were assigned to the same subset, namely training, validation, or testing. In addition, the dataset was divided chronologically rather than randomly to preserve temporal independence. Within each subset, samples were constructed using temporally ordered sliding windows. The first 70% of the time-ordered samples were used for training, the next 10% for validation, and the remaining 20% for testing. To further reduce dependency between adjacent samples, overlapping windows were not shared across different splits, and a temporal buffer was inserted between the training and testing segments. Normalization statistics were computed only from the training set and then applied consistently to the validation and test sets. In the experimental design, we follow the task consistency principle. Regardless of whether the input is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}$$\end{document} (sensor), \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}$$\end{document} (image), or a combination of both, all models ultimately aim to predict \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y \in \left\{ {0,1} \right\}$$\end{document}. To ensure a fair comparison, image-based models such as YOLO were allowed to use only fabric images, whereas KnitLLM was given access to both sensor signals and images. This setting was used to examine whether visual information alone is sufficient for identifying early dynamic abnormalities, or whether it needs to be combined with sensor-based process information. To illustrate the validity of the dataset and the progression of fault events, Fig. 8 compares normal operating trajectories with fault-triggering moments. The main RPM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document} and yarn tension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{2}$$\end{document} exhibit abrupt changes in the labeled fault region at time t, deviating from their normal operating ranges.Fig. 8Visualization of sensor signal trajectories during normal operation and fault occurrence. Visualization of sensor signal trajectories during normal operation and fault occurrence. Evaluation metrics To quantify the performance of KnitLLM in subsequent experiments and conduct horizontal comparison with traditional models, seven commonly used metrics for fault detection were adopted. These metrics not only assess classification accuracy but also the reliability of fault judgment. The following metrics are calculated based on true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Precision (P) measures the proportion of actual faults among samples judged as faults:22\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P = \frac{TP}{{TP + FP}}$$\end{document} Recall (R) measures the proportion of all actual faults successfully detected during production, serving as the core indicator for assessing the risk of missed alarms for warp breakage shutdowns:23\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R = \frac{TP}{{TP + FN}}$$\end{document} To balance precision and recall and prevent models from improving recall through blind alarming, the harmonic mean of P and R, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-Score, is introduced:24\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1} = 2 \cdot \frac{P \cdot R}{{P + R}}$$\end{document} False Alarm Rate (FAR) is defined as the frequency of erroneous alarms issued by the model under normal operating conditions, corresponding to unnecessary shutdown costs in industrial scenarios:25\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$FAR = \frac{FP}{{FP + TN}}$$\end{document} Missed Alarm Rate (MAR) is defined as the frequency of fault samples being identified as normal:26\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$MAR = 1 - R = \frac{FN}{{TP + FN}}$$\end{document} Area Under the Curve (AUC) integrates true positive rate (TPR) and false positive rate (FPR) across different discrimination thresholds, measuring the model’s global discriminative capability under complex manifolds:27\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$AUC = \mathop \smallint \nolimits_{0}^{1} TPRd\left( {FPR} \right)$$\end{document} Mean Time to Detection (MTTD) measures the time difference between the physical fault occurrence moment and the model alarm generation moment:28\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$MTTD = \frac{1}{{N_{fault} }}\mathop \sum \limits_{i = 1}^{{N_{fault} }} (\hat{t}_{detect}^{\left( i \right)} - t_{start}^{\left( i \right)} )$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{t}_{detect}$$\end{document} depends on the moment when the model output first exceeds the decision threshold. This metric directly reflects the real-time response capability of KnitLLM. Benchmark comparison experiments and result analysis To ensure fairness and authority of the experiments, three groups of comparison models were selected for benchmark testing. The first group consists of industrial/warp knitting specialized models, including D-CNN44, which adopts dual-kernel optimization for fabric defect feature extraction; YOLOv5-Fabric45, a typical representative of applying visual detection to warp knitting conditions; MS-CN46, which utilizes multi-scale convolution to capture weaving anomalies at different densities. The second group consists of SOTA models from recent three years, including: Informer35, which employs ProbSparse attention to process long sequences; TimesNet33, currently recognized as one of the strongest benchmarks in the time series analysis domain; PatchTST34, which greatly improves semantic extraction capability of Transformer on time series tasks through patch thinking. The third group consists of classic baseline models, including: Vanilla CNN47, used to evaluate basic spatial feature extraction capability; LSTM48, representing traditional recurrent sequence modeling approaches; Vanilla Transformer26, used to verify the improvement effect of semantic abstraction in we relative to native attention. Table 3 reports the average performance of all models on seven evaluation metrics. KnitLLM achieves optimal results across all seven metrics. Particularly for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-Score, KnitLLM reaches 0.961, significantly higher than the SOTA model TimesNet (0.931). This is attributed to the VQ-VAE tokenizer, which effectively filters out pure fault primitives from the high-frequency electromagnetic noise of the factory. Additionally, the MTTD of KnitLLM is only 124 ms, proving that structural priors trained through MFM enable the model to make predictions based solely on subtle semantic deviations before the fault fully erupts.Table 3Performance comparison of different models across multiple evaluation metrics.Model namePrecision (P)Recall (R)F1-ScoreFARMARAUCMTTD (ms)KnitLLM0.9680.9540.9610.0120.0460.985124D-CNN0.8820.8450.8630.0540.1550.912285YOLOv5-Fabric0.9010.8210.8590.0420.1790.898198MS-CN0.8750.8320.8530.0580.1680.905310Informer0.9150.8980.9060.0350.1020.942242TimesNet0.9380.9250.9310.0240.0750.968185PatchTST0.9290.9180.9230.0280.0820.961164CNN0.8120.7850.7980.0950.2150.854352LSTM0.8340.8420.8380.0820.1580.876412Transformer0.8920.8750.8830.0480.1250.921215 Table 3Performance comparison of different models across multiple evaluation metrics.Model namePrecision (P)Recall (R)F1-ScoreFARMARAUCMTTD (ms)KnitLLM0.9680.9540.9610.0120.0460.985124D-CNN0.8820.8450.8630.0540.1550.912285YOLOv5-Fabric0.9010.8210.8590.0420.1790.898198MS-CN0.8750.8320.8530.0580.1680.905310Informer0.9150.8980.9060.0350.1020.942242TimesNet0.9380.9250.9310.0240.0750.968185PatchTST0.9290.9180.9230.0280.0820.961164CNN0.8120.7850.7980.0950.2150.854352LSTM0.8340.8420.8380.0820.1580.876412Transformer0.8920.8750.8830.0480.1250.921215 Performance comparison of different models across multiple evaluation metrics. Among the comparison models, YOLOv5-Fabric achieves relatively high precision (0.901) but lower recall (0.821) than KnitLLM. This suggests that image-based models are effective for visually apparent defects such as oil stains, but are less sensitive to anomalies caused by tension imbalance or other early-stage process irregularities, which may not be clearly reflected in the image modality. For the MTTD metric, image-based methods generally require more than 200 ~ ms, with YOLOv5-Fabric reaching 198 ~ ms, because visual processing depends on pixel-level feature extraction and often requires defects to become sufficiently visible. By contrast, KnitLLM operates on lower-dimensional semantic representations, which leads to faster response time. In addition, some faults, such as main-shaft bearing overheating \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{6}$$\end{document}, do not exhibit obvious visual symptoms. As a result, image-based models show very low recall on these cases, which reduces their overall performance. They are also more sensitive to variations in factory illumination over the 10,000-h production period, which increases the false-alarm rate. In contrast, the sensor abstraction mechanism used by KnitLLM is not affected by lighting conditions, which helps explain its lower FAR of 0.012. For high-speed warp-knitting equipment, image-only recognition is therefore insufficient for meeting strict real-time fault-response requirements, whereas KnitLLM provides better coverage of fault patterns through semantic compression of multimodal information. Figure 9 shows the grouped bar chart for Precision, Recall, and F1-score. KnitLLM achieves a high Recall of 0.954, which is meaningful in the warp-knitting scenario because minor fluctuations in yarn tension may appear before severe faults such as warp breakage. Compared with CNN-based models such as D-CNN and MS-CN, which mainly respond to visible texture changes on the fabric surface, KnitLLM uses Masked Fault Modeling (MFM) to capture latent process patterns and identify early structural deviations that are not yet visible in either the temporal or spatial domain.Fig. 9Evaluation of Precision, Recall, and F1-Score. Results demonstrate that KnitLLM outperforms other models by a significant margin. Evaluation of Precision, Recall, and F1-Score. Results demonstrate that KnitLLM outperforms other models by a significant margin. Figure 10 presents the global performance heatmap. The transition from cooler to warmer colors from CNN to KnitLLM indicates a shift from basic feature matching to more effective semantic reasoning. The performance difference is especially clear in the vibration and tension channels. KnitLLM maintains strong performance across all channels, suggesting that its learned representations can better approximate the underlying state manifold of the machine. In contrast, models such as LSTM are less effective at modeling long-range mechanical dependencies.Fig. 10Heatmap of normalized performance metrics across all evaluated models. Heatmap of normalized performance metrics across all evaluated models. Figure 11 shows the distribution of detection latency, measured by Mean Time to Detection (MTTD). KnitLLM exhibits both a lower median latency and a smaller variance, which can be attributed to the diffusion module used for data augmentation. By generating additional samples for rare fault cases, the model learns more reliable decision boundaries in the latent feature space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document}. As a result, it can distinguish short-term load fluctuations from true fault-related deviations more quickly than threshold-based methods or conventional recurrent models.Fig. 11Box plot of detection latency across multiple experimental runs. Box plot of detection latency across multiple experimental runs. Figure 12 presents the Precision-Recall (PR) curve. The shape of the KnitLLM curve suggests stronger robustness under industrial domain shift. In practical textile production, sensor characteristics may drift over long operating periods, but the semantic representation learned by KnitLLM helps preserve the identity of fault-related patterns even when raw signal amplitudes change. This allows the model to maintain high precision at relatively high recall, which is important for reducing unnecessary shutdowns while avoiding missed warp-breakage alarms.Fig. 12Precision-Recall curves of different models. Precision-Recall curves of different models. Figure 13 compares the AUC values with confidence intervals. The improvement in AUC indicates that the discrete semantic representation used by KnitLLM is effective in suppressing noisy components of the mechanical signal. Overall, the results suggest that KnitLLM not only performs fault classification, but also captures meaningful process-level relationships in warp-knitting operations, leading to stronger and more stable performance in industrial fault diagnosis.Fig. 13AUC-ROC comparison with 95% confidence intervals. AUC-ROC comparison with 95% confidence intervals. To further evaluate performance on faults without clear visual manifestations, we introduced a late-fusion baseline by averaging the prediction scores of the visual model and the time-series model. In addition, an ensemble baseline was implemented using soft voting over the outputs of both modalities. Table 4 These baselines provide a stronger comparison than single-modality models.Table 4Comparison with late-fusion and ensemble baselines.MethodPrecision (P)Recall (R)F1-ScoreAUCMTTD (ms)KnitLLM0.9680.9540.9610.985124Visual-only0.8940.7980.8430.901248Sensor-only0.8790.9050.8920.935135Late Fusion0.9180.9210.9190.953156Ensemble0.9240.9290.9260.957149 Table 4Comparison with late-fusion and ensemble baselines.MethodPrecision (P)Recall (R)F1-ScoreAUCMTTD (ms)KnitLLM0.9680.9540.9610.985124Visual-only0.8940.7980.8430.901248Sensor-only0.8790.9050.8920.935135Late Fusion0.9180.9210.9190.953156Ensemble0.9240.9290.9260.957149 Comparison with late-fusion and ensemble baselines. The superiority of KnitLLM stems from semantic alignment rather than late fusion. While late-fusion baselines combine modality-specific outputs, they cannot reconstruct latent process semantics or infer pre-fault deviations from incomplete observations. KnitLLM addresses this limitation by learning a discrete process grammar over aligned multimodal observations, which is especially beneficial for faults that have weak or delayed visual manifestations. Ablation study and modal sensitivity analysis To verify the superiority of the joint feature extraction operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{F}}_{\theta } \left( {{\boldsymbol{S}},{\boldsymbol{I}}} \right)$$\end{document} we designed three sets of ablation experiments: Full-Modal (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${S + I}$$\end{document}) represents standard KnitLLM receiving complete sensor time series and image inputs; Sensor-Only (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}$$\end{document}) masks allowing the model to rely solely on vectors for semantic abstraction; Image-Only (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}$$\end{document}) masks degrading the model to a pure visual discriminator. Table 5 presents the performance degradation under different modality configurations.Table 5Performance sensitivity of KnitLLM under different modality input configurations.Input configurationF1-ScoreRecall (R)Precision (P)MTTD (ms)VarianceFull-modal (S + I)0.9610.9540.9681240.021Sensor-only (S)0.8920.9050.8791350.045Image-only (I)0.8430.7980.8942480.082 Table 5Performance sensitivity of KnitLLM under different modality input configurations.Input configurationF1-ScoreRecall (R)Precision (P)MTTD (ms)VarianceFull-modal (S + I)0.9610.9540.9681240.021Sensor-only (S)0.8920.9050.8791350.045Image-only (I)0.8430.7980.8942480.082 Performance sensitivity of KnitLLM under different modality input configurations. Experimental results indicate that the sensor modality plays a more important role in this task. When the image input is removed, the Recall decreases only moderately from 0.954 to 0.905. This suggests that the sensor signals carry the main dynamic information of the warp-knitting process. In particular, faults such as bearing wear or main-shaft jitter often appear first as changes in the vibration \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{3}$$\end{document} and current \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{4}$$\end{document} channels, before they become visible in fabric images. These results also show that the structural prior operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}_{str}$$\end{document} is mainly driven by temporal information. Even without image input, the model can still determine whether the machine is in an abnormal state by relying on learned semantic patterns from the sensor sequence. By contrast, when the sensor input is removed, performance drops much more sharply, and MTTD increases to 248 ms. This is consistent with the fact that visual information is delayed: the model has to wait until physical damage appears on the fabric surface, such as gaps caused by warp breakage, before an anomaly can be detected. At that point, although the reconstruction residual δ can still indicate a fault, the detection time is later than the actual onset because the model no longer has access to the process dynamics. Visual features are useful for identifying clear defects such as oil stains, but they are much less effective for latent faults caused by tension fluctuations, which leads to more missed alarms. These results suggest that KnitLLM benefits from combining early sensor-based diagnosis with visual confirmation. When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{S}}$$\end{document} captures minor manifold deviations, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{I}}$$\end{document} serves as a spatial constraint to further confirm whether the deviation has evolved into fabric surface defects. This multimodal coupling not only improves \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-Score by approximately 7% to 12%, but also reduces δ discrimination variance to the minimum (0.021), greatly enhancing system robustness during ultra-long cycle operation of 10,000 h. To validate the theoretical analysis of semantic abstraction, we track the convergence of the quantization error \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{E}}_{q}$$\end{document} and reconstruction residual δ during training, as shown in Fig. 14. The convergence proves that codebook \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{C}}$$\end{document} can effectively learn and cover the physical state manifold \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{M}}$$\end{document} of warp knitting machines, while the convergence proves that the Transformer backbone successfully learns the structural prior \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}_{str}$$\end{document} in the weaving process, thereby accurately predicting missing mechanical semantics.Fig. 14Convergence analysis of core loss functions during training. Convergence analysis of core loss functions during training. To further examine the role of discretization in semantic learning, we compare KnitLLM (Discrete) with a Continuous-Baseline that directly uses continuous features. The discretization step is intended to suppress industrial background noise and extract higher-level semantics through fault primitives. Figure 15a compares the performance of the two methods using grouped bars, and Fig. 15b shows the residual distributions obtained for fault samples. In the Continuous-Baseline, the residual distributions of normal and faulty states overlap substantially because of random sensor fluctuations. By contrast, KnitLLM maps observations to discrete fault primitives through VQ-VAE, which acts as a nonlinear filtering process. As a result, faulty states appear as clearer outliers in semantic space, which helps reduce the false alarm rate (FAR).Fig. 15Effectiveness of discrete tokenization for semantic fault understanding. Effectiveness of discrete tokenization for semantic fault understanding. The masking ratio, denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r_{mask}$$\end{document}, affects the difficulty of learning structural priors. If the ratio is too small, the task becomes overly easy and the model may rely on shallow feature matching. If the ratio is too large, important semantic links are disrupted and reconstruction becomes difficult. Figure 16a shows the variation in performance under different masking ratios, and Fig. 16b illustrates the relationship between masking ratio and MTTD. The results show that performance reaches its best level when the masking ratio is close to 0.25. At this setting, the model must recover masked states through long-range dependencies, which helps it capture the relationship between needle-bed motion and yarn tension more effectively. When the ratio exceeds 0.45, the disruption of key causal links in the warp-knitting process leaves insufficient context for reconstruction, and the detection latency increases noticeably. This confirms that the structural prior operator depends strongly on information completeness Table 6.Fig. 16Impact of masking ratio \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r_{{{\mathrm{mask}}}}$$\end{document} on model performance and detection latency.Table 6Physical validity evaluation of reconstructed sensor signals.VariantVQ-VAE TokenizerMasked fault modelingDiffusion augmentationInterpretability moduleF1AUCFull KnitLLM✓✓✓✓0.9610.985w/o VQ-VAE✗✓✓✓0.9320.967w/o Masked Modeling✓✗✓✓0.9180.958w/o Diffusion✓✓✗✓0.9410.972w/o Interpretability✓✓✓✗0.9560.982 Impact of masking ratio \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r_{{{\mathrm{mask}}}}$$\end{document} on model performance and detection latency. Table 6Physical validity evaluation of reconstructed sensor signals.VariantVQ-VAE TokenizerMasked fault modelingDiffusion augmentationInterpretability moduleF1AUCFull KnitLLM✓✓✓✓0.9610.985w/o VQ-VAE✗✓✓✓0.9320.967w/o Masked Modeling✓✗✓✓0.9180.958w/o Diffusion✓✓✗✓0.9410.972w/o Interpretability✓✓✓✗0.9560.982 Physical validity evaluation of reconstructed sensor signals. The results indicate that each module contributes in a complementary manner: the tokenizer provides a structured discrete representation, masked modeling improves reasoning over incomplete observations, diffusion augmentation enhances robustness to rare faults, and the interpretability mechanism facilitates transparent diagnosis. Physical validity validation of diffusion-generated samples To address concerns about the physical plausibility of samples generated by the diffusion module, we further validate the synthesized signals through a dedicated experimental procedure. This validation is intended to confirm that the latent vectors correspond to physically meaningful process states rather than being limited to statistical regularities. Specifically, the augmented embeddings are first decoded back into the original sensor domain. The validation is conducted from two complementary perspectives. First, a physical simulation test is performed by feeding the reconstructed trajectories of yarn tension and main shaft rotational speed into a warp-knitting dynamics model, where their consistency with momentum conservation and the viscoelastic behavior of yarns is examined. Second, a blind expert assessment is carried out in which three senior textile machinery engineers independently evaluate the process rationality of the reconstructed signals on a five-point scale without knowing whether the samples are real or synthesized. The results indicate that the samples produced by KnitLLM exhibit stronger physical consistency than those generated by the conventional GAN-based augmentation method. As reported in Table 7, KnitLLM achieves a kinetic residual of 0.042, which is much lower than the 0.185 obtained by the GAN baseline, suggesting better agreement with the underlying physical dynamics. In addition, the mean expert score of KnitLLM reaches 4.72 out of 5, indicating that the reconstructed signals are highly consistent with actual production behavior. Frequency-domain analysis based on the FFT also shows that the spectral energy distribution of the generated signals closely matches that of real industrial fault samples, with a deviation of only 0.021. These results suggest that the diffusion head, conditioned on Transformer hidden states, is able to generate signals that preserve the periodic structure of the weaving process. For example, in simulated yarn-breakage cases, the reconstructed tension signal shows a sudden drop that remains synchronized with the main-shaft phase, rather than exhibiting random numerical drift. This confirms that the augmented samples in KnitLLM are physically meaningful and can improve the model’s generalization capability under extreme operating conditions.Table 7Physical validity evaluation of reconstructed sensor signals.DimensionAssessment methodKnitLLMBaseline (GAN)Physical meaningKinetic residualSimulation check0.0420.185Adherence to lawsProcess authenticityExpert score (Mean)4.723.15Logical consistencySpectral deviationFFT energy distribution0.0210.094Noise realism Table 7Physical validity evaluation of reconstructed sensor signals.DimensionAssessment methodKnitLLMBaseline (GAN)Physical meaningKinetic residualSimulation check0.0420.185Adherence to lawsProcess authenticityExpert score (Mean)4.723.15Logical consistencySpectral deviationFFT energy distribution0.0210.094Noise realism Physical validity evaluation of reconstructed sensor signals. Generalization test on novel faults To simulate the challenge posed by newly emerging faults in practical industrial environments, we extract a subset of samples from the dataset and introduce three previously unseen fault patterns that are excluded from the training stage through physical simulation: Sinker Wear, which appears as slight high-frequency sinusoidal perturbations in the tension signal \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{2}$$\end{document}; Jacquard Misalignment, which is characterized by phase decoupling between yarn delivery \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{5}$$\end{document} and RPM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{1}$$\end{document}; and Intermittent Lubrication Blockage, which manifests as a very slow nonlinear drift between oil temperature \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{6}$$\end{document} and current \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s_{4}$$\end{document}. KnitLLM does not depend on explicit fault category labels. Instead, its decision criterion is based on the reconstruction residual δ. In principle, any state that violates the physical structural prior \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}_{str}$$\end{document} will induce a substantial deviation on the manifold \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{M}}$$\end{document}. We compare the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-score of KnitLLM with two strong baseline models, TimesNet and PatchTST, under both known-fault and unknown-fault settings, as shown in Fig. 17a. The results indicate that when the models are confronted with unseen fault types, conventional SOTA methods such as TimesNet suffer a pronounced drop in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-score, decreasing from 0.93 to 0.72. This behavior suggests that such models primarily learn statistical feature distributions. Once a new fault changes the data distribution, their performance deteriorates because they do not capture the underlying operational logic of the machine. By contrast, KnitLLM maintains an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-score of 0.912 in the unknown-fault scenario. This stronger generalization ability can be attributed to the completeness of the MFM task: the model learns the physical grammar of the machine through masked modeling. Regardless of the specific form of a new fault, as long as it is physically inconsistent, it yields an unreconstructible δ residual in semantic space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{Z}}_{e}$$\end{document}.Fig. 17Comparison of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-score and t-SNE visualization of hidden representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{H}}$$\end{document} under known-fault and unknown-fault scenarios. Comparison of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1}$$\end{document}-score and t-SNE visualization of hidden representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{H}}$$\end{document} under known-fault and unknown-fault scenarios. To provide a more intuitive view of how the model separates normal states from multiple fault conditions, we apply t-SNE to the hidden representations \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\boldsymbol{H}}$$\end{document}, as illustrated in Fig. 17(b). The visualization reveals the internal behavior of KnitLLM. The green region corresponds to the normal manifold formed by coordinated operation of the main shaft, tension system, and needle bed. Known faults, marked by blue squares, form compact high-density clusters during training. Unknown faults, shown as red crosses, are more dispersed in the latent space, but they remain separated from the normal manifold by a clear physical margin. This observation supports the discriminative boundary induced by the structural prior operator \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal{P}}_{str}$$\end{document}. Because KnitLLM compresses the normal manifold into a narrow semantic envelope during training, any new pattern that violates physical constraints is projected outside the manifold. As a result, KnitLLM can identify potential new safety risks in complex workshop environments with a very low false alarm rate (FAR = 0.012). This form of invariance under changing fault conditions is a key advantage of KnitLLM in industrial intelligence applications. Discussion and conclusion Discussion The proposed KnitLLM framework delivers clear gains in warp knitting machine fault detection. As shown in Table 3, it achieves a precision of 0.968, recall of 0.954, and F1-score of 0.961, outperforming the strongest baseline, TimesNet, which obtains 0.938, 0.925, and 0.931, respectively. These results suggest that cross-modal attention is effective in capturing complementary relationships between sensor signals and fabric images. In particular, the VQ-VAE-based tokenizer appears to help the model extract more useful information from both temporal dynamics and spatial texture, which improves robustness under industrial operating conditions. Compared with previous studies, the main difference of our approach is the use of a semantic fault-primitive representation that adapts to the physical properties of the input data, rather than relying on fixed handcrafted or static feature extraction methods that may become less effective when the data distribution changes. Ablation experiments further confirm the contribution of each module. Removing the image modality reduces the F1-score from 0.961 to 0.892, while removing the sensor modality lowers it to 0.843, indicating that both modalities are useful, with sensor signals playing a particularly important role in early fault diagnosis. The sensor-only setting still achieves a recall of 0.905, only slightly lower than the full model, which suggests that temporal information is more informative than static visual appearance in this task. The results on Mean Time to Detection (MTTD) show a similar trend: image-only models reach 248 ms, whereas the full KnitLLM system requires only 124 ms, demonstrating the advantage of incorporating sensor-based process information for early warning. Compared with recent representative methods, KnitLLM also shows advantages in efficiency and robustness. Although the MFM module introduces additional computation, the use of discrete semantic abstraction keeps the overall complexity manageable. The model is slower than PatchTST in inference, but the difference is acceptable given the improvement in detection performance. In terms of reliability, KnitLLM achieves a FAR of 0.012, which is much lower than D-CNN’s 0.054. This improvement is likely due to the semantic primitive representation, which reduces sensitivity to electromagnetic noise and high-frequency vibration while preserving fault-related information. Even so, the model still struggles in extremely rare fault cases, such as main shaft bearing overheating with very few samples. This is consistent with the limitations of data-driven models in low-data settings and indicates that the diffusion-based augmentation strategy remains important. Conclusion This study proposes KnitLLM, a multimodal semantic learning framework for early fault diagnosis in warp knitting machines. By combining sensor signals and fabric images through a VQ-VAE tokenizer, masked fault modeling, and diffusion-based augmentation, the model can detect faults earlier and more accurately than conventional baselines. Experiments on a real industrial dataset show that KnitLLM achieves strong performance in precision, recall, F1-score, and detection latency, while also maintaining a low false alarm rate. Despite these results, several limitations remain. First, the experiments were conducted mainly on a single machine type, so further validation is needed across different warp knitting machines and production environments. Second, the model still requires considerable computational resources, which makes deployment on edge devices difficult. Third, although attention visualization provides some interpretability, the decision process is not yet fully transparent. Future work will focus on three directions. One is improving cross-device generalization through transfer learning or meta-learning. Another is developing lighter model architectures, for example by distilling the Transformer into smaller networks or using sparse attention mechanisms. The third direction is improving interpretability by incorporating causal analysis so that the relationship between sensor changes and fault decisions can be explained more clearly. In addition, building larger and more challenging benchmark datasets with device variation, missing modalities, and long-term degradation will help evaluate the method more thoroughly.