Diff-MomentFormer: Generative Diffusion-Augmented Transformer for End-to-End Joint Moment Estimation Authors: Chengyu Qiao, Eryun Liu, Jingwei Ren, Long He Published: 2026 ABSTRACT Accurate estimation of human joint moments from multimodal sensor signals is essential for lower-limb exoskeleton control. Recent studies have addressed this problem in an end-to-end manner, but remain limited by insufficient long-range temporal modeling, limited training data, and class imbalance. To address these issues, we propose Diff-MomentFormer, a generative diffusion-augmented Transformer framework for end-to-end joint moment estimation from multimodal wearable sensor signals. The framework integrates a classifier-free conditional diffusion model for activity-conditioned synthetic data generation, together with a Transformer-based regression network for modeling long-range temporal dependencies and cross-modal interactions. Through the combination of controllable data augmentation and global temporal modeling, Diff-MomentFormer can learn more robust multimodal representations for accurate and stable joint moment estimation. Extensive experiments on a public lower-limb biomechanics dataset show that the proposed method consistently improves hip and knee joint moment estimation performance across different activity categories, while ablation studies further confirm the effectiveness of the proposed framework. 1. Introduction With the rapid development of robotics, wearable sensing technologies, and intelligent control algorithms, lower-limb exoskeletons have shown great potential in human assistance [1,2,3,4,5], rehabilitation [6,7], and physical augmentation [8,9]. In industrial and physically demanding scenarios, they can provide assistance to reduce muscular load, improve operational efficiency, and lower the risk of musculoskeletal injury [3,10,11]. In rehabilitation and mobility support, they can facilitate motor recovery and help maintain long-term functional independence through precise biomechanical assistance [6,12]. These application demands highlight the importance of effective lower-limb exoskeleton control, particularly for enabling accurate, adaptive, and natural human–robot interaction. As a core component of lower-limb exoskeleton systems, control strategies have attracted extensive research attention [13,14]. Conventional controllers typically follow a hierarchical architecture [15], in which locomotion modes are first recognized at the high level [16,17,18,19,20,21], assistive torques are then generated at the mid-level using predefined rules or state-machine logic [5,22,23], and the resulting commands are finally executed by the low-level actuator controller. Although such pipelines have shown satisfactory performance in structured laboratory settings, they rely heavily on handcrafted rules and careful parameter tuning. As a result, their adaptability and generalization are often limited when facing diverse movement patterns, transition phases, and inter-subject variability. Recent studies have therefore increasingly explored data-driven approaches that estimate joint moments from multimodal sensor signals in an end-to-end manner [24,25,26,27,28,29,30], providing a promising route toward more continuous and subject-adaptive exoskeleton control. End-to-end joint moment estimation aims to directly predict lower-limb joint moments from multimodal time-series signals collected by exoskeleton-embedded sensors, such as inertial measurement units (IMUs), joint encoders, and plantar-pressure sensors [29,30]. During training, the models are supervised by ground-truth joint moments computed from inverse dynamics using motion-capture data and ground reaction force (GRF) measurements. Compared with conventional pipelines that depend on explicit locomotion-mode recognition or phase-based parameterization, this paradigm enables continuous and task-agnostic modeling of user biomechanics and provides a more direct basis for assistive torque generation. Existing studies have shown that temporal convolutional network (TCN)-based estimators can achieve accurate hip and knee joint moment prediction across diverse movement conditions and can be integrated into closed-loop exoskeleton control, demonstrating the practical promise of this direction. Despite this progress, two key challenges remain. First, current methods do not fully exploit the long-range temporal dependencies and cross-modal interactions inherent in multimodal exoskeleton sensor signals. Most existing methods rely on convolutional neural networks (CNNs) as the backbone. Although CNNs are effective at extracting local temporal patterns, they are less suited to modeling long-range dependencies and complex coupling relationships across sensor modalities. Second, the availability of high-quality labeled data remains limited. Ground-truth joint moments must typically be obtained through synchronized motion-capture and force measurements, which makes data collection costly and restricts the scale of usable training data. In addition, publicly available datasets often provide limited coverage of rare or highly dynamic activities, leading to class imbalance and reduced generalization on underrepresented motion types. Therefore, improving end-to-end joint moment estimation in real-world scenarios requires both stronger temporal and multimodal modeling at the network level and more effective data augmentation at the data level. Recent advances in Transformers and diffusion models provide a promising foundation for addressing these challenges. Transformer architectures [31] have demonstrated strong sequence modeling capability across a wide range of domains, largely because the self-attention mechanism enables global dependency modeling between any pair of positions in an input sequence. This property makes Transformers particularly suitable for capturing long-range temporal correlations and cross-modal interactions in multimodal exoskeleton sensor data. At the same time, diffusion models [32] have emerged as a powerful class of generative models for learning complex data distributions and synthesizing high-quality samples from random noise. In particular, classifier-free conditional guidance [33] enables controllable generation without requiring an auxiliary classifier, making diffusion models especially attractive for class-conditioned data augmentation and distribution balancing. Together, these advances suggest a natural way to improve end-to-end joint moment estimation by strengthening temporal modeling and enriching training data simultaneously. To address the above challenges, we propose Diff-MomentFormer, a generative diffusion-augmented Transformer framework for end-to-end joint moment estimation. Figure 1 illustrates the overall pipeline of the proposed framework. Specifically, we first train a classifier-free conditional diffusion model to learn class-conditioned multimodal temporal distributions and generate category-specific synthetic samples for data augmentation. The generated samples are then fused with real measurements to train a Transformer-based joint moment estimation network, which is designed to better capture long-range temporal dependencies and cross-modal interactions in multimodal sensor signals. Under this design, the diffusion module alleviates data scarcity and class imbalance, while the Transformer module enhances temporal modeling and multimodal feature fusion. Together, these two components enable more accurate and stable joint moment estimation, especially under complex movements and task transitions. The main contributions of this work are summarized as follows:We introduce conditional diffusion-based augmentation into the joint moment estimation pipeline, enabling class-conditioned and controllable generation of synthetic multimodal temporal data to alleviate data scarcity and class imbalance.We develop a Transformer-based end-to-end joint moment estimation network that more effectively captures long-range temporal dependencies and cross-modal feature interactions, thereby improving estimation stability and accuracy.We validate the proposed framework through extensive experiments on a public lower-limb biomechanics dataset, demonstrating consistent performance improvements over counterpart models without data augmentation or architectural improvements. We introduce conditional diffusion-based augmentation into the joint moment estimation pipeline, enabling class-conditioned and controllable generation of synthetic multimodal temporal data to alleviate data scarcity and class imbalance. We introduce conditional diffusion-based augmentation into the joint moment estimation pipeline, enabling class-conditioned and controllable generation of synthetic multimodal temporal data to alleviate data scarcity and class imbalance. We develop a Transformer-based end-to-end joint moment estimation network that more effectively captures long-range temporal dependencies and cross-modal feature interactions, thereby improving estimation stability and accuracy. We develop a Transformer-based end-to-end joint moment estimation network that more effectively captures long-range temporal dependencies and cross-modal feature interactions, thereby improving estimation stability and accuracy. We validate the proposed framework through extensive experiments on a public lower-limb biomechanics dataset, demonstrating consistent performance improvements over counterpart models without data augmentation or architectural improvements. We validate the proposed framework through extensive experiments on a public lower-limb biomechanics dataset, demonstrating consistent performance improvements over counterpart models without data augmentation or architectural improvements. 2. Related Work 2.1. Joint Moment Estimation In recent years, increasing attention has been devoted to data-driven joint moment estimation and its integration into exoskeleton control frameworks. Compared with conventional mode-dependent control strategies, these approaches reduce reliance on explicit locomotion classification and manual parameter tuning, thereby improving adaptability and usability in real-world scenarios. Gasparri et al. [25] designed an ankle exoskeleton controller by fitting the relationship between foot-force measurements and ankle moments during level walking, enabling adaptive and real-time torque modulation. Subsequent studies extended this idea to more complex tasks, including incline and decline walking, stair ascent and descent, and mixed terrain locomotion, showing promising energetic benefits and practical applicability for both able-bodied users and clinical populations [34,35,36]. In parallel, energy-shaping methods provided an alternative route for multi-joint assistance based on joint motion, offering a more unified control strategy across different lower-limb joints [24,37]. With the development of wearable sensing and deep learning, end-to-end estimation of biomechanical quantities from sensor measurements has become increasingly feasible. Dorschky et al. [28] demonstrated that CNNs could be used to estimate joint angles, joint moments, and ground reaction forces from inertial sensing data collected from simulated and experimental sources. Molinaro et al. [26] first introduced a temporal convolutional network (TCN)-based estimator for hip joint moments during multimodal overground ambulation. They systematically evaluated the model’s performance under diverse steady-state walking conditions. Subsequent work further investigated how temporal history affects delayed estimation and anticipation of sagittal-plane hip moments [27]. Building on these findings, they integrated TCN-based estimation with mid-level control strategies to build a unified framework for robotic hip exoskeleton assistance [29]. This framework achieved accurate moment estimation across multiple walking modes while reducing user metabolic cost. Later studies expanded the task set to a larger collection of 28 activities, including both cyclic and unstructured motions, and achieved improved task-agnostic estimation and control of multi-joint moments such as hip and knee moments [30]. To reduce the burden of collecting high-quality labeled data, deep domain adaptation has been used to transfer open-source biomechanics data to device-specific sensor domains, thereby improving model performance while reducing the demand for costly labeled measurements on the target device [38]. Overall, these studies have made substantial progress in dataset construction, in-the-loop validation, and real-world relevance. However, most existing approaches still rely on convolutional temporal backbones, which are less effective at modeling long-range dependencies and cross-modal interactions in multimodal exoskeleton signals. Moreover, under limited access to labeled data, it remains underexplored how to learn motion-category-conditioned temporal distributions and generate diverse, high-continuity multimodal data for effective augmentation and class balancing. Motivated by these limitations, the present work focuses on enhancing temporal modeling and introducing more effective generative data augmentation to develop a more generalizable and stable joint moment estimation framework for complex movements and task transitions. 2.2. Transformer Transformers have emerged as a powerful backbone for sequence modeling across a wide range of deep learning tasks. Their core self-attention mechanism enables flexible and effective modeling of long-range dependencies, while also showing strong scalability as data and model sizes increase. In natural language processing, the Transformer architecture demonstrated breakthrough performance in machine translation [31] and later drove the development of large-scale pretrained language models [39,40]. In computer vision, Vision Transformer (ViT) [41] demonstrated that images can be represented as patch sequences and modeled effectively with pure Transformer architectures, while Detection Transformer (DETR) [42] reformulated object detection as a set prediction problem within an end-to-end Transformer encoder–decoder framework. In speech processing, Transformers have also been widely adopted for sequence modeling, where self-attention helps capture long-range acoustic context and can be combined with convolutional modules to jointly model local and global temporal structure [43,44]. More recently, Transformers have been incorporated into generative modeling, for example in Diffusion Transformer (DiT) [45] and Latent Diffusion Models (LDM) [46], further highlighting their flexibility and scalability. Motivated by these advances, Transformers provide a natural solution for modeling complex multimodal time-series data. In particular, their ability to capture global temporal dependencies and cross-modal interactions makes them well-suited for exoskeleton sensor signals, which exhibit strong temporal coupling and inter-sensor correlations. Therefore, in this work, we adopt a Transformer-based architecture to enhance temporal modeling and multimodal feature fusion for end-to-end joint moment estimation. 2.3. Diffusion Models Diffusion models have achieved remarkable progress in generative modeling in recent years, showing strong capability in distribution learning and sample synthesis. Ho et al. [32] first established diffusion probabilistic models as a powerful framework for high-fidelity image synthesis, and subsequent developments such as the Denoising Diffusion Implicit Model (DDIM) improved sampling efficiency by introducing a non-Markovian reverse process while preserving the original training objective [47]. For conditional generation, classifier guidance was proposed to steer the reverse process using gradients from an auxiliary classifier [48]. Classifier-free guidance later provided a unified framework that jointly learns conditional and unconditional generation and enables a practical trade-off between sample quality and diversity without requiring an additional classifier [33]. Building on these developments, diffusion models have been extended to a broad range of conditional and image-to-image tasks, including text-guided synthesis [49], image restoration [50], and realistic image editing [51]. Diffusion models have also shown strong potential in one-dimensional signal generation and speech synthesis, as demonstrated by DiffWave for raw audio waveform generation [52]. However, compared with image and speech applications, diffusion-based generation of multimodal physical-world time-series signals remains relatively underexplored. In such scenarios, generated signals must not only preserve temporal continuity within each channel, but also maintain strict synchronization and coupling relationships across different sensor modalities. This makes multimodal biomechanical sensing a particularly challenging yet promising application domain for diffusion models. In this work, we investigate the use of diffusion models for multimodal temporal data generation and augmentation in exoskeleton-related sensing scenarios, with the goal of enriching the training set, alleviating class imbalance, and improving the robustness of downstream end-to-end joint moment estimation. 3. Method As illustrated in Figure 2, Diff-MomentFormer consists of two core modules: (i) a classifier-free conditional diffusion model for data augmentation and (ii) a Transformer-based joint moment estimation module. Given a set of variable-length multimodal motion sequences S={si}i=1M, where si denotes the i-th raw multimodal sequence formed by the multimodal sensor signals and the corresponding ground-truth joint-moment sequence, we first preprocess the raw data and convert them into a fixed-length segment set X={xi}i=1N. A classifier-free conditional diffusion model is then trained on X to learn the class-conditioned distribution of multimodal temporal segments and generate synthetic segment samples X′ under motion-class conditions. In the subsequent joint moment estimation module, real and synthetic samples are fused to train a Transformer-based regression network, which captures long-range temporal dependencies and cross-modal interactions in multimodal sensor signals and outputs the corresponding joint moment predictions. The following sections provide detailed descriptions of the preprocessing, classifier-free conditional diffusion for data augmentation, Transformer-based joint moment estimation, and the corresponding loss function. 3.1. Multimodal Time-Series Preprocessing and Representation Due to variations in recording duration across subjects and trials, the raw data consists of multiple variable-length sequences. To obtain a unified representation for subsequent modeling, each raw sequence is converted into a set of fixed-length temporal segments, which are further normalized along the feature dimension. This preprocessing step reduces scale differences across sensor channels and inter-subject variability, providing a consistent input space for both diffusion-based data generation and Transformer-based joint moment estimation. Specifically, the raw input at each time step includes IMU measurements from the foot, shank, and thigh, plantar-pressure signals, and encoder readings from the hip and knee joints. These multimodal sensor signals are concatenated along the feature dimension and paired with the corresponding ground-truth joint moments to form the raw sample set, denoted as S={si}i=1M, where si is the i-th raw multimodal motion sequence. In addition, the subject’s body weight is repeated over all time steps and concatenated with the sequence features as an auxiliary input. Each variable-length sequence is then sequentially segmented into fixed-length temporal windows of length Ts, yielding a segment set X={xi}i=1N, where xi∈RTs×D denotes the i-th segment containing both the multimodal sensor signals and the aligned joint-moment sequence, and D is the total feature dimension of the joint representation. After segmentation, for the conditional diffusion model only, we apply dimension-wise min-max normalization to each segment. This design is adopted because the diffusion model requires inputs in the range [−1,1], where the value ranges of multimodal signals vary substantially across motion classes. Using a single global minimum and maximum for all classes would compress the dynamic range of some categories, making it difficult to preserve their characteristic temporal patterns. Therefore, for each motion class c and each feature dimension d∈{1,…,D}, the minimum and maximum values are computed from the training samples of class c, denoted as mc,dmin and mc,dmax, respectively. For any segment xi belonging to class c, the normalized representation x¯i is defined as(1)x¯i(t,d)=xi(t,d)−mc,dminmc,dmax−mc,dmin,t=1,…,TsAfter this transformation, each feature dimension is normalized to [0,1] and then linearly rescaled to [−1,1] before being used as the clean sample for the diffusion model, which facilitates stable optimization and subsequent class-conditioned temporal generation. 3.2. Data Augmentation via Conditional Diffusion Models 3.2.1. Diffusion Preliminaries Diffusion models learn the underlying data distribution through a forward noising process and a corresponding reverse denoising process. In the forward process, Gaussian noise is gradually added to a clean sample x0∼q(x0) through a Markov chain of length T, resulting in a sequence of latent variables x1,x2,…,xT. When T is sufficiently large, the terminal state xT approaches a standard Gaussian distribution. The forward process is defined as(2)q(xt∣xt−1)=Nxt;αtxt−1,(1−αt)I where {αt}t=1T denotes a predefined noise schedule. The generative objective is to learn a parameterized reverse process pθ(xt−1∣xt) that progressively removes noise and reconstructs samples from the data distribution. A common practice is to train a neural network ϵθ(xt,t) to predict the noise added at diffusion step t. Under this parameterization, the training objective can be simplified to a mean-squared error loss:(3)L=Ex0,t,ϵ∥ϵ−ϵθ(xt,t)∥22 where ϵ∼N(0,I) and t is uniformly sampled from {1,…,T}. After training, new samples can be generated by initializing the reverse process with Gaussian noise xT∼N(0,I) and iteratively applying the learned denoising transitions to obtain x0. 3.2.2. Conditional Diffusion Modeling To enable controllable generation of specific motion patterns, we extend the basic diffusion formulation to a class-conditioned setting and model the conditional distribution p(x∣c), where c denotes the motion class label. Given an original sample x0∈RTs×D, the corresponding noisy sample at diffusion step t is constructed as follows:(4)xt=α¯tx0+1−α¯tϵ where α¯t=∏s=1tαs is the cumulative noise schedule and ϵ∼N(0,I) is standard Gaussian noise. To incorporate class information, we adopt a classifier-free conditioning strategy with a learnable class embedding layer. Let E∈RK×d denote the embedding matrix, where K is the number of motion classes and d is the embedding dimension. For a class label c, the corresponding class embedding is defined as follows:(5)embcls(c)=E[c] where E[c] denotes the c-th row of the embedding matrix. We also introduce a learnable null embedding embnull∈Rd to represent the unconditional branch. During training, a gating variable g∼Bernoulli(p) is sampled to randomly switch between the conditional and unconditional inputs, yielding the final condition embedding:(6)embc=g·embcls(c)+(1−g)·embnullThis design allows the model to be trained jointly under both conditional and unconditional settings, which enables classifier-free guidance during inference. To encode diffusion-step information, we adopt a one-dimensional sinusoidal positional encoding for the time embedding embt, defined as follows:(7)PE(t,i)=sin(t·wk),ifi=2k,cos(t·wk),ifi=2k+1; where wk = 1/10,0002k/d and d is the encoding dimension. The class embedding embc is projected through a linear layer to match the hidden feature dimension of the denoising network, and together with embt, serves as the conditioning input for noise prediction. In the reverse denoising process, the network predicts the noise component under both temporal and class conditions:(8)ϵ^=ϵθ(xt,t,c) where ϵθ(·) denotes the parameterized denoising network. We implement it as an encoder-decoder U-Net. The encoder, built with stacked ResNet blocks and linear attention modules, progressively downsamples the input to enlarge the receptive field and capture temporal dependencies. The decoder further upsamples the latent representation and fuses it with multi-scale encoder features through skip connections under the guidance of conditioning embeddings to produce the final noise prediction ϵ^. 3.2.3. Temporal Data Generation with Classifier-Free Guidance During inference, we employ classifier-free guidance to generate multimodal temporal segments conditioned on a specified motion class. The reverse process is initialized from Gaussian noise xT∼N(0,I) and proceeds iteratively from t=T to t=1. At each diffusion step, the denoising network produces both a conditional noise prediction ϵ^θ(xt,t,c) and an unconditional prediction ϵ^θ(xt,t). The difference between these predictions is used to construct the guidance term:(9)Δϵ^=ϵ^θ(xt,t,c)−ϵ^θ(xt,t)The final guided noise estimate is given by the following:(10)ϵ^t=ϵ^θ(xt,t)+s·Δϵ^ where s denotes the guidance scale. Based on this guided noise estimate ϵ^t, an estimate of the clean sample x^0 can be obtained as follows:(11)x^0=xt−1−α¯tϵ^tα¯t where α¯t denotes the cumulative diffusion coefficient at step t. To improve sampling stability and prevent extreme values, x^0 is clipped to the interval [−1,1]. Using the clipped estimate x^0, together with the current noisy sample xt and the diffusion coefficients at step t, we compute the mean and variance of the reverse posterior distribution q(xt−1∣xt,x^0). The posterior mean μt is expressed as follows:(12)μt=α¯t−1βt1−α¯t·x^0+αt1−α¯t−11−α¯t·xt where βt=1−αt is the variance schedule coefficient. The corresponding posterior variance σt2 is given by the following:(13)σt2=1−α¯t−11−α¯tβtThe sample z∼N(0,I) at the previous step is then obtained by the following:(14)xt−1=μt+σt·zAt the final reverse step, we set z = 0 and directly use the posterior mean as the generated sample x0. To exploit the generated data for training, the synthetic temporal segments are combined with real samples to construct an augmented dataset. Specifically, for each motion class c, synthetic segments containing both the sensor sequence and the corresponding joint-moment target are generated according to a predefined augmentation ratio ρ and merged with the corresponding real segments. This class-wise augmentation strategy increases data diversity and improves category balance while preserving the underlying structure of the real multimodal temporal distribution. 3.3. Transformer-Based Joint Moment Estimation Given the real and synthesized multimodal temporal segments, we employ a Transformer encoder-based regression network for end-to-end joint moment estimation. By leveraging multi-head self-attention, the network can model long-range temporal dependencies across the input window and capture interactions among heterogeneous sensor modalities, enabling joint prediction of hip and knee moments from multimodal time-series data. Given a temporal segment, either a real sample x or a synthetic sample x′, we partition it along the feature dimension into a multimodal sensor sequence xs∈RTs×Ds and the corresponding joint-moment sequence xm∈RTs×2. We extract a cropped sub-segment x˜s∈RTc×Ds as the input to the regression network and obtain the aligned ground-truth joint-moment sequence {mi*}i=1Tc⊆R2 from xm, where Tc