A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises

Kazi Rafat ([email protected]), Md. Ismail Hossain ([email protected]), M M Lutfe Elahi ([email protected]), Sifat Momen ([email protected]), Fuad Rahman ([email protected]), Nabeel Mohammed ([email protected]), Shafin Rahman ([email protected]), [4pt] Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · 2026

Read A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises by Kazi Rafat ([email protected]), Md. Ismail Hossain ([email protected]), M M Lutfe Elahi ([email protected]), Sifat Momen ([email protected]), Fuad Rahman ([email protected]), Nabeel Mohammed ([email protected]), Shafin Rahman ([email protected]), [4pt] Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30309.txt.

Introduction

Kazi Rafat and six colleagues grade rehabilitation exercises by computer from three-dimensional joint data, so patients can train at home. Correct form restores function after diseases like Parkinson's or back injury. Expert supervision is costly and often unavailable to patients. RGB-D cameras capture joint motion during home practice. Their system reads those point clouds directly.

A curve-based feature step enriches the point cloud information. Axial self-attention lets the model weigh important joints and their roles. Assessment then scores how well each routine was performed. The system beats existing approaches on three rehab datasets. They cover varied movements and patient groups. Small model size keeps inference fast on modest hardware. Generalization to similar routines sharing joints is also reported. The authors stress practical relevance for remote monitoring programs.

Catalog record

CREATORKazi Rafat ([email protected]), Md. Ismail Hossain ([email protected]), M M Lutfe Elahi ([email protected]), Sifat Momen ([email protected]), Fuad Rahman ([email protected]), Nabeel Mohammed ([email protected]), Shafin Rahman ([email protected]), [4pt] Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30309

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Point Cloud Transformer for Remote Monitoring and Automated Assessment of Physical Rehabilitation Exercises Kazi Rafat ([email protected]), Md. Ismail Hossain ([email protected]), M M Lutfe Elahi ([email protected]), Sifat Momen ([email protected]), Fuad Rahman ([email protected]), Nabeel Mohammed ([email protected]), Shafin Rahman ([email protected]), [4pt] Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh Abstract Rehabilitation exercises are essential in restoring lost physical functions of patients suffering from various diseases (e.g., Parkinson’s, back pain). Carrying out these rehabilitation exercises, often prescribed by health experts, is costly, unavailable, and requires expert supervision. The availability of RGBD images and movement/position data of joints along with expert annotation of exercise data has prompted the use of automatic assessment of the quality of rehabilitation exercises, which is cost-effective and can be carried out at home. However, existing approaches do not extract relevant features, lack practical application, require expensive pre-processing, or overlook crucial features. This study proposes a transformer-based framework for point clouds to extract features and assess rehabilitation exercises by analyzing joint positions collected through RGBD data. We adapt and utilize a curve-based point-cloud feature aggregation technique to augment point-cloud information that aids model output. The transformer architecture also uses axial self-attention, recognizing important joints and their roles to assist users in performing the exercise better. The guided system outperforms existing approaches and is also practically relevant due to its small size, fast inference, and generalization on specific joints in similar exercises. We conduct our experiments on three crucial baseline datasets for rehabilitation exercises: Kimore, UI-PRMD, and IRDS. I Introduction Physical therapy through rehabilitation exercises is effective in restoring lost functional abilities and rehabilitating patients suffering from different diseases (e.g., Parkinson’s, back pain) that cause physical inability. Prescribed rehabilitation exercises are only powerful when performed correctly. Patients usually learn different rehabilitation exercises from physical therapists or informative videos on the Internet. Most patients (about 90%) [ 1 ] practice these exercises at home without expert supervision. This affects exercise quality and effectiveness. Performing incorrect exercises can cause unwanted problems or further debilitate the patient. This also prevents patients from being motivated to continue or exercise regularly. In addition, physical therapy and rehabilitation appointments are costly and can put substantial financial burdens on patients and healthcare providers. The cost of physical rehabilitation programs in the US in 2007 was 13.5 billion dollars [ 2 ] . The lack of trained rehabilitation professionals also increased the cost of rehabilitation programs. Fig. 1: We visually illustrate popular techniques against our technique. (a) Processing and Hand-crafting features [ 3 , 4 , 5 ] require manual effort and introduce bias to the system. These techniques, at times, do not pay attention to important features. (b) Convolutional Neural Networks for assessing Rehabilitation [ 6 ] exercises fail to capture some important spatio-temporal information. (c) Spatio-Temporal Graph Neural Networks [ 7 , 8 ] make topologically similar images and cannot map some topological information. (d) Curve-based cloud aggregation ( Ours ) paired with axial attention can augment data points according to their position and pay attention to important information. Traditionally, patient progress in rehabilitation is assessed clinically through visual inspection, often based on comparisons with expert demonstrations or prior performances through qualitative annotations [ 9 ] . To reduce subjectivity and improve scalability, researchers have increasingly explored automated methods for assessing rehabilitation exercises. Early work [ 10 , 11 , 12 ] primarily focused on classifying exercises as correct or incorrect. However, such binary classification provides limited feedback and does not support monitoring gradual improvements over time. More recent approaches instead formulate exercise assessment as a continuous evaluation problem, using regression models to assign performance scores across a range of quality levels [ 6 , 7 ] . These methods typically rely on movement and body-position data collected from sensors. In parallel, researchers have developed robotic assistive systems [ 13 ] , haptic devices [ 14 ] , and virtual gaming environments [ 15 ] for exercise evaluation. However, such mechanical and electronic systems are often expensive and require specialized hardware, limiting their widespread adoption. Advancements in deep neural networks in classifying and interpreting vision tasks and their ease of use have made deep learning a frontrunner in the evaluation of physical rehabilitation tasks. The introduction of affordable motion and depth sensors, such as the Microsoft Kinect [ 16 ] , capable of tracking skeletal details, has spurred a growing interest in deep neural networks. These sensors can accurately record sequential joint positions or angles without lag. This high-frequency skeletal data allows for the differentiation between correctly executed exercise patterns and maladaptive exercise. In the context of rehabilitation, the ability to automatically distinguish these states allows for real-time feedback, ensuring that patients maintain the biomechanical alignment necessary for functional recovery and injury prevention, and allows the patient to continue and not leave physical therapy. However, the comprehensive body data collected by motion sensors is often excessive and not ideal for direct use in neural networks. Kinect sensor captures nieghbouring data points in the same depth which have the same information making them spatially redundant, noisy or incomplete [ 17 , 18 ] . These redundancies result in an excessively large input space that is computationally expensive to process and prone to overfitting, as the model may focus on noise rather than the underlying biomechanical movement patterns. Here, we summarize different strategies and discuss challenges in assessing exercise in physical rehabilitation. (1) Several studies [ 3 , 4 , 5 ] rely on handcrafted features that require experts to engineer features, which makes it costly, time-consuming and erroneous. Figure 1 a shows hand-crafted features from joint coordinates which are then used to train machine learning models. (2) Studies [ 6 ] also explored the use of computational deep learning, such as CNN, to extract features from joint positions in the body. The problem with CNNs as feature extractors is that they miss crucial spatial and temporal information in the skeleton data. Basic CNNs, Figure 1 b, are unable to capture long-scale spatial and temporal information due to lack of attention mechanisms (3) Studies [ 7 ] , [ 8 ] also used ST-GCNs (Spatio-Temporal Graph Convolution Networks) and LSTM to evaluate exercises. However, the graphs applied in ST-GCNs are predefined and are topologically similar across all frames that fail for different action classes and cannot map semantic information in all layers [ 19 ] . ST-GCNs cannot also map information such as bone lengths and directions, which are usually important. Existing approaches ignore joint importance or use self-attention to learn important joints, which is computationally inefficient [ 19 ] . Other studies [ 20 , 21 , 22 ] have also used modified GCNs with transformers and self-attention, but their performance was limited by the lack of ST-GCNs. Figure 1 c shows st-gcns that are unable to map different topologies. In this study, we investigate the use of transformer models that utilize curve-based point cloud analysis [ 23 ] with axial attention to perform a regression task on skeletal sequences. Specifically, the model predicts a quantitative assessment score (normalized between 0 and 1) representing the kinematic similarity between the patient’s performance and an expert’s execution. This predicted score serves as a direct objective metric for

Read the complete text