An upper limb stroke rehabilitation exercise video dataset Authors: S. Nandana, M. Thenmozhi Dharshini, Vismaya Viswanathan, Binoy B. Nair Published: 2026 ABSTRACT Stroke is one of the leading causes of disability worldwide with a disproportionately high burden in low and middle-income countries. In such countries, limited access to rehabilitation centres, shortage of trained physiotherapists and socioeconomic constraints impede continuous post-stroke care resulting in poorer recovery. Telerehabilitation has emerged as a scalable and cost-effective solution enabling remote monitoring and therapy personalization. Robust vision-based rehabilitation exercise assessment systems can play a significant role in such telerehabilitation programs. This article presents a well-annotated, exercise-specific video dataset consisting of 491 videos representing four distinct upper limb muscular strengthening exercises recommended for home rehabilitation performed by ten volunteers with the data captured using conventional RGB cameras at 30 frames per second under different background and lighting conditions. This dataset can be used to develop deep learning based low-cost telerehabilitation systems enabling availability of better post-stroke care for the patients from economically disadvantaged backgrounds. Specifications TableSubjectComputer SciencesSpecific subject areaDeep learning, computer visionType of dataVideo, Format: MP4, Frame Rate: 30 frame per secondData collectionThe dataset was collected by recording four hundred and ninety-one videos of volunteers performing four distinct upper limb muscular strengthening exercises designed for self-rehabilitation of hemiplegic patients. Each video represents one exercise cycle from start of the rehabilitation exercise to end of the exercise. The dataset was collected using ten volunteers that included seven female and three male participants, ensuring diversity in motor abilities. Each video captured variations in movement capability, range of motion, execution, and task success. The recorded data was manually reviewed and categorized into two groups: ‘complete’, where volunteers successfully performed the exercises, and ‘incomplete’, where they experienced difficulty in completing the exercise or left the exercise incomplete. No specialized motion capture hardware was used, making the dataset suitable for building AI models that can be deployed on low-cost camera-based edge devices. By leveraging this dataset, AI-driven telerehabilitation systems can be developed to monitor patient progress, deliver personalized therapy, and support stroke recovery.Data source locationAmrita School of Engineering, Coimbatore, Amrita Vishwa Vidyapeetham, India.Data accessibilityRepository name: Mendeley DataData identification number: doi: 10.17632/49h9dcwx5v.1Direct URL to data: https://data.mendeley.com/datasets/49h9dcwx5v/1The dataset is publicly available through Mendeley Data for use under CC BY 4.0 LicenseRelated research articleNone SubjectComputer SciencesSpecific subject areaDeep learning, computer visionType of dataVideo, Format: MP4, Frame Rate: 30 frame per secondData collectionThe dataset was collected by recording four hundred and ninety-one videos of volunteers performing four distinct upper limb muscular strengthening exercises designed for self-rehabilitation of hemiplegic patients. Each video represents one exercise cycle from start of the rehabilitation exercise to end of the exercise. The dataset was collected using ten volunteers that included seven female and three male participants, ensuring diversity in motor abilities. Each video captured variations in movement capability, range of motion, execution, and task success. The recorded data was manually reviewed and categorized into two groups: ‘complete’, where volunteers successfully performed the exercises, and ‘incomplete’, where they experienced difficulty in completing the exercise or left the exercise incomplete. No specialized motion capture hardware was used, making the dataset suitable for building AI models that can be deployed on low-cost camera-based edge devices. By leveraging this dataset, AI-driven telerehabilitation systems can be developed to monitor patient progress, deliver personalized therapy, and support stroke recovery.Data source locationAmrita School of Engineering, Coimbatore, Amrita Vishwa Vidyapeetham, India.Data accessibilityRepository name: Mendeley DataData identification number: doi: 10.17632/49h9dcwx5v.1Direct URL to data: https://data.mendeley.com/datasets/49h9dcwx5v/1The dataset is publicly available through Mendeley Data for use under CC BY 4.0 LicenseRelated research articleNone Value of the Data •Telerehabilitation systems using computer vision and deep learning can play an important role in enhancing the recovery for hemiplegic patients. This dataset consists of 491 upper limb rehabilitation exercise videos corresponding to all the upper limb muscular strengthening exercises as recommended in [1]. The rehabilitation exercises recommended in [1] are designed for home-based therapy without specialized equipment.•The dataset contains contributions from both male and female volunteers collected under different lighting conditions and backgrounds, thus providing a rich source for training adaptive and resilient rehabilitation systems. This diversity guarantees inclusivity in therapy use.•This labeled video dataset can be used in computer vision-driven telerehabilitation programs for remote monitoring and instruction, enabling clinicians to evaluate and modify rehabilitation regimens for patients via virtual consultations. •Telerehabilitation systems using computer vision and deep learning can play an important role in enhancing the recovery for hemiplegic patients. This dataset consists of 491 upper limb rehabilitation exercise videos corresponding to all the upper limb muscular strengthening exercises as recommended in [1]. The rehabilitation exercises recommended in [1] are designed for home-based therapy without specialized equipment. Telerehabilitation systems using computer vision and deep learning can play an important role in enhancing the recovery for hemiplegic patients. This dataset consists of 491 upper limb rehabilitation exercise videos corresponding to all the upper limb muscular strengthening exercises as recommended in [1]. The rehabilitation exercises recommended in [1] are designed for home-based therapy without specialized equipment. •The dataset contains contributions from both male and female volunteers collected under different lighting conditions and backgrounds, thus providing a rich source for training adaptive and resilient rehabilitation systems. This diversity guarantees inclusivity in therapy use. The dataset contains contributions from both male and female volunteers collected under different lighting conditions and backgrounds, thus providing a rich source for training adaptive and resilient rehabilitation systems. This diversity guarantees inclusivity in therapy use. •This labeled video dataset can be used in computer vision-driven telerehabilitation programs for remote monitoring and instruction, enabling clinicians to evaluate and modify rehabilitation regimens for patients via virtual consultations. This labeled video dataset can be used in computer vision-driven telerehabilitation programs for remote monitoring and instruction, enabling clinicians to evaluate and modify rehabilitation regimens for patients via virtual consultations. Background Stroke is a major cause of death and disability across the world [2], with low and middle-income countries [3] such as India being the most affected. It is also observed that in case of countries like India, rural population which has limited access to healthcare facilities are at a higher risk [4,5]. It has also been reported [6] that female stroke survivors tend to have poorer outcomes in terms of disability-adjusted life years (DALYs) compared to males. Stroke can lead to physical impairments such as motor dysfunction [7] and cognitive impairments such as memory loss and aphasia [8]. Hemiplegia resulting from stroke typically causes loss of motor control on one side of the body and limits the ability to perform daily activities. Rehabilitation process commonly includes upper limb strengthening exercises to improve muscle coordination, restore movement, and aid recovery. Functional tasks that mimic activities of daily living (ADLs) are widely used in neurorehabilitation to promote recovery through repetitive, goal-directed movement practice. Even though dedicated stroke centre-based care [9] and traditional-medicine based rehabilitation [10] can significantly improve post-stroke outcomes, the expenses associated with attending regular rehabilitation therapy sessions [11] and the lack of availability of trained professionals [11]. Home-based telerehabilitation can be helpful in such scenarios, allowing the medical professionals to monitor the progress of multiple patients remotely at a fraction of the cost of in-person visits to the rehabilitation centres. Recent developments in computer vision and machine learning have made it possible to evaluate rehabilitation exercises automatically, making low-cost telerehabilitation feasible. Sensor based telerehabilitation systems [[12], [13], [14]] require sensors to be worn on the limb at pre-defined positions, making it potentially uncomfortable for some users. Camera based systems appear to alleviate this issue [15] since there is no physical contact with the patient, making the rehabilitation exercises easier to carry out. However, as can be seen from the survey of datasets presented in Table 1, the datasets available, tend to employ costly data acquisition sensors such as Kinect [[16], [17], [18]] making the entire telerehabilitation setup expensive and unaffordable for patients from economically disadvantaged backgrounds. The goal of creating this dataset was to enable development of deep learning based low-cost exercise status identification systems that might eventually be deployable on low-cost edge devices such as a single board computer or a mobile phone. Use of low-cost and easily available cameras for detecting and monitoring the exercises also serves to keep the overall system costs down.Table 1Summary of publicly available rehabilitations datasets.Table 1 dummy alt text`DatasetRehabilitation DomainExercises PerformedParticipants Female/ MaleType of dataSensorsData AvailabilityToronto Rehab Stroke Posture [16]Upper limb robotic rehabilitationReach Forward/Backward, Reach Side to Side10/93-D joint positions in world coordinateKinectYesUI-PRMD [17]Upper and Lower LimbDeep squat; Hurdle step; Lunges: Inline, Side; Sit to stand, Exercises while standing: active standing, straight leg raise, shoulder abduction, shoulder extension, shoulder internal-external rotation, shoulder scaption.5/5Positions and Angles valuesVicon optical tracker, KinectNot available via link in paperKIMORE [19]Back Pain Rehabilitation (Upper and Lower Limb)Lifting of the arms, Lateral tilt of the trunk with the arms in extension, Trunk rotation, Pelvis rotations on the transverse plane, Squatting.34/44RGB, depth videos, and skeleton joint positionsRGB-D sensorNot available via link in paperIRDS [18]Upper and Lower LimbElbow Flexion: Left, Right; Shoulder Flexion: Left, Right, Shoulder Abduction: Left, Right; Shoulder Forward Elevation; Side tap: Left, Right11/183D coordinates of 25 joints, raw depth map ImagesKinect OneYesDUO-GAIT [13]Lower LimbWalking; walking while performing a cognitive task in unfatigued and fatigued states. Duration: 6 min.8/8Tri-axial acceleration, angular velocityIMUsYes[14]Lower LimbWalking. distance: 60 - 200 m3/7Tri-axial acceleration, angular velocityIMUsYesProposed Dataset [20]Upper limbLifting an object, Extending the Elbow, Extending the Wrist, Opening the Hand7/3RGB VideoWebcam, Mobile CameraYes Table 1Summary of publicly available rehabilitations datasets.Table 1 dummy alt text`DatasetRehabilitation DomainExercises PerformedParticipants Female/ MaleType of dataSensorsData AvailabilityToronto Rehab Stroke Posture [16]Upper limb robotic rehabilitationReach Forward/Backward, Reach Side to Side10/93-D joint positions in world coordinateKinectYesUI-PRMD [17]Upper and Lower LimbDeep squat; Hurdle step; Lunges: Inline, Side; Sit to stand, Exercises while standing: active standing, straight leg raise, shoulder abduction, shoulder extension, shoulder internal-external rotation, shoulder scaption.5/5Positions and Angles valuesVicon optical tracker, KinectNot available via link in paperKIMORE [19]Back Pain Rehabilitation (Upper and Lower Limb)Lifting of the arms, Lateral tilt of the trunk with the arms in extension, Trunk rotation, Pelvis rotations on the transverse plane, Squatting.34/44RGB, depth videos, and skeleton joint positionsRGB-D sensorNot available via link in paperIRDS [18]Upper and Lower LimbElbow Flexion: Left, Right; Shoulder Flexion: Left, Right, Shoulder Abduction: Left, Right; Shoulder Forward Elevation; Side tap: Left, Right11/183D coordinates of 25 joints, raw depth map ImagesKinect OneYesDUO-GAIT [13]Lower LimbWalking; walking while performing a cognitive task in unfatigued and fatigued states. Duration: 6 min.8/8Tri-axial acceleration, angular velocityIMUsYes[14]Lower LimbWalking. distance: 60 - 200 m3/7Tri-axial acceleration, angular velocityIMUsYesProposed Dataset [20]Upper limbLifting an object, Extending the Elbow, Extending the Wrist, Opening the Hand7/3RGB VideoWebcam, Mobile CameraYes Summary of publicly available rehabilitations datasets. This paper presents a comprehensive video dataset with four hundred and ninety-one (491) videos covering all the four upper limb muscular strengthening exercises recommended in [1] for home-based stroke rehabilitation. The exercises considered are: (a) Exercise 1- Lifting an Object, (b) Exercise 2- Extending the Elbow, (c) Exercise 3- Lifting the Wrist and (d) Exercise 4- Opening the Hand. Visualization of these exercises is presented in Table 2.Table 2Upper limb rehabilitation exercises.Table 2 dummy alt text Table 2Upper limb rehabilitation exercises.Table 2 dummy alt text Upper limb rehabilitation exercises. The exercises considered in this dataset and their relevance to upper limb rehabilitation are described below: Exercise 1- Lifting an Object: This is a task-oriented rehabilitation activity designed to restore upper-limb motor function, coordination, and muscle strength in individuals recovering from stroke. In this exercise, the patient begins by grasping a weighted object (e.g., a filled bottle) with both hands while sitting at a table. This bilateral grasp stabilizes the affected limb and reduces compensatory movement. This also helps patients initiate controlled movement of shoulders and elbow. The subsequent lifting motion activates multiple muscle groups in the shoulder, biceps, wrist and hand that are responsible for grip control. Once the object has been lifted to the topmost point possible, the object is then slowly lowered down to the table, thereby completing the exercise. This object lowering phase is important as it requires precise modulation of force and joint positioning thereby helping improve motor control and coordination. An example of performance of Exercise 1 is illustrated in Table 2. Exercise 2- Extending the Elbow: Stroke patients commonly exhibit reduced voluntary control and spasticity in the flexor muscles of the arm. This leads to difficulty in extending the elbow and performing reaching movements. In the second exercise considered here, the patient begins by placing the elbow on a soft surface (such as a soft cloth that can also allow for sliding movement) positioned on a table while seated. It must be ensured that the forearm is supported and the shoulder remains relatively stable. Now, the patient gradually extends the elbow. This movement helps train the triceps muscles responsible for elbow extension. This exercise also engages shoulder and forearm muscles. As the patient extends the elbow, extensor muscles are activated while inhibiting abnormal flexor dominance. The exercise also tends to improve joint range of motion and endurance. An example of performance of Exercise 2 is illustrated in Table 2. Exercise 3- Lifting the Wrist: In the third exercise considered here, the patient places the forearm on a slightly raised support (e.g. on a thick book or a shoe box etc.) with the wrist positioned at the edge so that the hand hangs freely downward. This positioning stabilizes the forearm and isolates movement primarily at the wrist joint. The patient is then instructed to raise the hand upward by extending the wrist while keeping the forearm in contact with the support surface. The process is illustrated in Table 2. This movement primarily activates the wrist extensor muscles. Repeated execution of this motion promotes strengthening of the wrist extensors and improved coordination between the forearm and hand. Exercise 4- Opening the Hand: This Exercise is designed to improve hand function in post-stroke patients. In this exercise, the patient positions the forearm on an elevated support surface (e.g. a thick book or a shoe box etc.) with the hand extending beyond the edge, allowing the wrist and fingers to move freely without restriction (The exercise is illustrated in Table 2 for easier visualization). From an initial position in which the hand may be partially or fully closed, the patient gradually opens his/her hand by extending the fingers and spreading them as much as possible while maintaining the wrist in an extended position. This movement is used to train the finger extensor muscles as well as intrinsic hand muscles. The exercise helps improve voluntary finger control. Data Description The Rehabilitation Dataset presented here includes four hundred and ninety-one labeled videos of upper-limb muscular strengthening rehabilitation exercises, carried out as per the procedure recommended in [1]. The dataset includes recordings from 10 healthy, able-bodied volunteers in the age group 18–29 years as per the age groups considered in [4]. Both male and female volunteers have participated allowing us to show variations in motor skills and movement patterns. These variations are important for analyzing movement quality and functional ability. The dataset was collected under different lighting conditions and backgrounds, reflecting variations observed in real-world conditions. In addition, different props were used by different volunteers for carrying out the same exercise, further adding diversity to the dataset and enabling the development of more robust deep learning models. The dataset consisting of 491 videos is divided across four exercises, with an average of about 122 videos per exercise, as listed in Table 5. The data were recorded using a smartphone (Samsung Galaxy A13) camera and an HP True Vision 720p FHD camera. Both devices are consumer-grade cameras capable of capturing images at 30 FPS. These devices vary in terms of the camera sensor characteristics such as sensor noise profile, focal length and field of view. The variations in focal length and field of view can affect the apparent scale of the upper limb parts in the video frames. The variations in sensor noise profiles and the image processing pipelines can affect the pixel intensity distributions and texture representations, thereby influence feature extraction, for example, when Convolutional Neural Network (CNN) based deep feature extractors are employed. The data captured using such heterogeneous devices can improve the robustness and generalization capability of deep learning models as they incorporate the variations in imaging conditions that a typical field-deployed model is likely to encounter. This variability also reduces the likelihood of the trained model overfitting to a specific imaging condition. This is particularly relevant for telerehabilitation applications where the patient may use different hardware platforms such as smartphones or laptops in home settings. Each of the four exercises considered in this study is grouped into two distinct sub-categories: Complete and Incomplete based on if the video recorded is of the exercise being completed successfully (indicating good functionality of the limb) or if it was left incomplete (simulated by the volunteers as inability to complete the exercise till the end of video recording, indicating incomplete recovery from the stroke episode). The upper limb rehabilitation exercises are illustrated in Table 2. Four representative sample frames from one randomly selected video in the dataset for each case is presented. The frames shown in Table 2 should be read from left to right with the leftmost frame extracted from the starting part of the video representing the start of exercise and the rightmost frame in the same row of Table 2, extracted from the last few seconds of the video representing the end of the exercise. The two intermediate frames in each row illustrate the progression of the exercise. Exercise 1- Lifting an Object: The activity starts at the Exercise Start stage (as given in Table 2) where the hand rests on a flat surface while holding the object. During the Exercise in Progress stages, the volunteer lifts the object upwards in a smooth and controlled manner. The activity ends at the Exercise End stage, where the object reaches the final intended position. For successful completion of Exercise 1, the object is lifted steadily throughout all the stages and reaches the Exercise End position without stopping or losing the grip, as shown in Table 2. For incomplete execution of Exercise 1, the activity will not be completed or will end before reaching the Exercise End position due to decreased grip strength, limited motion, loss of control, or early termination. Exercise 2- Extending the Elbow: The activity starts at the Exercise Start stage (as given in Table 2) where the arm is held upright with forearm perpendicular to the table. During the Exercise in Progress stages, the volunteer extends the arm smoothly and in controlled manner. The activity ends at the Exercise End stage, where the elbow and the forearm become parallel to the table and touch the surface. For successful completion of Exercise 2, the activity moves continuously through all stages and reaches the Exercise End position, as shown in Table 2. For incomplete execution of Exercise 2, the Volunteer cannot achieve Full extension, and the movement stops before the Exercise End position due to limited range in motion, reduced movement control, or early stopping. Exercise 3- Lifting the Wrist: The activity starts at the Exercise Start stage where the forearm rests on an elevated support (e.g. a thick book or a shoe box etc.) placed on the table, and the wrist is in a left to hang freely. During the Exercise in Progress stages, the volunteer slowly lifts the wrist upward in a controlled way. The movement ends at the Exercise End stage, which is the final lifted wrist position. For the successful completion of Exercise 3, the wrist moves continuously through all the stages and reaches the Exercise End position, as shown in Table 2. For the incomplete exercises, the wrist movement is limited, lacks control, or fails to reach the Exercise End position. Exercise 4- Opening the Hand: The activity starts at Exercise Start stage, with the hand on an elevated support (e.g. a thick book or a shoe box etc.) extending beyond the support in a closed or relaxed position. During the Exercise in Progress stages, the volunteer gradually spread their fingers apart. The activity ends at the Exercise End position when the hand is fully open. For successful completion of Exercise 4, the fingers extend smoothly throughout all the stages when the hand is fully open. The exercise is considered incomplete when the fingers do not fully extend and they remain partially closed, or the movement stops before reaching the Exercise End position, as illustrated in Table 2. Table 3 presents the video parameters of upper-limb rehabilitation exercises for complete and incomplete execution conditions, including total number of videos, average duration (in seconds), average Frames Per Second (FPS), average number of frames per video sequence, average of the mean brightness across the videos, standard deviation of average brightness taken across the videos, average bitrate (in Mbps), video resolution (in pixels) and the aspect ratio of the captured videos. All videos are stored in MP4 format using the H.264 codec with RGB pixel format, Main profile, 8-bit depth, progressive scan, and BT.709 colour space. The difference in aspect ratios and resolutions seen in Table 3 can be attributed to the different devices employed (Samsung Galaxy A13 smartphone camera and HP True Vision 720p FHD camera). To ensure consistent input dimensions for deep learning based model development, it is recommended that all videos in the dataset be resized to a fixed input resolution as a preprocessing step. From the feature extraction perspective, deep learning architectures are typically capable of learning features such as textures, edges and motion patterns without being affected by minor variations in spatial composition. Hence, the difference in resolution does not adversely impact the robustness of the extracted features. Pixel intensity is considered here as a measure of the lighting conditions. Average and standard deviation of pixel intensities given in Table 3 are calculated as follows: As the first step, ten evenly spaced frames from each video are selected. Second step involves converting these frames into grayscale images. As the third step, mean and standard deviation of the pixel intensities of each of these grayscale images is computed. The mean and standard deviation of pixel intensities for each video is then estimated as the average of the mean and standard deviation of the pixel intensities across all the ten frames considered. The values presented in Table 3 are the average of these mean and standard deviation values computed across all the videos in the given completion status for the given exercise. As can be seen from Table 3, the pixel intensity value (range: 0–255) shows significant variability in each category, as can be seen from the standard deviation of pixel intensities, thereby indicating that the videos have been recorded under different lighting conditions.Table 3Parameters and dataset statistics for upper-limb rehabilitation exercise videos.Table 3 dummy alt textExerciseCompletion StatusTotal VideosAverage Duration (s)Average FPSAverage FramesPixel IntensityAverage Bitrate (Mbps)Resolution (Pixels)Aspect RatioAverageStandard DeviationExercise 1: Lifting an ObjectComplete6213.7729.87411.25121.3454.121.78810 × 14409:16Incomplete6111.5429.77343.63121.656.8721.80810 × 14409:16Exercise 2: Extending the ElbowComplete6011.5928.76333.35128.4755.3721.34810 × 18009:20Incomplete6711.2528.98325.86125.1956.3522.39810 × 18009:20Exercise 3: Lifting the WristComplete6014.1629.23415.21135.2541.3216.62810 × 14409:16Incomplete6115.8029.40465.88135.5842.6716.49810 × 14409:16Exercise 4: Opening the HandComplete6014.4029.23422.55129.1439.3516.53810 × 14409:16Incomplete6015.1629.37445.9130.0940.8321.25810 × 14409:16 Table 3Parameters and dataset statistics for upper-limb rehabilitation exercise videos.Table 3 dummy alt textExerciseCompletion StatusTotal VideosAverage Duration (s)Average FPSAverage FramesPixel IntensityAverage Bitrate (Mbps)Resolution (Pixels)Aspect RatioAverageStandard DeviationExercise 1: Lifting an ObjectComplete6213.7729.87411.25121.3454.121.78810 × 14409:16Incomplete6111.5429.77343.63121.656.8721.80810 × 14409:16Exercise 2: Extending the ElbowComplete6011.5928.76333.35128.4755.3721.34810 × 18009:20Incomplete6711.2528.98325.86125.1956.3522.39810 × 18009:20Exercise 3: Lifting the WristComplete6014.1629.23415.21135.2541.3216.62810 × 14409:16Incomplete6115.8029.40465.88135.5842.6716.49810 × 14409:16Exercise 4: Opening the HandComplete6014.4029.23422.55129.1439.3516.53810 × 14409:16Incomplete6015.1629.37445.9130.0940.8321.25810 × 14409:16 Parameters and dataset statistics for upper-limb rehabilitation exercise videos. The distribution of video frame counts for complete and incomplete exercise videos across the four upper-limb rehabilitation exercises is shown in Fig. 1. The histograms indicate the number of videos within different frame-count ranges for (a) Lifting an Object, (b) Extending the Elbow, (c) Lifting the Wrist, and (d) Opening the Hand. For all exercises, ‘Complete’ and ‘Incomplete’ sets show overlapping distribution in terms of the number of frames per video sequence. This indicates that exercise execution time alone may not be the ideal metric for concluding if the exercise has been completed successfully or has been left incomplete. More advanced machine learning/deep learning techniques are therefore needed to identify the exercise completion status accurately.Fig. 1Distribution of video frame counts for complete and incomplete samples of upper-limb rehabilitation exercises: (a) Exercise 1: Lifting an Object, (b) Exercise 2: Extending the Elbow, (c) Exercise 3: Lifting the Wrist, and (d) Exercise 4: Opening the Hand. The x-axis represents the number of frames per video, and the y-axis represents the number of videos corresponding to each frame count range.Fig 1 dummy alt text Distribution of video frame counts for complete and incomplete samples of upper-limb rehabilitation exercises: (a) Exercise 1: Lifting an Object, (b) Exercise 2: Extending the Elbow, (c) Exercise 3: Lifting the Wrist, and (d) Exercise 4: Opening the Hand. The x-axis represents the number of frames per video, and the y-axis represents the number of videos corresponding to each frame count range. Table 4 presents the summary statistics of frame counts for the upper-limb rehabilitation exercise videos under complete and incomplete execution conditions across all four exercises. The count indicates the total number of video samples available for each exercise and execution category. In Table 4, the column Mean represents the average number of frames per video reflecting the typical temporal length of the recordings, Standard Deviation captures the variability in frame counts within each category indicating differences in execution duration among subjects, Minimum and Maximum denote the shortest and longest video lengths in terms of frames, respectively. The column Q1 in Table 4 represents the first quartile and Q3 represents the third quartile which describe the dispersion of the of the data, providing insight into the consistency of video durations across complete and incomplete task executions.Table 4Summary Statistics of frames per recorded video in the dataset.Table 4 dummy alt textExerciseCompletion StatusMeanStandard DeviationMinimumQ1Q3MaximumExercise 1: Lifting an ObjectComplete411.2591.22197346435684Incomplete343.6370.5326306372528Exercise 2: Extending the ElbowComplete333.3559.88145297359.5483Incomplete325.8643.08201303.5353419Exercise 3: Lifting the WristComplete415.2191.98277338.75491581Incomplete465.88132.92312343547833Exercise 4: Opening the HandComplete422.55128.28205328525.25710Incomplete445.9132.55296340.25547.25745Note: All units in Table 4 are in terms of number of frames per video. Table 4Summary Statistics of frames per recorded video in the dataset.Table 4 dummy alt textExerciseCompletion StatusMeanStandard DeviationMinimumQ1Q3MaximumExercise 1: Lifting an ObjectComplete411.2591.22197346435684Incomplete343.6370.5326306372528Exercise 2: Extending the ElbowComplete333.3559.88145297359.5483Incomplete325.8643.08201303.5353419Exercise 3: Lifting the WristComplete415.2191.98277338.75491581Incomplete465.88132.92312343547833Exercise 4: Opening the HandComplete422.55128.28205328525.25710Incomplete445.9132.55296340.25547.25745Note: All units in Table 4 are in terms of number of frames per video. Summary Statistics of frames per recorded video in the dataset. Note: All units in Table 4 are in terms of number of frames per video. Experimental Design, Materials and Methods The dataset was created through a controlled and supervised recording process to ensure accurate and consistent labelling of upper-limb rehabilitation exercises as Complete or Incomplete. Reference pictures were displayed sequentially to guide volunteers through four exercises: Exercise 1-Lifting an Object, Exercise 2-Extending the Elbow, Exercise 3-Lifting the Wrist, and Exercise 4-Opening the Hand. Volunteers were made to be seated in a fixed position sideways with respect to the camera so that the upper limb under exercise is clearly visible in frame, as shown in Fig. 2.Fig. 2Representative frames showing the volunteer in seated posture performing Exercise 1- Lifting an Object: (a) Start of exercise, (b) and (c) Exercise in progress, (d) Exercise end.Fig 2 dummy alt text Representative frames showing the volunteer in seated posture performing Exercise 1- Lifting an Object: (a) Start of exercise, (b) and (c) Exercise in progress, (d) Exercise end. The uniquely identifiable physical features such as face have either been blurred or not included in the video field of view of the recording device to preserve privacy of the participants. Exercise execution was monitored by a supervisor, with the performance of exercise recorded using a mobile smartphone or an HP True Vision 720p FHD web-camera and stored in MP4 format in a structured directory based on exercise type and execution label, resulting in a dataset suitable for computer vision and machine learning–based rehabilitation analysis. The detailed algorithm used for exercise data acquisition is presented below:Exercise_List ← {E1: Lifting an Object, E2: Extending the Elbow, E3: Lifting the Wrist, E4: Opening the Hand}Target_Complete[E] ← required number of correctly performed (Complete) videos for exercise EStep 1: Initialize dataset directory 1. Create root folder: Exercise/. 2. For each exercise E in Exercise_List, create subfolders: Exercise/1_Lifting_an_Object/ Exercise/2_Extending_the_Elbow/ Exercise/3_Lifting_the_Wrist/ Exercise/4_Opening_the_Hand/ 3. Within each exercise folder created in Step 2, create two class folders: Complete/ Incomplete/ 4. Within each of the Complete/ and Incomplete/ folders, create two subfolders for dataset partitioning: Train/ Test/Step 2: Video acquisition and labeling 5. START 6. For each exercise E in Exercise_List: 6.1 Valid_Complete_Count(E) ← 0. 6.2 Repeat until the required number of Complete videos is obtained: a. Current_Exercise ← E. b. Display Reference image corresponding to Current_Exercise. c. Volunteer observes the reference picture and performs the exercise. d. The supervisor monitors the execution. e. Record the trial using the camera (MP4 format). f. Evaluate execution quality IF the exercise is performed correctly THEN Label ← Complete. Store the video in Exercise// Complete/ (temporarily, before splitting into Train/Test). Valid_Complete_Count(Current_Exercise)←Valid_Complete_Count(Current_Exercise)+1. ELSE Label ← Incomplete. Optionally allow the volunteer to reattempt the trial or discard the recording. Store the video in Exercise//Incomplete/ (temporarily, before splitting into Train/Test). END IF g. Continue acquiring videos for Current_Exercise until Valid_Complete_Count(Current_Exercise) = Target_Complete[Current_Exercise]. Step 3: Train-test partitioning and progression to next exercise 7. When the required number of Complete videos for Current_Exercise has been recorded: 7.1 For each label L in {Complete, Incomplete}: a. List all videos currently stored in Exercise///. b. Select 10 videos (e.g., random selection if >10 are available) and move them to: Exercise///Test/. c. Move all remaining videos for label L to: Exercise///Train/. 8. Switch to the next exercise in Exercise_List and repeat Steps 6–7. 9. END Exercise_List ← {E1: Lifting an Object, E2: Extending the Elbow, E3: Lifting the Wrist, E4: Opening the Hand}Target_Complete[E] ← required number of correctly performed (Complete) videos for exercise EStep 1: Initialize dataset directory 1. Create root folder: Exercise/. 2. For each exercise E in Exercise_List, create subfolders: Exercise/1_Lifting_an_Object/ Exercise/2_Extending_the_Elbow/ Exercise/3_Lifting_the_Wrist/ Exercise/4_Opening_the_Hand/ 3. Within each exercise folder created in Step 2, create two class folders: Complete/ Incomplete/ 4. Within each of the Complete/ and Incomplete/ folders, create two subfolders for dataset partitioning: Train/ Test/Step 2: Video acquisition and labeling 5. START 6. For each exercise E in Exercise_List: 6.1 Valid_Complete_Count(E) ← 0. 6.2 Repeat until the required number of Complete videos is obtained: a. Current_Exercise ← E. b. Display Reference image corresponding to Current_Exercise. c. Volunteer observes the reference picture and performs the exercise. d. The supervisor monitors the execution. e. Record the trial using the camera (MP4 format). f. Evaluate execution quality IF the exercise is performed correctly THEN Label ← Complete. Store the video in Exercise// Complete/ (temporarily, before splitting into Train/Test). Valid_Complete_Count(Current_Exercise)←Valid_Complete_Count(Current_Exercise)+1. ELSE Label ← Incomplete. Optionally allow the volunteer to reattempt the trial or discard the recording. Store the video in Exercise//Incomplete/ (temporarily, before splitting into Train/Test). END IF g. Continue acquiring videos for Current_Exercise until Valid_Complete_Count(Current_Exercise) = Target_Complete[Current_Exercise]. Step 3: Train-test partitioning and progression to next exercise 7. When the required number of Complete videos for Current_Exercise has been recorded: 7.1 For each label L in {Complete, Incomplete}: a. List all videos currently stored in Exercise///. b. Select 10 videos (e.g., random selection if >10 are available) and move them to: Exercise///Test/. c. Move all remaining videos for label L to: Exercise///Train/. 8. Switch to the next exercise in Exercise_List and repeat Steps 6–7. 9. END As can be observed from the above algorithm, there are two scenarios considered for each exercise: Complete and Incomplete, resulting in eight possible scenarios in total. The dataset collected for each scenario has been further partitioned into training (Train) and testing (Test) sets to enable a more streamlined deep learning model building workflow for researchers intending to use this dataset. The count of videos for each scenario is given in Table 5.Table 5Count of videos in the dataset for each scenario with train-test split (To be read in conjunction with folder structure and organisation).Table 5 dummy alt textExerciseCompleteIncompleteTotal VideosTrainingTestingTrainingTestingExercise 1: Lifting an Object52105110123Exercise 2: Extending the Elbow50105710127Exercise 3: Lifting the Wrist50105110121Exercise 4: Opening the Hand50105010120 Table 5Count of videos in the dataset for each scenario with train-test split (To be read in conjunction with folder structure and organisation).Table 5 dummy alt textExerciseCompleteIncompleteTotal VideosTrainingTestingTrainingTestingExercise 1: Lifting an Object52105110123Exercise 2: Extending the Elbow50105710127Exercise 3: Lifting the Wrist50105110121Exercise 4: Opening the Hand50105010120 Count of videos in the dataset for each scenario with train-test split (To be read in conjunction with folder structure and organisation). Folder structure and data organization The upper-limb stroke rehabilitation exercise video dataset presented here is organized to support deep learning based model development workflows. The hierarchical organization and folder structure adopted for the dataset is illustrated in Fig. 3. As shown in Fig. 3, the root folder is named Exercise and comprises of four subfolders labelled based on the type of exercise being performed. The naming convention used for each of these four subfolders is: Exercise Number_Exercise Name. For example: All the video data acquired for Exercise 1-Lifting an Object is in the subfolder: 1_Lifting an Object and so on for the remaining three exercises considered here. Each of these four folders consists of two subfolders labelled Complete and Incomplete and each of these subfolders has two further subfolders labelled Train and Test. Individual video files in .MP4 format are stored in these subfolders. Each video file is named as per the following convention: Volunteer ID_Exercise Number_Completion Status_Video Number.MP4. Each volunteer performing the exercise has been assigned a unique two-digit identification number such that Volunteer ID = {00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10}. Since four exercises are being considered here, the Exercise Number is represented using a two digit identification number such that Exercise Number= {01, 02, 03, 04}, where, 01-Exercise 1: Lifting an Object, 02-Exercise 2: Extending the Elbow, 03- Exercise 3: Lifting the Wrist and 04- Exercise 4: Opening the Hand. The Completion Status can take two values: 00 indicating incomplete exercise or 01 indicating complete exercise. Hence, Completion Status = {00,01}. The Video Number is simply the identification number of video in the subfolder it is located. Hence, Video Number={01, 02, …, 15}. It must be noted that video samples are separated into training and testing subsets using a subject-wise (volunteer-level) split. This strategy ensures that recordings from the same volunteer do not appear in both subsets, thereby preventing subject-level data leakage and improving the generalizability of trained models. Such a subject-independent split is particularly critical for rehabilitation datasets, where individual motor patterns could otherwise bias performance evaluation.Fig. 3Hierarchical organization of the upper-limb stroke rehabilitation exercise video dataset.Fig 3 dummy alt text Hierarchical organization of the upper-limb stroke rehabilitation exercise video dataset. Developing a deep learning based rehabilitation exercise completion status identification system Deep learning-based techniques are increasingly used for automated assessment of patient’s performance in telerehabilitation systems. The typical methodology involves accessing rehabilitation movement datasets (such as the one presented in this study), followed by preprocessing, if necessary. The pre-processed dataset is then used to train deep learning models. Once trained, the model is deployed for real-time rehabilitation exercise assessment. It must be noted that modern deep learning architectures are capable of automatically learning hierarchical feature representations from raw data, thereby reducing the need for handcrafted feature extraction and feature selection methods. However, explicit feature refinement or selection modules may still be useful in improving the discriminative capability and model generalization. The proposed workflow is presented in Fig. 4.Fig. 4Proposed deep learning based rehabilitation exercise completion detection system workflow.Fig 4 dummy alt text Proposed deep learning based rehabilitation exercise completion detection system workflow. In order to illustrate the effectiveness of the proposed workflow and to empirically validate the separability of ‘Complete’ and ‘Incomplete’ classes, baseline results obtained using a simple 3D-CNN based exercise status identification system are being presented here. The model hyperparameters have been identified using trial and error, researchers may utilize the dataset presented with more complex deep architectures and techniques such as Bayesian optimization or grid search for better results. The 3D CNN network architecture is presented in Table 6 below.Table 63D CNN model architecture.Table 6 dummy alt textLayer No.Layer (type)Output ShapeLearnable parameters1video_input (InputLayer)(10, 45, 45, 3)02conv3d_1 (Conv3D)(10, 45, 45, 8)6563pool3d_1 (MaxPooling3D)(10, 22, 22, 8)04conv3d_2 (Conv3D)(10, 22, 22, 8)17365pool3d_2 (MaxPooling3D)(5, 11, 11, 8)06conv3d_3 (Conv3D)(5, 11, 11, 8)17367pool3d_3 (MaxPooling3D)(2, 5, 5, 8)08gap3d (GlobalAveragePooling3D)809dropout_1 (Dropout)8010dense_1 (Dense)3228811dropout_2 (Dropout)32012classifier (Dense)8264 Table 63D CNN model architecture.Table 6 dummy alt textLayer No.Layer (type)Output ShapeLearnable parameters1video_input (InputLayer)(10, 45, 45, 3)02conv3d_1 (Conv3D)(10, 45, 45, 8)6563pool3d_1 (MaxPooling3D)(10, 22, 22, 8)04conv3d_2 (Conv3D)(10, 22, 22, 8)17365pool3d_2 (MaxPooling3D)(5, 11, 11, 8)06conv3d_3 (Conv3D)(5, 11, 11, 8)17367pool3d_3 (MaxPooling3D)(2, 5, 5, 8)08gap3d (GlobalAveragePooling3D)809dropout_1 (Dropout)8010dense_1 (Dense)3228811dropout_2 (Dropout)32012classifier (Dense)8264 3D CNN model architecture. The model was trained using the Adam optimizer with an initial learning rate of 0.001. Categorical cross entropy was the loss function employed. Input videos were resized to 45 × 45 pixels and uniformly sampled to obtain 10 frames per video (i.e. sequence length considered is 10), for faster training. However, it is recommended that researchers attempting to build deep learning based systems using the dataset presented here, keep the extracted frame resolution as close as possible to the original resolution in the dataset. It is also recommended that the sequence length be as long as possible so that the variations in movement during the exercise are correctly captured by the model. As shown in Table 6, the proposed baseline 3D CNN architecture consisted of three convolutional layers, each with 8 filters respectively, with each convolutional layer followed by a max pooling layer. The final convolutional layer is followed by a global average pooling layer which is then followed by the first dropout layer with a dropout rate of 0.4 and a fully connected layer with 32 neurons. This is followed by the final dropout layer with a dropout rate of 0.3 and the dense classification layer with 8 neurons. A batch size of 4 was used for training over 40 epochs. Performance was evaluated using accuracy, precision, recall, F1-score, and confusion matrices. Experiments were implemented in Python using TensorFlow and executed on a Windows system equipped with an NVIDIA RTX 4060 GPU. Performance of the baseline model (weighted across all classes) is presented in Table 7 below.Table 7Baseline model performance.Table 7 dummy alt textPerformance MetricTrainTestAccuracy0.80050.4Precision0.83270.4217Recall0.80050.4F10.79540.3831 Table 7Baseline model performance.Table 7 dummy alt textPerformance MetricTrainTestAccuracy0.80050.4Precision0.83270.4217Recall0.80050.4F10.79540.3831 Baseline model performance. As can be seen from Table 7, the baseline model presented here is able to successfully distinguish between the exercises and the ‘Complete’ as well as ‘Incomplete’ exercise completion scenarios. Limitations All the volunteers considered in this study belong to the age group 18–29 years as per the groups considered in [4]. Typically, stroke incidence leads to impairment of both upper and lower limbs at the same time with accompanying cognitive impairments as well. Since lower limb and cognitive rehabilitation exercises differ significantly from the upper limb rehabilitation exercises presented in this dataset, and need to be carried out separately, these exercises have not been considered here. Ethics Statement All contributors involved in the creation of this video dataset participated voluntarily with complete awareness and consent. The exercises were performed under supervision. To ensure privacy, the videos do not include uniquely identifiable physical features. In consequence, there are no ethical concerns regarding the health and safety of participants. CRediT authorship contribution statement S. Nandana: Investigation, Data curation, Writing – original draft, Visualization. M. Thenmozhi Dharshini: Investigation, Resources, Data curation, Writing – original draft, Project administration. Vismaya Viswanathan: Investigation, Data curation, Writing – original draft, Visualization. Binoy B. Nair: Conceptualization, Methodology, Validation, Writing – review & editing.