Deep learning-based classification of interphalangeal finger joints in erosive hand osteoarthritis Authors: Zakaria Oubbi, Gust Verbruggen, Sam Leroux, Pieter Simoens Published: 2026 Introduction After osteoarthritis (OA) affects the knee and hip joints, it most commonly develops in the hand. A significant proportion of the patients with hand OA suffer from the erosive type of OA. The prevalence of erosive hand OA ranged from 3% in men to 10% in women of Caucasian origin [1]. This distinct form of OA affects primarily the interphalangeal (IP) finger joints and is rapidly progressing and destructive [[2], [3], [4], [5], [6]]. Therefore, many patients with erosive hand OA experience significant discomfort and functional disability [[7], [8], [9], [10]]. Accurate staging of the structural damage sustained by the affected joint tissues is essential for developing an effective therapeutic approach for these patients. The tissue changes in erosive hand OA include resorption of articular cartilage usually preceding osteolysis and resorption of the subchondral bone endplate. At the same time osteolytic changes occur in the epiphyseal subchondral bone area [11,12]. Articular tissue destruction is then followed by reparative features such as remodeling of the subchondral bone plate and the formation of bony nodules at the margins of the affected IP joints. On radiographs, in addition to classic OA features such as joint space narrowing, subchondral bone plate sclerosis, subchondral cysts, and osteophyte formation, IP joints affected by erosive OA exhibit more aggressive changes. These include a complete loss of joint space, which typically precedes or coincides with the collapse of the subchondral bone plate. This destructive phase is followed by a reparative stage, characterized by the reappearance of a new subchondral plate covered with fibrocartilaginous tissue within the previously eroded joints. These erosive and remodeling changes can occur concurrently in both proximal and distal IP joints, often in an unpredictable order within the same patient. As a result, the disease may remain active for several decades, progressing toward a terminal remodeling phase in all affected joints [11]. To standardize the assessment of erosive hand OA progression, the five-stage Verbruggen-Veys (V&V) radiographic scoring system was developed [11]. This ordinal system classifies IP joints into five categories: normal (N), static with typical OA features (S), joint space loss (J), erosive (E), and remodeling (R), representing a characteristic progression: N ≺ S ≺ J ≺ E ≺ R. The defining features of erosive hand OA include alterations in the subchondral plate, specifically resorption (E) and subsequent repair (R). Progression through these anatomical stages has been documented earlier [[13], [14], [15]] Over one year, around 4.0–7.0% of N, S or J joints in placebo patient cohorts showed erosive (E) progression. Erosive evolution was higher in interphalangeal joints showing signs of clinical inflammation, i.e. swelling [13]. Another 5.0%–8.0% E joints from placebo treated patients in these reports showed remodeling (R) within 48 weeks of follow-up. Conventional radiography remains the primary imaging modality for diagnosing and monitoring OA. Increasingly, computer-aided diagnosis systems assist in radiographic interpretation by providing an objective second opinion, reducing the reliance on multiple expert readers. For erosive hand OA, an automated approach to V&V scoring could improve diagnostic efficiency and consistency. Given the discrete nature of the labeling scale, this problem is well-suited for machine learning-based classification. Deep learning is especially suitable due to the use of image data as input. In particular, convolutional neural networks have shown strong performance in medical image analysis, including applications to musculoskeletal disorders. To address data limitations, transfer learning with pre-trained models can be employed [[16], [17], [18]]. Furthermore, the ordinal nature of the V&V scoring system presents an opportunity to explore ordinal classification techniques, which can potentially capture the inherent relationship between disease stages more effectively than traditional multi-class classification methods. To our knowledge, no prior work has leveraged deep learning to automate the scoring of erosive hand OA. This study aims to fill that gap by developing and evaluating a deep learning-based automated system for scoring erosive hand OA using full-hand radiographs. We employed a two-step approach, first detecting individual IP joints using YOLOv8 [19], and then classifying them according to the V&V scoring system using a fine-tuned ResNet-50 architecture [20]. Critically, we investigated the impact of data augmentation, class imbalance mitigation, and transfer learning strategies on model performance. We explored the potential of improving model performance by leveraging the ordinal nature of the V&V scores through an ordinal classification framework, specifically Conditional Ordinal Regression for Neural Networks (CORN) [21]. By developing a reliable automated scoring system, we aim to improve the efficiency and consistency of IP OA assessment, ultimately facilitating more effective clinical management and research. Materials and methods DATASET: SOURCE AND CONSTRUCTION: The datasets utilized consisted of full hand radiographs that originated from three controlled pharmacological interventional studies [[13], [14], [15]]. The three studies included independent patient cohorts of 60, 85, and 100 participants, with no patient participating in more than one study. Patients were selected based on IP joints showing J and/or E phases. Posteroanterior full hand radiographs were taken at baseline and after 24 and 48 weeks during these studies. For each patient and timepoint, eight DIP and eight PIP joints were obtained, providing a total of 11104. Detailed information is given in a supplementary table. Radiographs were provided in JPEG/TIFF format to enable the reading on PCs of the different readers. The protocol allowed the creation of a machine-learning-ready dataset containing 11096 IP finger joint images and their respective V&V scores. Five IP joints with surgical intervention were excluded. The dataset was divided into training end test sets at the patient level, ensuring that all images from a given patient were assigned exclusively to one subset to prevent data leakage. Each trial was split using the same approach, preserving proportional representation across training and test sets. The final training and test sets contained 9792 and 1304 joint images, respectively, corresponding to 27 patients in the test set. A stratified approach maintained class distribution in each subset to address class imbalance. Percentage distribution for the anatomical phases N, S, J, E and R was 28%, 35%, 7%, 8% and 22%, respectively. SCORING METHOD: The V&V categorical scoring system assesses the progression of erosive hand OA through five distinct anatomic phases: N (Normal), S (Stationary (non-erosive)), J (Joint Space Loss), E (Erosive), and R (Remodeling). The phases reflect the evolution of the affected IP finger joint, from normal to eroded and remodeled states, and follow an ordinal relationship, representing the sequential progression of the disease: N ≺ S ≺ J ≺ E ≺ R [11]. Each phase is characterized by specific features observed in imaging. The defining features of erosive hand OA include alterations in the subchondral plate, specifically resorption (E) and subsequent repair (R). (Fig. 1).Fig. 1Sequential anatomical phases in a distal (D) and proximal (P) IP finger joint. Sequential anatomical phases in a distal (D) and proximal (P) IP finger joint. LABELING: V&V scores were given for all patients' 16 IP joints at baseline and at each study time point. Consistency in scoring across datasets was maintained as scoring was done independently by 2 experienced readers in the Ghent rheumatology department. Intra- and inter-reader reproducibility were both excellent. Labels assigned to the IP joints from the 3 studies were adopted in the current dataset for subsequent supervised deep learning analysis. IMPLEMENTATION: This work utilizes the open-source machine learning libraries PyTorch [22] and PyTorch Lightning [23] for implementing the neural networks. Additional software libraries include YOLOv8 [19] for IP joint detection and PyTorch Image Models (timm) [24] for loading the pretrained ResNet-50. IP FINGER JOINT DETECTION: Erosive hand OA affects individual IP finger joints, which necessitates an approach that can effectively utilize full-hand radiographs as input. The strategy used was inspired by similar work for rheumatoid arthritis and involves a two-step process where joint detection and scoring are treated as separate tasks [25,26]. This separating detection from scoring allows models to be more precisely optimized for each task, potentially improving performance. Joint detection was performed using YOLOv8 [19], pretrained on the COCO dataset [27] and finetuned on a manually annotated subset of 100 full-hand X-ray images. While the model distinguishes DIP and PIP joints separately with high precision and recall (Fig. 2A), it does not inherently assign a joint index, i.e., the corresponding finger.Fig. 2A- Precision-recall curve of the YOLOv8 joint detector on an unseen test set of 32 full hand radiographs. B- The joint extraction process. The left upper image shows the original hand radiograph while the right upper image shows the same radiograph with the joint predictions from the joint detection model. These joints are cropped on the bounding boxes resulting in the 8 joint images at the bottom. C - CLAHE's effect on a joint image. A- Precision-recall curve of the YOLOv8 joint detector on an unseen test set of 32 full hand radiographs. B- The joint extraction process. The left upper image shows the original hand radiograph while the right upper image shows the same radiograph with the joint predictions from the joint detection model. These joints are cropped on the bounding boxes resulting in the 8 joint images at the bottom. C - CLAHE's effect on a joint image. A custom algorithm was therefore implemented to determine hand orientation (left or right) based on bounding box positions, enabling automatic indexing. The detected bounding boxes were used to automatically crop regions of interest (PIP and DIP joints) from full-hand radiographs, forming the V&V dataset. The bounding box's largest dimension was used for cropping on square dimensions, with an additional fixed 15-pixel margin (applied uniformly to all joints) to retain relevant joint features and support data augmentation. Cropping was performed on the original-resolution radiographs; therefore, the margin was constant in pixel units and does not correspond to a fixed physical size (mm). DIP and PIP joints were modeled jointly, as the V&V scoring system applies uniformly across IP joints. These cropped images, along with their labels, served as inputs for the joint scoring models (Fig. 2B). IP FINGER JOINT SCORING - MODEL TRAINING: In all experiments, a ResNet-50 model pretrained on the ImageNet dataset [28] was used. The output layer was adapted to the five-score classification of the V&V scoring system. Starting with the conventional (i.e. vanilla) classification, we explored different strategies for data augmentation such as random rotations, horizontal flips, translations, and scaling [29,30]. Some studies suggest using Contrast Limited Adaptive Histogram Equalization (CLAHE) [29] to enhance contrast in X-ray images [[31], [32], [33]]. CLAHE adjusts contrast locally while limiting noise amplification, improving image clarity and potentially aiding classification. To assess its impact, the augmentation pipeline was tested with and without CLAHE. The clip limit, which controls contrast intensity, was empirically set to 2. Fig. 2C illustrates CLAHE's effect on a joint image. We then explored different finetuning strategies where increasingly larger parts of the model were finetuned. Finally, we moved from conventional classification to ordinal classification. To implement ordinal classification, we tested one framework: CORN [21]. This framework is model-agnostic and compatible with any neural network, allowing us to adapt the pretrained ResNet-50's architecture for ordinal classification. The same dataset splits, preprocessing steps, and data augmentation methods as those employed for conventional classification were applied. STATISTICAL ANALYSIS: For evaluating the results obtained from automating the V&V scoring system, unless stated otherwise, all metrics are reported in macro-averaged form. In this approach, metrics are calculated independently for each class and then averaged with equal weight across classes, ensuring that minority classes contribute equally despite class imbalance. During the pretraining phase, a 5-fold stratified cross-validation scheme was used to obtain averaged results. Given the extensive training time required for finetuning large model segments, a systematic approach was adopted for the finetuning phase. First, the ResNet-50 was pretrained using the best configuration across three independent runs, each with a different random seed. Each pretrained model was then finetuned using its corresponding seed. Final performance was averaged across the three runs to obtain a more robust estimate. This approach was applied consistently to both conventional and ordinal classification experiments. Ground-Truth Verification: To assess potential ambiguity in ground-truth labels, a subset of misclassified joints from the test set was re-evaluated. Fifty misclassified joints (10 per anatomical phase) were randomly selected. Sample identifiers were pseudorandomized to conceal study origin, patient identity, and trial information. An expert reader independently re-scored the selected joints without access to the original labels or model predictions. Agreement between the expert, ground-truth labels, and model predictions was quantified using percentage agreement and Cohen's kappa coefficient. Results MODEL TRAINING Conventional classification: As a baseline, we only finetuned the output layer of the model. We explored different hyperparameter combinations using the ADAM optimizer [34] (β1 = 0.9, β2 = 0.999, no AMSGrad) with a batch size of 32. A 5-fold stratified cross-validation was applied, averaging evaluation metrics Area-Under-Curve (AUC) score and cross-entropy loss across all five runs. Each run used a different random seed, but the same random seeds were chosen across the different hyperparameter configurations for fair comparison. Hyperparameter combinations tested are shown in Table 1. Overfitting occurred before decay had an impact, and Learning rate (LR) = 1e-4 provided the most stable results, making it the chosen configuration.Table 1Hyperparameter search for the ADAM optimizer.Table 1Learning rate (LR)DecayLossAUC1e−301.13 ± 0.040.77 ± 0.021e−31e−31.12 ± 0.030.77 ± 0.021e−401.11 ± 0.030.78 ± 0.021e−41e−31.11 ± 0.030.77 ± 0.021e−501.13 ± 0.030.77 ± 0.021e−51e−31.14 ± 0.030.77 ± 0.02Metrics averaged over 5 runs, Mean ± SD given. Table 1Hyperparameter search for the ADAM optimizer.Table 1Learning rate (LR)DecayLossAUC1e−301.13 ± 0.040.77 ± 0.021e−31e−31.12 ± 0.030.77 ± 0.021e−401.11 ± 0.030.78 ± 0.021e−41e−31.11 ± 0.030.77 ± 0.021e−501.13 ± 0.030.77 ± 0.021e−51e−31.14 ± 0.030.77 ± 0.02Metrics averaged over 5 runs, Mean ± SD given. Hyperparameter search for the ADAM optimizer. Metrics averaged over 5 runs, Mean ± SD given. Overfitting and Class Imbalance: Phases J and E, typically the most critical stages in the progression of erosive hand OA, were underrepresented in the dataset. We tested three strategies to mitigate this imbalance (Table 2). All experiments used the optimal hyperparameter configuration from the previous section, with a baseline comparison of the configuration without any imbalance or overfitting strategies. The best results came from random oversampling combined with data augmentation that included CLAHE as a step in the pipeline. The class-penalty strategy using class weights in cross-entropy loss performed poorly. It is important to note that the minimum loss was higher with imbalance and overfitting strategies, as the model was still in the pretraining phase and had limited capacity. Without these strategies, the model focused on overrepresented classes, leading to higher “apparent" performance.Table 2Class imbalance and overfitting strategy.Table 2StrategyLossAUCBaseline (no strategy)1.11 ± 0.030.78 ± 0.02ROS with augmentation1.16 ± 0.030.79 ± 0.01ROS with augmentation + CLAHE1.16 ± 0.030.80 ± 0.01Class weighting1.21 ± 0.040.77 ± 0.02ROS with augmentation + class weighting1.57 ± 0.040.77 ± 0.01Metrics averaged over 5 runs, Mean ± SD given. AUC: Area Under Curve; ROS: Random Oversampling; CLAHE: Contrast Limited Adaptive Histogram Equalization. Table 2Class imbalance and overfitting strategy.Table 2StrategyLossAUCBaseline (no strategy)1.11 ± 0.030.78 ± 0.02ROS with augmentation1.16 ± 0.030.79 ± 0.01ROS with augmentation + CLAHE1.16 ± 0.030.80 ± 0.01Class weighting1.21 ± 0.040.77 ± 0.02ROS with augmentation + class weighting1.57 ± 0.040.77 ± 0.01Metrics averaged over 5 runs, Mean ± SD given. AUC: Area Under Curve; ROS: Random Oversampling; CLAHE: Contrast Limited Adaptive Histogram Equalization. Class imbalance and overfitting strategy. Metrics averaged over 5 runs, Mean ± SD given. AUC: Area Under Curve; ROS: Random Oversampling; CLAHE: Contrast Limited Adaptive Histogram Equalization. Transfer learning: To improve the performance of the model, we experimented with finetuning larger parts of the model. We found that finetuning the entire ResNet-50 yielded the best performance, suggesting the dataset was large enough to support deep tuning. However, increasing the number of trainable parameters also heightened the risk of overfitting. As finetuning depth increased, overfitting occurred earlier, as reflected in the epochs of minimal validation loss (Table 3). For instance, when finetuning only the first residual block (i.e. the block before the output layer), minimal loss occurred at epoch 342, whereas for the full model (at the same learning rate of 1e-5), it occurred at epoch 24. This indicates that deeper finetuning was necessary due to the substantial difference between the source (natural images) and target (hand radiographs) domains.Table 3Finetuning up to different block-depths in the ResNet-50.Table 3Block depthLearning rate (LR)Epoch min. LossLossAUCB11e-53421.060 ± 0.0100.850 ± 0.002B1-21e-5791.040 ± 0.0180.864 ± 0.0081e-64701.053 ± 0.0140.850 ± 0.003B1-31e-5760.975 ± 0.0160.875 ± 0.002B1-41e-5760.967 ± 0.0120.883 ± 0.002B1-151e-5240.892 ± 0.0130.903 ± 0.001B1-161e-5240.889 ± 0.0080.901 ± 0.0011e-430.846 ± 0.0130.911 ± 0.001Metrics averaged over 3 runs, Mean ± SD given. LR: Learning Rate; AUC/Area Under Curve. Table 3Finetuning up to different block-depths in the ResNet-50.Table 3Block depthLearning rate (LR)Epoch min. LossLossAUCB11e-53421.060 ± 0.0100.850 ± 0.002B1-21e-5791.040 ± 0.0180.864 ± 0.0081e-64701.053 ± 0.0140.850 ± 0.003B1-31e-5760.975 ± 0.0160.875 ± 0.002B1-41e-5760.967 ± 0.0120.883 ± 0.002B1-151e-5240.892 ± 0.0130.903 ± 0.001B1-161e-5240.889 ± 0.0080.901 ± 0.0011e-430.846 ± 0.0130.911 ± 0.001Metrics averaged over 3 runs, Mean ± SD given. LR: Learning Rate; AUC/Area Under Curve. Finetuning up to different block-depths in the ResNet-50. Metrics averaged over 3 runs, Mean ± SD given. LR: Learning Rate; AUC/Area Under Curve. We also experimented with adjusting the learning rate. Lowering it for configurations such as finetuning two residual blocks led to worse results. However, increasing it to 1e-4, despite causing overfitting after just three epochs, significantly improved performance. This suggests that lower learning rates may have trapped the model in local minima, preventing effective adaptation. Additionally, the substantial domain shift likely required more aggressive weight updates. The best-performing model was selected based on the best evaluation metrics, and its confusion matrices on the train and test sets are presented in Fig. 3a and b.Fig. 3Confusion Matrices of the best performing models for classification. Confusion Matrices of the best performing models for classification. Ordinal classification: Finally, we used the CORN framework to implement ordinal classification which is a more natural fit for the ordinal scoring scale. We explored the same hyperparameter space used for conventional classification. The ADAM optimizer [34] (β1 = 0.9, β2 = 0.999, no AMSGrad) was applied with a batch size of 32, a 5-fold stratified cross-validation scheme, and consistent random seeds. Since full fine-tuning previously yielded the best performance, the CORN model was fully fine-tuned, with the results presented in Table 4 for comparison. Confusion matrices from the best-performing runs are shown in Fig. 3a and b (vanilla classifier) and Fig. 3c and d (CORN classifier).Table 4Comparison of the optimal results for each model type.Table 4ModelLRMAERMSEACCBaseline1e−40.583 ± 0.03890.964 ± 0.04190.583 ± 0.0435Vanilla1e−40.398 ± 0.01100.811 ± 0.02490.704 ± 0.0093CORN1e−40.393 ± 0.00780.791 ± 0.01900.700 ± 0.0064Metrics averaged over 3 runs, Mean ± SD given. A baseline model, trained without pretrained weights, random oversampling (ROS), or data augmentation, is included for reference.LR: Learning Rate; MAE: Mean Absolute Error; RMSE: Root Mean Squared Error; ACC: Accuracy. Table 4Comparison of the optimal results for each model type.Table 4ModelLRMAERMSEACCBaseline1e−40.583 ± 0.03890.964 ± 0.04190.583 ± 0.0435Vanilla1e−40.398 ± 0.01100.811 ± 0.02490.704 ± 0.0093CORN1e−40.393 ± 0.00780.791 ± 0.01900.700 ± 0.0064Metrics averaged over 3 runs, Mean ± SD given. A baseline model, trained without pretrained weights, random oversampling (ROS), or data augmentation, is included for reference.LR: Learning Rate; MAE: Mean Absolute Error; RMSE: Root Mean Squared Error; ACC: Accuracy. Comparison of the optimal results for each model type. Metrics averaged over 3 runs, Mean ± SD given. A baseline model, trained without pretrained weights, random oversampling (ROS), or data augmentation, is included for reference. LR: Learning Rate; MAE: Mean Absolute Error; RMSE: Root Mean Squared Error; ACC: Accuracy. Results on a balanced test set with verified labels Class distribution has a direct impact on the performance of a model. To investigate this effect we constructed a balanced test set of 608 samples drawn from the full dataset, including only joints with non-debatable scores. This set contained roughly 120 samples per class, with PIP and DIP joints equally represented across all scores. A corresponding training set of 6186 samples was created from non-test patients to prevent data leakage, though it remained unbalanced. An instance of the best-performing model configuration, CORN, was trained on this set for three runs, with the averaged results: MAE 0.370 ± 0.0110 and RMSE 0.731 ± 0.0041. Surprisingly, the model performed slightly better on this new test set than the reference model did on the original test set, despite training on 40% less data. This indicates that employing non-debatable criteria for score assignment, together with a balanced class distribution, has a significant effect on model performance. For reproducibility, the data set and the labels are given in a supplemental file: DOI: https://doi.org/10.5281/zenodo.17297344. ANALYSIS of MODEL mispredictions We examined the errors made by CORN, our preferred model. CORN misclassified 439 instances in the test set, with errors distributed across different datasets (Fig. 4A). The most misclassified score was S, followed by R and N, which aligns with class distribution: S being the largest, followed by N and R. Despite using random oversampling, early overfitting limited the model's exposure to S samples, potentially contributing to its misclassification. Additionally, N–S confusion discussed further below likely played a role in S being the most wrongly predicted class. Finally, cautious review of these misclassifications showed that human error caused few of these wrong scores. Moving the original labels to spreadsheet documents for automation caused some errors to appear in the ground-truth data. Reviewing individual misclassified samples revealed that other misreadings were, however, justifiable.Fig. 4A Distribution of the mispredictions per dataset normalized over all mispredictions. Of the 439 mispredictions, 171 (39%) came from the DNB dataset, 144 (33%) from the ADA dataset, and 124 (28%) from the ETA dataset. Most mispredictions happened to the S-joints. B Examples of model mispredictions: IP joint A: erroneous label in the ground-truth; IP joint C: misprediction of the model. A Distribution of the mispredictions per dataset normalized over all mispredictions. Of the 439 mispredictions, 171 (39%) came from the DNB dataset, 144 (33%) from the ADA dataset, and 124 (28%) from the ETA dataset. Most mispredictions happened to the S-joints. B Examples of model mispredictions: IP joint A: erroneous label in the ground-truth; IP joint C: misprediction of the model. Fig. 4B illustrates four misclassified joints, where, in some cases, the model's prediction appeared more reasonable than the ground-truth label. For instance, the joint (A) in Fig. 4B, labeled as N, seems to align more closely with the model's prediction of R. Since these joints retained the same ground-truth labels across 3 trial periods, the model accumulated 13 errors from these four joints alone. Further analysis showed that 9 out of 27 test-set patients had 20 or more misclassifications each, collectively accounting for over half of all errors. This suggested that many ground-truth labels may be ambiguous or require re-evaluation. As described in the Methods section, fifty misclassified joints were re-scored by an expert reader to evaluate potential ground-truth ambiguity. Percentage of agreement between the expert and the ground-truth labels was 46% (Cohen's Kappa coefficient: 0.32). Percentage of agreement between the expert and the model predictions was 48% (Cohen's Kappa coefficient: 0.33). The percentage of agreement between the ground-truth labels and the model was 0%, as the 50 joints were all misclassified. As shown, re-scoring showed similar agreement with the model and with the ground-truth labels. This suggests that the model's actual performance may surpass its reported metrics, as some errors may reflect that many joints have labels that are subject to interpretation. Discussion Erosive hand OA is a fast-advancing type of OA that affects the interphalangeal finger joints. Radiographs reveal progression through distinct anatomical phases, each phase marking its progression and providing a scoring system presenting unique opportunities for diagnosis and assessment [11]. Machine learning has hitherto been applied in radiology for image classification. Previous research has used deep learning to localize joints on hand radiographs [35,36] and to differentiate types of joint diseases, e.g. osteoarthritis and rheumatoid arthritis [37,38]. After diagnosing joint disease, grading its severity is crucial as it guides the clinician's treatment options. Deep learning methods graded the severity of joint damage in patients with rheumatoid arthritis [25,26,36]. However, deep learning has not yet been employed to automate the scoring of erosive hand OA. This study addresses that gap by applying deep learning to automate the ordinal V&V scoring system. Our dataset included radiographs from 245 subjects taken at three intervals, covering over 10,000 IP finger joints. X-ray images obtained from roughly equal numbers of patients enabled the development of deep learning methods elsewhere. Hirano et al. used 216 radiographs from 108 patients with rheumatoid arthritis; 11160 images of PIP and MCP joints were used to assess joint space narrowing and erosions [25]. Peng et al. X-rayed 344 hands and developed scores for arthritis injuries [38]. Most of these studies applied augmentation techniques to extract more data from limited datasets. Our approach compared conventional and ordinal classification using a pretrained ResNet-50. Both methods performed well, with deep tuning and class imbalance strategies proving effective. As shown, CORN and the conventional “vanilla” classifier performed similarly. Despite their comparable overall performance, differences emerged in the confusion matrices: the vanilla classifier better distinguished N from S joints, whereas CORN was more effective at separating E from R joints. However, distinguishing between certain successive anatomical phases remained challenging. Both models frequently misclassified N and S joints, with S joints often incorrectly labeled as N, and less commonly as J. Misclassifications between E and R joints were also frequent. These observations are explored further below. CORN was chosen as the preferred model for three reasons: (1) it performed better on underrepresented scores J and E, which are crucial for assessment; (2) it exhibited less overfitting than the vanilla classifier, as observed in the confusion matrices; and (3) it achieved slightly better MAE and RMSE scores, which quantify prediction accuracy for ordinal labels. The CORN classifier performed best on classes J and E (erosive OA joints), achieving accuracies of 76% and 77%, respectively. This finding is important as the presence of ≥1 IP joint in the J or E phase of the anatomical phase scoring system is a key inclusion criterion for patients in trials exploring the effects of pharmacological agents on structure modification in erosive hand OA. The model's performance (in between 70 and 80% confidence) is in line with the inherent reader's hesitancy while deciding on specific anatomical phases of affected IP finger joints. The subtle changes in the anatomy of non-affected N IP joints and those showing the first signs of OA may be dubious to discern. The frequent occurrence of N–S confusion is due to the minimal visual changes in the joints during this phase transition. Regarding S-J confusion, it was observed that severely narrowed and barely visible joint spaces often lead to misclassification. Expert readers assess joint space narrowing ideally by comparing with the contralateral IP joint. Such comparison is not done by the ordinal classification model. Misclassification of IP joints by the ordinal classifier arises from the difficulty in determining when a joint transitions from one to another anatomical phase. Likewise, these arguable anatomical changes lead to the inter- and intrareader variabilities by experienced readers. Furthermore, even with a detailed definition of each anatomical phase, interphalangeal finger joints advancing through these successive phases may show considerable overlapping in phase characteristics. An IP finger joint in the S phase showing a 90 % reduced joint space width strictly does not meet the definition of an J phase joint when some erosion of the subchondral plate already may appear. Technically, such an IP joint can be classified as an E joint. Likewise, a partly restored E phase IP joint can only be classified as an R phase joint when the subchondral plates entirely reappeared. Lastly, it should be emphasized that radiographs in the clinical studies were collected at set intervals. The same joint then appears multiple times in the dataset. This has its implications: unlike the model, clinicians can use prior scores to make informed assessments. If a joint was previously classified as J, it can only transition to J, E, or R in subsequent periods. However, the model treats each radiograph as an independent sample, lacking longitudinal context. This limitation may affect performance, as the model cannot leverage temporal progression information that clinicians naturally use. Future research could explore how to incorporate longitudinal data into model training to enhance predictive accuracy. It should be emphasized, however, that guidelines for clinical trials on hand osteoarthritis generally require radiographs to be blinded to sequence. For most studies, radiographs of each patient are presented to the readers in random order, with the date of acquisition masked. In studies that include inactive controls, some argue that blinding to sequence is not necessary as long as the readers are blinded to treatment allocation [39,40]. In conclusion, automating the V&V scoring system addresses two significant challenges: the time-consuming nature of manual scoring and the complexity that restricts its use to highly trained experts. The selection of patients for inclusion in therapeutic studies requires the participation of one or more experienced and well-trained readers. Our results showed that trained models performed comparably to expert clinicians on test sets, while completing assessments in a fraction of the time, and even reducing the risk of human error. Using a pretrained ResNet-50 and comparing conventional and ordinal classification approaches, we demonstrated that deep learning methods can support experts in the assessment process. Finally, such an AI assisted classification system will reduce the costs associated with therapeutical trials and enable large-scale studies. Author contributions Author contributions GV, ZO, PS and SL conceived the study. GV selected and labeled the data. ZO prepared deep learning strategy and analyzed the data. GV and ZO drafted the manuscript with critical input from all authors. PS and SL provided strategic direction and senior guidance. All authors critically reviewed and approved the final manuscript. Ethical approval Not applicable. Funding None. Declaration of competing interest The authors declare no competing interests.