Morphology-Aware KOA Classification: Integrating Graph Priors with Vision Models
Marouane Tliba 1,3 , Mohamed Amine KERKOURI 2 , Yassine Nasser 2 , Nour Aburaed 5 Aladine Chetouani 3 , Ulas Bagci 4 , Rachid Jennane 1 · 2025
Read Morphology-Aware KOA Classification: Integrating Graph Priors with Vision Models by Marouane Tliba 1,3 , Mohamed Amine KERKOURI 2 , Yassine Nasser 2 , Nour Aburaed 5 Aladine Chetouani 3 , Ulas Bagci 4 , Rachid Jennane 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21801.txt.
Introduction
Marouane Tliba and six co-authors, Mohamed Amine Kerkouri, Yassine Nasser, Nour Aburaed, Aladine Chetouani, Ulas Bagci, and Rachid Jennane, work on grading knee osteoarthritis from X-ray images. The damage signs are subtle shape details that ordinary image models often miss, so the team adds explicit information about the joint's anatomy.
A segmentation tool first outlines the bones, and those outlines become a graph describing the joint's structure. The graph description is then aligned with what a vision encoder sees in the X-ray, so shape knowledge guides the reading. On a large public osteoarthritis dataset, the combined system beat image-only baselines by up to ten percent in accuracy and also passed earlier leading methods on both accuracy and F1 score.
Catalog record
| CREATOR | Marouane Tliba 1,3 , Mohamed Amine KERKOURI 2 , Yassine Nasser 2 , Nour Aburaed 5 Aladine Chetouani 3 , Ulas Bagci 4 , Rachid Jennane 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21801
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Morphology-Aware KOA Classification: Integrating Graph Priors with Vision Models Marouane Tliba 1,3 , Mohamed Amine KERKOURI 2 , Yassine Nasser 2 , Nour Aburaed 5 Aladine Chetouani 3 , Ulas Bagci 4 , Rachid Jennane 1 Abstract Knee osteoarthritis (KOA) diagnosis from radiographs remains challenging due to the subtle morphological details that standard deep learning models struggle to capture effectively. We propose a novel multimodal framework that combines anatomical structure with radiographic features by integrating a morphological graph representation—derived from Segment Anything Model (SAM) segmentations—with a vision encoder. Our approach enforces alignment between geometry-informed graph embeddings and radiographic features through mutual information maximization, significantly improving KOA classification accuracy. By constructing graphs from anatomical features, we introduce explicit morphological priors that mirror clinical assessment criteria, enriching the feature space and enhancing the model’s inductive bias. Experiments on the Osteoarthritis Initiative dataset demonstrate that our approach surpasses single-modality baselines by up to 10% in accuracy (reaching nearly 80%), while outperforming existing state-of-the-art methods by 8% in accuracy and 11% in F1 score. These results underscore the critical importance of incorporating anatomical structure into radiographic analysis for accurate KOA severity grading. † † address: Author Affiliation(s) † † address: 1 University of Orleans, Orleans, France; 2 F-initiatives, Paris, France; 3 University of Sorbonne Paris Nord, Paris, France; 4 Northwestern University, Chicago, USA; 5 University of Dubai, Dubai, UAE 1 Introduction Knee osteoarthritis (KOA) is a degenerative joint disease marked by progressive cartilage loss, osteophyte formation, and subchondral bone remodeling, leading to debilitating symptoms such as chronic pain, stiffness, and swelling. Radiography remains the gold standard for diagnosing KOA due to its widespread availability, low cost, and rapid imaging capabilities. Radiographic assessment relies on the Kellgren-Lawrence (KL) grading system [ 1 ] , which classifies disease severity into five stages (0–4) based on structural biomarkers like osteophyte and joint space narrowing (JSN) width [ 2 ] . Recently, various deep Learning approaches have emerged to diagnose knee OA using X-ray images [ 3 , 4 , 5 ] . In [ 6 ] , Nguyen et al. proposed a semi-supervised framework that leverages the mixup algorithm to synthesize out-of-distribution samples, enforcing prediction consistency and improving model robustness in scenarios with scarce labeled training data. Nasser et al. [ 7 ] propose a Discriminative Shape-Texture Convolutional Network that embeds a Gram Matrix Descriptor block to compute texture features from intermediate CNN layers [ 8 ] , which are fused with high-level shape features to improve early-stage OA detection. [ 9 ] employed an a hierarchical representation learning approach that helps propagate low level features deeper into the network, thus enriching the representation ability of the network. While deep learning has shown promise in automating KOA assessment, conventional vision models often overfit to local textural artifacts rather than relevant anatomical structures. Most of SoTA methods work directly on the raw pixel image and neglect the fact that the KL grading system is based on structural biomarkers [ 1 , 5 , 2 ] . In this work, we address these challenges by redefining how structural relationships are encoded. We posit that the spatial and morphological relationships between the femoral and tibial bones provide critical diagnostic information. To capture this, we construct a graph from SAM-derived joint masks [ 10 ] , explicitly encoding the anatomical geometry of the joint. This graph, which reflects key morphological attributes (e.g., joint space width, bone curvature), is leveraged to generate compact, discriminative embeddings that complement the deep features extracted by a vision backbone. Motivated by the need to reduce high-dimensional pixel-level complexity and guide learning toward anatomy-aware representations, our framework integrates graph-based and vision-based modalities. By minimizing a hybrid objective function, we force our network to align these complementary feature spaces through mutual information maximization and a learnable cross-modal translation module while optimizing for accurate KOA classification. Overall, our work aims to develop a robust and interpretable diagnostic framework that overcomes the limitations of conventional models by incorporating explicit anatomical priors derived from graph representations. These priors enrich feature learning with clinically validated biomarkers (e.g., KL-grade severity criteria), bridging the gap between data-driven learning and radiological expertise. 2 Proposed Method We propose a fully automated end-to-end multimodal approach that integrates graph-based morphological cues with radiographic features for robust KOA severity assessment. An overview of the full pipeline is depicted in Fig. 1 . 2.1 Automatic Graph Construction Mask Selection. Given an input radiograph X\in\mathbb{R}^{H\times W\times C} , we prompt the Segment Anything Model (SAM) [ 10 ] using two dense point grids to generate a set of candidate segmentation masks, \mathcal{M}=\{m_{i}\mid i=1,2,\dots,M\} . To isolate the joint region, we identify the optimal masks m^{*}_{U} and m^{*}_{L} for the upper and lower bones, respectively, by comparing each m_{i} against predefined anatomical templates T_{U} and T_{L} .These templates are crafted to capture representative femoral and tibial morphologies spanning the full spectrum of KOA severity. The selected mask m^{*} maximizes the intersection-over-union (IoU) with its corresponding template: m^{*}=\arg\max_{m\in\mathcal{M}}\mathrm{IoU}(m,T). Graph Construction. From the segmented joint boundary, we uniformly sample N points \{p_{i}\}_{i=1}^{N} , where each p_{i}=(x_{i},y_{i})\in\mathbb{R}^{2} ensures equidistant coverage of the bone contour. These points form the vertex set V=\{p_{i}\} of an undirected graph G_{\mathrm{joint}}=(V,E). To define the edge set E , we perform a k -nearest neighbor search under a threshold \tau , retaining only edges in the relevant region. Specifically, for each p_{i}\in V , let: \mathcal{N}(p_{i})=\left\{p_{j}\;\middle|\;\|p_{i}-p_{j}\|_{2}\leq\tau\right\}\cup\kappa_{k}(p_{i}), where \kappa_{k}(p_{i}) returns the k -nearest neighbors of p_{i} . The edge set is then defined as : \mathcal{E}=\left\{(p_{i},p_{j})\;\middle|\;p_{j}\in\mathcal{N}(p_{i})\right\} If this graph is initially disconnected, we iteratively increase \tau until full connectivity is achieved. The resulting G_{\mathrm{joint}} succinctly captures global joint morphology, offering a geometric prior for KOA assessment. Thus allows our approach to encode broader anatomical cues that might otherwise be neglected by local-intensity–driven models [ 8 , 11 ] . Figure 1 : Overview of our pipeline: First, SAM is prompted to generate candidate masks \mathcal{M} . The best mask m^{*} is chosen based on IoU with upper and lower bone templates (T_{U},T_{L}) . Next, a morphological graph G_{\mathrm{joint}} is constructed from the joint boundary and processed by the graph encoder f_{\mathrm{graph}}(\cdot;\Theta) . Simultaneously, the radiograph is passed through a vision encoder f_{\mathrm{vision}}(\cdot;\Theta) . The translation head T_{\mathrm{head}}(z_{\mathrm{vision}};\Theta) aligns the vision embedding with the graph domain, and the fusion module f_{\mathrm{merge}}([\;z_{\mathrm{graph}},\,z_{\mathrm{trans}}];\Theta) combines both representations. Both T_{\mathrm{head}} and f_{\mathrm{merge}} are linear multi perceptron layers. Finally, a linear layer produces the KOA severity logits \hat{y} . 2.2 MorphoGrpah : Graph Morphological Classification Building upon the joint graph G_{\text{joint}}=(V,E) , we employ a three-layer EdgeConv-style operation [ 12 ] to capture the geometrical relationships among its vertices. EdgeConv has proven effective in learning complex structural cues in point clouds and 3D shapes, making it well-suited for modeling the morphological geometry relevant to KOA. By stacking multiple layers, our network progressively refines the node embeddings, promoting higher-order interactions that highlight clinically significant bone features. Let \mathbf{X}^{(\ell)}\in\mathbb{R}^{|V|\times d_{\ell}} denote the node features at layer \ell . For each node i with neighbors \mathcal{N}(i) , EdgeConv produces an updated feature \mathbf{x}_{i}^{(\ell+1)} by aggregating local subgraphs as: \mathbf{x}_{i}^{(\ell+1)}=\max_{j\in\mathcal{N}(i)}\,\phi_{\Theta}\!\left(\left[\mathbf{x}_{i}^{(\ell)},\,\mathbf{x}_{j}^{(\ell)}-\mathbf{x}_{i}^{(\ell)}\right]\right) (1) where \bigl[\cdot,\cdot\bigr] denotes feature concatenation, and \phi_{\Theta}(\cdot) is an embedding layer parameterized by learnable weights \Theta . Conceptually, \phi_{\Theta} transforms both