YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception Authors: Marios Impraimakis, Daniel Vazquez, Feiyu Zhou Published: 2026 ABSTRACT AbstractThe trustworthy object detection capabilities of a novel Kolmogorov-Arnold network framework are examined here. The approach addresses a key limitation in computer vision for vehicle detection perception, and beyond. These systems offer limited transparency regarding the reliability of their confidence scores in visually degraded or ambiguous scenes. To this end, a Kolmogorov-Arnold network is employed as an interpretable post-hoc surrogate to model the trustworthiness of the You Only Look Once (Yolov10) detections using seven geometric and semantic features. The additive spline-based structure of the Kolmogorov-Arnold network enables direct visualisation of each feature’s influence. This produces smooth and transparent functional mappings that reveal when the model’s confidence is well supported and when it is unreliable. Furthermore, a bootstrapped language-image (BLIP) foundation model generates descriptive captions of each scene. This tool enables a lightweight multimodal interface without affecting the interpretability layer. Experiments on both Common Objects in Context (COCO), and images from the University of Bath campus demonstrate that the framework accurately identifies low-trust predictions under blur, occlusion, or low texture. This provides actionable insights for acceptance, review, or downstream risk mitigation. The resulting system delivers interpretable object detection with trustworthy confidence estimates. It offers a powerful tool for transparent and practical perception component for autonomous and multimodal artificial intelligence applications.Keywords: Trustworthy object detection, Kolmogorov-Arnold Network (KAN), You Only Look Once (YOLOv10), Interpretable Explainable AI (XAI)Subject terms: Engineering, Mathematics and computing Object detection systems have attracted interest through the development of the one-stage You Only Look Once models. These detectors achieve real-time performance, making them attractive for deployment in visual perception tasks in autonomous vehicles1–4, traffic5, pedestrian detection6–8, and object tracking9–13. Other areas include material images14–16, or medical applications17,18. To this end, Wang et al.19 proposed to train the You Only Look Once models20–25 without a filtering step for faster and more accurate behavior. Dong et al.26 developed a method that uses wavelet-based processing, improved feature pooling, and a combined loss design. Later, Luz et al.27 showed that combining Internet of Things devices, edge computing, and deep learning enables vehicle detection across hardware with widely varying processing speeds. Furthermore, Wang et al.28 improved the detection of tiny objects by mixing information from different scales and surrounding context29. For materials, Haoyan et al.30 proposed a lightweight model for steel-surface defect detection by replacing standard layers with an adaptive downsampling structure31,32. Chhimpa et al.33, later, demonstrated that fine-tuning an object detection model can effectively identify brain tumors in medical images. Furthermore, An et al.34 introduced a small-target model which improves crack detection using a new head structure, a detail-enhancing pyramid, and a loss function. Zhang et al.35 also proposed an improved network specifically for aerial vehicle detection, Ding et al.36 developed a model that uses attention and multi-scale processing to detect small objects from drone imagery, while Chen et al.37 added an edge-focused feature module, improved multi-scale processing, and pruning and distillation steps. For complex applications, Sun et al.38 also combined a transformer-based backbone with an attention-driven feature-fusion method to capture tiny details. Srinivasu et al.39 studied how adjusting model settings and expanding the training data affects the detection performance. Mahapadi et al.40 later optimized two nature-inspired search algorithms. Along these lines, Qin et al.41 proposed a classroom-behavior detection system which improves recognition under crowded and multi-scale conditions. Furthermore, Xie et al.42 described a model for autonomous-driving detection43–49 that uses a new gating module, an improved feature pyramid, a redesigned multi-scale network, and a double-distillation training strategy. In mechanical engineering, Liu et al.50 introduced an enhanced model designed to locate defects in aircraft-engine components. Finally, Meng et al.51 combine ideas from pose-estimation models for agricultural automation. Regarding Kolmogorov-Arnold networks proposed by Liu et al.52, Shuai et al.53 introduced later a physics-informed framework that embeds scientific laws into the learning process54–58. Wu et al.59 presented Graph Kolmogorov-Arnold networks60, while Howard et al.61 developed multifidelity Kolmogorov-Arnold networks that use inexpensive low-quality data. Hasan et al.62 combined sequence-learning networks to analyze brain-signal time series, while Baravsin et al.63 conducted a study across over one hundred time-series datasets for real-world signals. Furthermore, Wang et al.64 proposed convolutional Kolmogorov-Arnold networks, which aim to create smaller and more interpretable intrusion-detection models. Saravani et al.65 compared Kolmogorov–Arnold networks with machine-learning approaches, Ansar et al.66 modified the loss functions of Kolmogorov-Arnold networks incorporating a correlation measure to improve performance, while Ta et al.67 introduced fully-composed Kolmogorov-Arnold networks that mix mathematical basis functions to learn complex relationships from low-dimensional inputs. Finally, Panahi et al.68 proposed a general discovery framework using Kolmogorov-Arnold networks that can uncover governing equations from data. To the vision-language modeling end69–73, Xue et al.74 introduced a curated dataset, training procedures, and a family of large models that handle both images and text75. Yang et al.76 presented VisionZip, a method that keeps only the most useful visual tokens, Xu et al.77 proposed a model with chain-of-thought that performs its own multi-stage reasoning steps, while Vasu et al.78 introduced a new vision encoder that produces fewer visual tokens to greatly speed up processing. To this end, Deng et al.79 studied how vision-language models balance visual and textual information when they are given images and different kinds of text inputs, Zhou et al.80 introduced the first benchmark for both space and time in videos, while Shu et al.81 proposed video-extra-large which summarizes the content of each time segment. Deitke et al.82 presented newly created datasets, Zhan et al.83 also curated a large remote-sensing instruction dataset84, and Hu et al.85 created a high-quality remote-sensing image-captioning dataset for aerial images. Furthermore, Wu et al.86 explored segmenting farmland applications, Dai et al.87 investigated the interactions between people and their environments, and Xin et al.88 examined a decomposed three-dimensional encoder, an improved image-text alignment method, and a dual-stream projector for medical images with text descriptions. Related to interpretable artificial intelligence in engineering89–93, Singh et al.94 introduced DiaXplain, a diabetes-diagnosis tool to deliver understandable results, and Roy et al.95 developed a system that emphasizes transparency for clinicians, while Yao et al.96 proposed a method that detects interpretable discharge states. Chudasama et al.97 introduced TrustKG, a knowledge-graph-based approach that improves the interpretability and reliability of hybrid medical systems. Along these lines, Gliner et al.98 demonstrated a tool for electrocardiogram images, and Song et al.99 analyzed climate and lightning data to understand which conditions increase the likelihood of forest fire-related lightning100. Rashid et al.101 presented an ensemble-based disease-detection method, while Salvi et al.102 combined uncertainty estimation with interpretability methods. Finally, Agrawal et al.103 explored explanation techniques to improve the clarity of model reasoning. Related to trustworthy artificial intelligence, Chander et al.104 described a range of modern relevant techniques with Wirz et al.105 argued that trust in outputs depends both on how users perceive the system and on the specific decision situation. Cousineau et al.106 considered findings from interviews with developers showing that creating trustworthy artificial-intelligence systems is difficult due to immature standards, inconsistent regulations, unclear definitions, and limited practical tools. Stamboliev et al.107 analyzed European Union policy discourse and concluded that promoting trustworthy artificial intelligence serves as a shared idea that unites diverse political and industry stakeholders. Importantly, Reinhardt et al.108 examined the philosophical views of trust, Goisauf et al.109 provided an interdisciplinary analysis of trust, while Jannel et al.110 proposed the concept of trustability as a threshold that distinguishes appropriate trust in these systems. Finally, Parra et al.111 introduced the reason framework, which aims to integrate trustworthiness into the full life-cycle of future communication networks, and Moreno et al.112 proposed a design approach intended to help developers embed trustworthy principles into medical systems. However, the internal behavior of such models remains difficult to interpret: the learned decision surfaces are highly nonlinear, their dependence on geometric and semantic features is opaque, and their confidence outputs offer limited insight into why detections are accepted, rejected, or assigned particular confidence levels. To this end, a framework that couples You Only Look Once with a Kolmogorov-Arnold networks is examined to model and interpret the internal confidence predictions of the detector. In this work, the Kolmogorov-Arnold network model is supervised using the original You Only Look Once confidence scores as the target signal, thereby learning a structured and interpretable mapping from detection features to predicted confidence. The focus is on the structured seven-dimensional set of interpretable features available at the output of the detection head: normalized bounding-box position, normalized size, predicted confidence, class index, and relative image scale. Additionally, natural language explanations offer complementary insight by aligning model outputs with human linguistic reasoning. The combination of these models results in a unified interpretability pipeline: You Only Look Once performs object detection, the Kolmogorov-Arnold network provides a transparent surrogate that exposes the structure of the model’s confidence function, and the visual-language model produces natural language descriptions for human review. This multimodal design113,114 brings together visual, numerical, and linguistic interpretability within a single coherent framework. Specifically, You Only Look Once is a one-stage object detector designed to perform real-time bounding-box prediction that directly regresses object locations and class probabilities, seen in Fig. 1. The network receives an input image of size with approximately 8400 candidate detections. Features are used by the Kolmogorov-Arnold Network, seen as a framework in Fig. 2. The Kolmogorov-Arnold network seen in Fig. 3 is used as a surrogate confidence model with architecture width=[7,16,1], grid=5 and spline order=3 (cubic b-splines), meaning that the network receives seven interpretable numerical features derived from detections. This architecture directly enables the interpretability results. Large vision-language foundation models also aim to learn a joint representation between images and natural language, seen in Fig. 4. Architectural framework of You Only Look Once. C2f stands for coarse-to-fine, SPPF stands for spatial pyramid pooling fast, and PSA stands for partial self-attention. Proposed framework for the computer vision perception. Architecture of a Kolmogorov-Arnold network. Architectural framework of visual-language models. BLIP stands for Bootstrapped Language-Image Pretraining. The remainder of this paper is organized as follows. Section Results presents the main results including the feature-level analyses, partial-dependence behaviour, hidden-unit specialization, edge-importance structure, fidelity evaluation, and qualitative interpretability examples on both the COCO dataset and University of Bath images. Section Discussion provides a discussion of the interpretability findings, and concludes the work. Section Methods describes the methodological foundations, detailing the You Only Look Once confidence formulation, the Kolmogorov-Arnold network architecture and spline-based representation, and the vision-language model used to generate descriptive captions. Results The modeling framework is examined on the Common Objects in COntext dataset focused on vehicle detection tasks. The goal is to characterize the model’s internal confidence behavior using the Kolmogorov-Arnold network, and to demonstrate how the spline-based structure provides transparent explanations of the detector’s geometric and semantic sensitivities. Here, You Only Look Once confidence scores are used as the target signal for Kolmogorov-Arnold network. The model, shown in Fig. 5, receives seven input features derived from You Only Look Once detections; bounding-box geometry, predicted confidence, class index, and image scale, and outputs a smooth approximation of the model’s confidence. The darker edges in the schematic indicate stronger learned importance. The mean absolute spline activation per input feature is shown in Fig. 6; here, the class index and confidence values dominate the activation magnitudes, while geometric terms (x, y, w, h) exhibit weaker but consistent influence. Kolmogorov-Arnold network model architecture with seven interpretable inputs and one output. Mean absolute spline activation per input feature. Figures 7, 8, 9, 10 present the partial dependence plots for individual features, demonstrating how the Kolmogorov-Arnold network output varies when only one input is changed. The dependence on bounding-box width (Fig. 7) shows a smooth monotonic increase, indicating that the model assigns higher confidence to wider objects. Height (Fig. 8) yields a similarly smooth but slightly weaker trend. The dependence on vertical position y (Fig. 9) is shallow with mild curvature, consistent with weak geometric sensitivity. Image scale (Fig. 10) exhibits only a small positive effect, confirming that detection confidence is not strongly biased across image resolutions. Across all plots, the curves are smooth and stable, illustrating that the spline components of the surrogate generalize without overfitting. Figures 11 and 12 show the feature-unit correlation bars. They provide fine-grained interpretability of the internal structure of the surrogate. The full input–hidden edge importance heatmap is also shown in Fig. 13, revealing that class and confidence supply strong signals to specific neurons, while geometric inputs connect more weakly. Figures 14, 15, 16 show the collection of spline functions learned by the Kolmogorov-Arnold network for all feature–unit pairs. These visualizations demonstrate the smoothness and diversity of the learned transformations. Each figure corresponds to one group of spline subplots. Finally, Figures 17, 18, 19, 20, 21, 22, 23 illustrate the application of the full system to real scenes. You Only Look Once detections are shown alongside Kolmogorov-Arnold network-predicted confidence estimates and vision-language captions. In high-quality images such as the truck (Fig. 17), police car (Fig. 18), and sand-car scenes (Fig. 19), the surrogate and detector agree closely. For more challenging examples featuring occlusions, blur, or clutter (Figs. 20, 21, 22, 23), the Kolmogorov-Arnold network highlights regions where confidence becomes less predictable, demonstrating the surrogate’s ability to expose when detection reliability decreases. The numerical interpretability complements the linguistic explanations generated by the vision-language model. Partial dependence of the Kolmogorov-Arnold network on bounding-box width. Partial dependence on bounding-box height. Partial dependence on vertical position y. Partial dependence on image scale. Feature influence on a representative hidden unit. Feature influence on a different hidden unit. Input–hidden edge importance heatmap. Spline functions for all feature - unit pairs (Group 1, horizontal-axis: feature value, vertical-axis: spline activation). Spline functions for all feature - unit pairs (Group 2). Spline functions for all feature - unit pairs (Group 3). Application to a truck scene. Application to a police car. Application to a vehicle in sand. This image is from the Common Objects in Context dataset, available at https://cocodataset.org. All human images appearing in this dataset are provided under its licensing terms. Application to a van example. Application to a challenging scene with partial occlusion and blur. Application to a scene with multiple objects. Application to a scene with unclear objects. Table 1 summarizes the per-feature statistics extracted from the Kolmogorov-Arnold network. The mean spline activation identifies class and confidence as the dominant drivers of the model’s confidence behavior, while the partial dependence deltas quantify each feature’s global effect across its domain. Geometric inputs play a secondary role, and the uniformly small saliency values demonstrate that the network remains smooth and stable. Table 2 provides a detailed description of hidden-unit behavior within the Kolmogorov-Arnold network. Strong correlations between hidden-unit activations and the class feature reveal a consistent specialization pattern, with a few neurons carrying disproportionately large influence over the output. This structure supports a transparent interpretation of the model, where individual units serve identifiable semantic functions rather than mixing signals in an opaque manner. Table 3 presents the complete input–hidden edge importance matrix computed from spline coefficients. The results show strong connections from class and, to a lesser extent, confidence, feed into specific hidden units, while geometric features contribute weaker. Table 4 consolidates all interpretability metrics into a unified feature-influence score. The strong agreement between normalized statistics confirms that class and confidence dominate the model’s confidence landscape from multiple perspectives; activation strength, global effect, and structural edge contribution. This agreement reinforces the internal coherence of the network and provides a robust grounding for the qualitative partial dependence plots analyses. Table 5 evaluates the Kolmogorov-Arnold network fidelity across quantile bins for each feature. The model maintains stable fidelity across most of the feature space, demonstrating that the additive spline formulation accurately captures the functional relationship between the model’s confidence and its input parameters. The slight reductions in fidelity for the lowest and highest confidence bins align with the expected difficulty of modeling edge-case detections. Finally, Table 6 shows the monotonicity scores, offering a global characterization of the directionality of the model’s confidence behavior. In addition, Table 7 extends the qualitative analysis by introducing a quantitative trust assessment scoring system based on the discrepancy between YOLO and KAN confidence scores. The defined threshold-based categorization (accept, review, consider) provides a mechanism which transforms visual comparisons into actionable outcomes. Table 8 presents the sensitivity analysis evaluating the contribution of individual input features and spline configurations to the KAN model. The geometric features (x, y, w, h) and image scale exhibit only a marginal influence, with minimal degradation observed upon their removal. Interestingly, excluding the class feature slightly improves performance, confirming that its apparent importance arises from its numerical encoding rather than meaningful semantic contribution. Table 9 presents a quantitative runtime analysis. The proposed framework maintains consistent performance, with the increased variance primarily driven by the image-dependent complexity of the detection stage. Table 1Feature-level statistics from the network.FeatureSplineActivationSaliencyPD Deltax0.1072818041.43-0.007746y0.1062764102.64-0.025138w0.0618907364.170.056968h0.0689723642.35-0.045587conf0.1913500001.060.706939cls3.7319320001.670.029724scale0.0827071202.77-0.0269816 Feature-level statistics from the network. Table 2Hidden-node statistics from the network.NodeActivationImportanceFeatureCorrelationn04.9088082.2996383cls0.999684334n13.89728380.19116642cls−0.999700487n22.43024350.10419141cls−0.996881783n36.0319183.575461cls0.999689877n45.6544853.7371411cls0.999682188n56.20708275.189023cls0.999588966n65.1031790.22900744cls−0.999275625n70.484215170.27280924cls−0.947130859n81.47902720.48364034cls−0.993801713n90.66062820.2996985conf−0.958717942n105.85336882.5290031cls0.999058664n119.3278250.38113362cls−0.999662876n122.33409480.15058956cls−0.997766674n130.47848460.076826476cls0.985139787n142.455560.08944808cls−0.999845147n152.60138321.7512177cls0.997587681 Hidden-node statistics from the network. Table 3Input-hidden feature influence values for each Kolmogorov-Arnold network hidden unit.Nodexywhconfclsscalen00.0298540160.0154238120.0029574780.0144971100.0467152402.2959375000.005541976n10.0005796060.0022485350.0016349800.0019249500.0039631630.1910896000.000456766n20.0005058800.0025970630.0020211600.0027410360.0050817950.1033935550.000257480n30.0114413490.0207717120.0319169570.0326816030.0859059203.5804431000.004258949n40.0233025300.0280582900.0266509150.0244210940.0559091533.7466292000.022205167n50.0393371540.0188115130.0399057980.0247998820.1369808505.1755233000.015163474n60.0034206510.0008052970.0010676190.0017930400.0062439260.2279850700.002086347n70.0127056870.0144797170.0044358830.0031442990.0864338060.2451385400.001393769n80.0064207810.0127922370.0099232250.0061435170.0566872470.4897893700.000277586n90.0458702070.0216160140.0541482420.0035106250.3055996000.0257786900.019549502n100.0214603320.0171674960.0172012780.0293651480.0958614952.5446603000.007906381n110.0001123050.0022321170.0012921590.0032631230.0072557360.3799502000.000122972n120.0023349080.0010764870.0008198060.0010199870.0090172140.1488957300.000324138n130.0099488540.0053388170.0036236440.0065260130.0031956040.0745847900.000295031n140.0001782300.0010639580.0007438220.0007462060.0007685380.0890537300.000452965n150.0197590160.0172857290.0239380410.0107373620.1151407101.7335058000.001825250 Input-hidden feature influence values for each Kolmogorov-Arnold network hidden unit. Table 4Feature influence metrics combining spline activation, saliency, partial dependence delta, and edge importance.FeatureSplineActSaliencyPDP_DeltaEdgeImportanceInfluencex0.107282.86-0.00770.227230.01739y0.106285.29-0.02510.181770.01392w0.061898.340.056970.222280.04231h0.068974.70-0.04560.167310.00371conf0.191352.120.706941.020760.52001cls3.731933.330.0297221.052360.52559scale0.082715.55-0.026980.082120.01082 Feature influence metrics combining spline activation, saliency, partial dependence delta, and edge importance. Table 5Kolmogorov-Arnold network fidelity across overall data and per-feature quantile bins.ScopeFeatureBinIndexBinRangeNR2MAERMSEOverall90980.995684670.0108576250.01510572Per-Featurex0[0.0078, 0.2581]18200.9955438740.0107388880.014922408Per-Featurex1[0.2581, 0.4369]18190.9955848500.0111357950.015436348Per-Featurex2[0.4369, 0.5569]18200.9952430650.0116859960.016162014Per-Featurex3[0.5569, 0.7306]18190.9957113810.0107152230.015072694Per-Featurex4[0.7306, 0.9956]18200.9961624950.0100122950.013840625Per-Featurey0[0.0090, 0.3730]18200.9937379230.0125182460.017162563Per-Featurey1[0.3730, 0.4995]18190.9962169000.0102240110.014291467Per-Featurey2[0.4995, 0.5889]18200.9965463710.0096005220.013792516Per-Featurey3[0.5889, 0.7164]18190.9959773980.0102525800.014582687Per-Featurey4[0.7164, 0.9839]18200.9950566660.0116920830.015466231Per-Featurew0[0.0078, 0.0633]18200.9945223310.0099167740.013265161Per-Featurew1[0.0633, 0.1175]18190.9960036010.0094205150.013088631Per-Featurew2[0.1175, 0.1942]18200.9963161090.0096625370.013505891Per-Featurew3[0.1942, 0.3645]18190.9955649890.0108143560.015195336Per-Featurew4[0.3645, 1.0000]18200.9929651350.0144731270.019501280Per-Featureh0[0.0134, 0.0996]18200.9936205860.0108334440.014169815Per-Featureh1[0.0996, 0.1767]18190.9947575250.0110149640.014902389Per-Featureh2[0.1767, 0.2982]18200.9946445700.0112129990.015743511Per-Featureh3[0.2982, 0.5189]18190.9949239020.0111664110.015544278Per-Featureh4[0.5189, 1.0000]18200.9951953390.0100605620.015118543Per-Featureconf0[0.2501, 0.3561]18200.6461036340.0130542170.018254275Per-Featureconf1[0.3561, 0.5248]18190.9080174820.0108150610.015045941Per-Featureconf2[0.5248, 0.7179]18200.9537213450.0092432340.012182652Per-Featureconf3[0.7179, 0.8677]18190.8800901880.0112911970.014700041Per-Featureconf4[0.8677, 0.9854]18200.6841497330.0098846280.014724099Per-Featurecls1[0.0000, 3.0000]35580.9990257150.0053205890.007107449Per-Featurecls2[3.0000, 29.0000]18780.9929356240.0149384660.019866999Per-Featurecls3[29.0000, 52.0000]18330.9921111450.0146693380.018721599Per-Featurecls4[52.0000, 79.0000]18290.9946338380.0136187400.016740484Per-Featurescale0[0.0745, 0.6250]18000.9953708060.0113330280.015715895Per-Featurescale1[0.6250, 0.6672]13030.9957585320.0105120750.014713371Per-Featurescale2[0.6672, 0.7375]23430.9958336730.0106585010.014828051Per-Featurescale3[0.7375, 0.7500]2250.9953843400.0115052650.015818063Per-Featurescale4[0.7500, 1.0000]34270.9957322020.0108329230.015066167 Kolmogorov-Arnold network fidelity across overall data and per-feature quantile bins. Table 6Monotonicity analysis of each input feature.FeatureMonotonicityScoreDirectionStrengthx0.021862028Flat/WeakWeaky0.034401234Flat/WeakWeakw0.376517522PositiveModerateh0.449024012PositiveModerateconf0.996658272PositiveStrongcls−0.146067093NegativeWeakscale0.022657916Flat/WeakWeak Monotonicity analysis of each input feature. Table 7A quantitative trust assessment scoring system using the KAN-YOLO confidence discrepancy (Δ) and BLIP captions.ImageYOLOKANDecisionBLIP CaptionFig. 170.860.860.00AcceptA red truck parked in front of a houseFig. 180.900.900.03AcceptA police car parked on the side of the roadFig. 190.910.890.02AcceptA man is standing next to a car in the sandFig. 200.940.930.01AcceptA white van with a blue and yellow designFig. 210.880.610.27ConsiderA tree that has been cut down and it is being removedFig. 220.420.230.19ReviewA bench on the side of a streetFig. 230.900.640.26ConsiderA tree stump on the side of a roadDecision rule: Accept (), Review (), Consider (Δ> 0.25). A quantitative trust assessment scoring system using the KAN-YOLO confidence discrepancy (Δ) and BLIP captions. Decision rule: Accept (), Review (), Consider (Δ> 0.25). Table 8Sensitivity analysis of the YOLO features for the KAN predictions, with indicated importance of vision-language captions for human interpretation and intervention.ExperimentR2MAEBLIP ImportanceFull model (7 features)0.9960.0109Case dependentRemove x0.9950.0116Case dependentRemove y0.9940.0141Case dependentRemove w0.9950.0116Case dependentRemove h0.9950.0122Case dependentRemove conf−5.1030.518HighRemove cls0.9960.0057Case dependentRemove scale0.9920.0157High Sensitivity analysis of the YOLO features for the KAN predictions, with indicated importance of vision-language captions for human interpretation and intervention. Table 9Runtime analysis of YOLO, KAN, and the combined system.ComponentMean (ms)Std (ms)DescriptionYOLOv10174383Baseline detectorKAN156Interpretability moduleYOLO+KAN215446Combined systemOverhead: 41 ms relative to YOLO inference time Runtime analysis of YOLO, KAN, and the combined system. Discussion The interpretability analysis produced by the Kolmogorov-Arnold network reveals several consistent patterns which require discussion for further potential applications115–127. The unified feature influence table is the most essential as it captures both the spline-activation behaviour and the structural edge importance values, effectively integrating the core components. The results show that the surrogate processes each input feature through dedicated spline transformations, highlighting the interpretability advantage of additive univariate nonlinearities. Importantly, unlike geometric features, image-quality descriptors are not implicitly represented in the input space. However, their influence is already embedded within the YOLO confidence score, which reflects detection reliability under varying visual conditions, thereby eliminating the need for explicit inclusion. The reduced fidelity observed at the lowest and highest confidence bins in Table 5 can be attributed primarily to data distribution effects rather than limitations of the framework. Low-confidence detections inherently exhibit higher uncertainty and noise, making their behavior more difficult to approximate, while high-confidence samples are relatively sparse, reducing the effective training signal in this region. Importantly, the class index is treated as a scalar input in the model. This provides a compact representation of class-conditioned variations in detection confidence, which are then approximated using smooth spline functions. Here, the sensitivity study in Table 8 shows that removing the class feature does not degrade performance and can even slightly improve it. Embedding-based encodings could be explored in future work to better represent categorical information. Conclusion This work presented a self-aware framework that enhances the transparency and trustworthiness of You Only Look Once detections through an interpretable Kolmogorov-Arnold network. The visual-language captioning component assisted the perception pipeline with lightweight multimodal descriptions. The framework provides an interpretable approach with: Numerical insight into whether You Only Look Once’s confidence values are trustworthy. Numerical insight into whether You Only Look Once’s confidence values are trustworthy. Real-time and practical provision of confidence across the full feature domain, while simultaneously identifying unreliable detections in visually ambiguous scenes where the detector may fail. Real-time and practical provision of confidence across the full feature domain, while simultaneously identifying unreliable detections in visually ambiguous scenes where the detector may fail. Analysis of hidden units and monotonicity patterns for a coherent internal structure, with specialised neurons that consistently encode class information, confidence trends, and interpretable geometric influences. Analysis of hidden units and monotonicity patterns for a coherent internal structure, with specialised neurons that consistently encode class information, confidence trends, and interpretable geometric influences. A multimodal extension using vision-language descriptive scene-level context. A multimodal extension using vision-language descriptive scene-level context. Importantly, the findings show that the proposed framework enables detection with more reliable confidence estimation, strengthening the transparency and robustness of modern automated systems. Methods You Only Look Once is a one-stage object detector. Given an input image , the model computes a hierarchy of feature maps through a backbone network, producing multiscale feature tensors that encode semantic and spatial information. At each spatial location on each feature map, the model predicts a bounding box parameterized by its center and dimensions: together with a score o ∈ [0,1] and a distribution over classes p(c|I). The final confidence for a detection is computed as: which serves as the scalar quantity for the Kolmogorov-Arnold Network. The network receives an input image of size , which is processed by the backbone network to extract hierarchical visual features. The backbone primarily consists of convolutional layers and coarse-to-fine modules that progressively learn spatial and semantic representations from the input image. These extracted features are then passed to the neck component to aggregate multi-scale contextual information and enhance feature fusion. The refined feature maps are subsequently forwarded to the detection head, before the model produces a set of dense predictions corresponding to potential object locations and class probabilities across the image, resulting in approximately 8400 candidate detections. Let denote the network’s predicted box and denote the ground truth box assigned to that feature-map location, the model employs a differentiable intersection over union (IoU) metric-based loss function as: The training objective is expressed as: where are balancing coefficients. Each detection consists of the tuple , from which seven numerical features are used by the Kolmogorov-Arnold network. These are the normalized spatial position (x, y), the normalized size (w, h), the predicted confidence , the class index c, and the relative image scale . The Kolmogorov-Arnold network is inspired by the Kolmogorov-Arnold representation theorem, which states that any multivariate continuous function can be expressed as a finite superposition of univariate continuous functions. Kolmogorov proved that there exist continuous functions and such that: Kolmogorov–Arnold networks operationalize this idea by replacing the learned functions with trainable spline functions to every connection between input feature k and hidden unit j, and replacing with linear combinations of hidden-unit activations as: where d is the input dimensionality. The model output is then obtained as: where m is the number of hidden units and are learned output weights. The model input signals are fed into a hidden layer in which each neuron applies a learnable univariate spline function, shown in the diagram as a white spline curve inside each hidden unit. The outputs of the hidden spline neurons are subsequently combined by a standard output neuron to produce the final prediction. In this work, the surrogate confidence model is a Kolmogorov-Arnold network with architecture width=[7,16,1], grid=5 and spline order=3 (cubic b-splines), meaning that the network receives seven interpretable numerical features derived from detections. These features consist of normalized bounding–box geometry (x, y, w, h), predicted confidence, discrete class index, and relative image scale . Finally, the model learning form is: where is the vector of You Only Look Once-derived features. This architecture directly enables the interpretability results. Partial-dependence plots arise naturally from evaluating a single spline while holding other dimensions fixed. Specifically, monotonicity emerges from the smoothness properties of the fitted splines, hidden-unit roles can be inferred from correlations between spline outputs and input features, and input-hidden edge importance corresponds to norms of learned spline coefficients. Importantly, each input–hidden connection uses a learnable spline made from B-spline basis functions with grid = 5 and spline order = 3, each with its own trainable coefficient. During training, these coefficients adjust the curve shape, enabling smooth and interpretable nonlinear mappings while keeping the model simple and stable. Large vision-language foundation models aim to learn a joint representation between images and natural language. An input image I and a textual sequence are embedded into a unified representation followed by a joint conditioning module: where, consists of stacked transformer blocks, and and are the unified representations of the input. The generative language model then predicts a probability distribution over the vocabulary conditioned jointly on vision and text as: where, is the decoder hidden state produced from the fused representation H. The input image is first processed by the processor, which performs the required tokenisation, normalisation, and embedding transformations. These processed embeddings are then passed to a vision encoder and a text decoder, which jointly generate a natural-language caption describing the visual content of the image. Bootstrapped language-image pretraining optimizes a contrastive objective of the form: where is a cosine-similarity operator and τ is a temperature parameter. The model functions as a language-based interpretability layer that complements the numerical explanations provided by the Kolmogorov-Arnold network. For each detection image I, BLIP receives the visual input and produces a caption by sampling from the conditional distribution p(T|I) given by the decoder. Acknowledgements The authors gratefully acknowledge the Microsoft Research (MSR) team for the dataset, while additional data were collected from the University of Bath campus in UK. Author contributions M.I. conceived the study, developed the methodology, implemented the models, performed the experiments, and wrote the original manuscript. D.V. and F.Z. contributed to the conceptual development, provided intellectual input, assisted with visualisation, and contributed to the discussion and interpretation of the results. Funding The authors gratefully acknowledge the University of Bath for supporting the publication fee associated with this work. The authors received no external funding for the research, development, or authorship of this work. Data availability The data that support the findings of this study are available from publicly accessible sources and locally collected datasets. The Common Objects in Context (COCO) dataset is publicly available. Additional image data collected from the University of Bath campus are available from the corresponding author upon reasonable request. Code availability The code used to support the findings of this study is available from the corresponding author upon reasonable request. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The data that support the findings of this study are available from publicly accessible sources and locally collected datasets. The Common Objects in Context (COCO) dataset is publicly available. Additional image data collected from the University of Bath campus are available from the corresponding author upon reasonable request. The code used to support the findings of this study is available from the corresponding author upon reasonable request.