A computer vision and dynamic strain fusion approach for urban bridge weigh-in-motion Authors: Yanjie Zhu, Yuchen Wang, Chenglin Gao, Wen Xiong Published: 2025 ABSTRACT Dynamic weighing systems are crucial for bridge safety, yet current weighing systems are either too costly for widespread urban use or unable to track vehicles reliably in dense traffic. Here we present a video–strain inverse estimation system that combines computer vision with structural strain sensing to measure dynamic vehicle loads on bridges. The method employs a lightweight deep learning detector to recognize vehicle features, a multi-object tracking model to capture trajectories and lane positions, and a developed analytical algorithm to estimate vehicle weight from measured strains. We validated the system with field data from a heavily trafficked urban bridge. The system achieves vehicle recognition with near-baseline accuracy while using only 17% of the original model parameters and running 1.72 times faster, identifies lanes with complete accuracy with a missed detection rate of just 0.56%, and estimates total vehicle weights within 2% error. This low-cost and reliable approach advances intelligent bridge monitoring and supports digital twins of critical urban infrastructure. Introduction The construction of new bridges is rapidly advancing, while the aging and deterioration of in-service bridges are becoming increasingly prominent. Consequently, ensuring long-term service has become the core goal of bridge maintenance1. Load testing is one of the most effective technical means for quickly assessing the operational safety of bridges2,3. However, this method usually involves fully or partially closing site traffic and requires designated load vehicles to perform tests at specific positions, severely impacting the normal operation of surrounding road networks4. Complete information on vehicles is crucial to load testing5. If vehicle weight, position, and time can be obtained in real-time without disrupting traffic, and structural responses (strains) can be measured via sensors, it becomes possible to conduct real-time light load tests under regular traffic flow, allowing continuous monitoring of bridge service safety6–9. The pavement weigh-in-motion system is a type of embedded equipment for testing vehicle loads without interrupting traffic10. As vehicle wheels pass over road-embedded sensors, axle weight information is obtained by measuring dynamic pressures11,12. However, due to high costs, destructive installation, susceptibility to damage, and difficulty in replacement, pavement weigh-in-motion systems have not been widely adopted13. Bridge Weigh-In-Motion (BWIM) systems have been a recent research focus, utilizing beam responses to directly deduce vehicle weights, axle weights, and axle spacings14,15. These systems offer uninterrupted traffic flow, high efficiency, easy installation, easy maintenance, and low costs16–18. A BWIM system comprises three parts: dynamic bridge responses, vehicle spatiotemporal information, and algorithms for the inverse estimation of vehicle weights. Among these, the Moses algorithm is particularly classic, recalculating axle and vehicle weights based on measured versus theoretical dynamic responses as vehicles traverse bridges. Many scholars have built on the Moses algorithm to enhance its accuracy and applicability. Deng et al.19,20 introduced the theory of virtual axles and beams, monitoring vehicle speed and axle count using only bending strain signals. Chatterjee et al.21 employed wavelet transformations to dynamically identify vehicle speeds, axle counts, and spacings. Ojio et al.22 proposed the strain area and support reaction methods using structural response energy ratios to inverse estimate vehicle weights. However, these methods generally apply only to simple bridge types and conditions, and complex traffic scenarios remain a major challenge for BWIM systems23. In recent years, with the rapid development of computer vision technology, continuous traffic monitoring imagery has effectively identified characteristics and spatiotemporal distributions of vehicles moving on bridge surfaces24. Zaurin et al.25 used background subtraction techniques for dynamic extraction and tracking of vehicle features in video images. Ojio et al.26 introduced a non-contact BWIM system, utilizing sub-millimeter camera arrays to monitor bridge mid-span deflections and identify traffic flow information. Chen et al.27 implemented vehicle trajectory tracking using computer vision, integrating this with weigh-in-motion systems to analyze spatiotemporal distributions of vehicle loads. Zhang et al.28 utilized the Faster Region-based Convolutional Neural Network (Faster-RCNN) network to detect vehicle features in drone video imagery. Although computer vision technology holds immense potential for recognizing vehicle operational spatiotemporal information, the increasing complexity of learning networks and models has led to a growing demand for computational power. Especially in scenarios involving real-time dynamic tracking and inverse estimation of vehicle weights, computer vision often requires processing large volumes of high-resolution video data and complex computational tasks, which traditional hardware platforms can no longer support. Current deep neural networks used for vehicle detection and tracking, such as You Only Look Once (YOLO)29 and Faster-RCNN30, typically require high-performance computing resources to process real-time video and extract features. These models, designed to ensure accuracy and robustness, typically include numerous convolutional layers, fully connected layers, and complex computation graphs, leading to a strong dependency on computational resources. In practical applications, without sufficient computational support, these complex networks often struggle to respond in real-time, thus impacting the immediate feedback on vehicle spatiotemporal information and bridge load states. These challenges are particularly acute within the urban landscape. The dense network of bridges, often characterized by shorter spans and subject to tight budgets, makes the widespread deployment of high-cost, computationally intensive monitoring systems economically unviable. This reality places a premium on computational efficiency. The need to process high-volume video data for real-time target detection and trajectory tracking, while simultaneously synchronizing it with dynamic strain measurements, poses a key technical dilemma. While existing deep learning models excel on static benchmarks, their computational demands often conflict with the real-time, low-cost hardware constraints of large-scale vehicle monitoring. More critically, the dynamic and often dense nature of urban traffic introduces complex, multi-vehicle loading events. In these scenarios, traditional inverse estimation algorithms become numerically unstable, struggling to accurately disentangle the superimposed structural responses. Therefore, a core difficulty lies not only in the real-time fusion of visual and strain data but in developing algorithms for both vehicle detection and weight inversion that are simultaneously fast and robust enough to handle the complexities of real-world urban traffic. Overall, despite the potential of computer vision technology in extracting vehicle features and recognizing spatiotemporal information, existing research still faces notable challenges in computational power requirements, real-time performance, data synchronization, and accuracy, which limits the achievement of rapid synchronization and real-time dynamic tracking of vehicle features, spatiotemporal information, and vehicle weight. These issues limit the practical application of existing technologies in bridge load monitoring systems. In response to the deficiencies of current bridge dynamic weighing system technologies, we propose a video-strain-based dynamic recognition system for vehicle load on bridges, which has been applied in practice on a steel truss bridge. We introduced the YOLOv7-MobileNetV3-SimAM target recognition algorithm and the Simple Online and Realtime Tracking with a Deep Association Metric (DeepSORT) tracking algorithm to extract vehicle spatiotemporal information, while establishing a system for the inverse estimation of vehicle weights based on the Back-Propagation Moses (BPMoses) method to calculate vehicle weights. Our results show that the detection model reduces computational demand by more than 80% while sustaining near-baseline accuracy, enabling efficient real-time deployment on standard hardware. The tracking module achieves precise spatiotemporal recognition, with counting accuracy above 99%, lane identification at 100%, and speed errors typically within 1%. The proposed inversion algorithm consistently constrains vehicle weight errors to within 2% in simple cases and remains stable under complex multi-vehicle conditions where traditional methods fail. Deployed continuously, the system produced high-fidelity load profiles confirming that operational loads remained well below design limits. These findings demonstrate a reliable and low-cost approach to intelligent bridge monitoring and the development of digital twins for critical infrastructure. Results and discussion Our study develops and validates an integrated system for real-time bridge traffic load monitoring, sequentially combining vehicle information recognition, dynamic traffic flow extraction, and inverse estimation of vehicle weight. We first present the performance of each core component and then evaluate the overall framework in a real-world application. Performance analysis of the lightweight model for dynamic vehicle information recognition Real-time structural monitoring requires a vehicle detection model that balances accuracy with computational efficiency for on-device deployment. To this end, we developed the YOLOv7-MobileNetV3-SimAM model. We benchmarked it against the standard YOLOv7 and YOLOv5 model to assess its suitability for resource-constrained hardware. Quantitatively, the proposed model demonstrates a radical reduction in computational and storage demands compared to the YOLOv7 baseline and the widely used YOLOv5 (Fig. 1a). We achieved an 82.3% reduction in model parameters (from 37.2 M to 6.6 M) and an 82.4% smaller storage footprint (from 74.8 MB to 13.2 MB). This substantial model compression is primarily attributed to the integration of the lightweight MobileNetV3 backbone, which utilizes depth-wise separable convolutions to minimize computational cost while effectively preserving feature representation. Critically, this efficiency translates to a 41.9% faster inference time than the YOLOv7 baseline on a standard CPU (337.1 ms vs. 580.4 ms), a decisive advantage for our target application on GPU-scarce edge hardware. Our model achieves a high F1 score of 0.957 and a recall of 0.943, representing a marginal decrease of just 1.3% and 1.4% respectively, compared to the YOLOv7 baseline (F1: 0.970, Recall: 0.956). Similarly, the mean Average Precision (mAP@0.5) sees only a slight 1.2% drop (0.754 vs. 0.763), highlighting the model’s exceptional efficiency-to-accuracy ratio.Fig. 1Performance comparison of the proposed model with baselines.a Comparison of key performance metrics for our proposed model (YOLOv7-MobileNetV3-SimAM), the YOLOv7 baseline, and YOLOv5s. Parameters are quantified in millions. Mean average precision (mAP) is calculated at an Intersection over Union threshold of 0.5. Inference times are measured in milliseconds (ms) per frame. b Normalized confusion matrices for the proposed model. c The baseline YOLOv7. d The baseline YOLOv5s. Diagonal elements represent the true positive rate (recall) for each class. The figure highlights our model’s superior CPU efficiency and its enhanced classification accuracy for common classes (e.g., Sedan car), alongside a more semantically consistent error pattern for challenging classes (e.g., 5 A truck). Performance comparison of the proposed model with baselines. a Comparison of key performance metrics for our proposed model (YOLOv7-MobileNetV3-SimAM), the YOLOv7 baseline, and YOLOv5s. Parameters are quantified in millions. Mean average precision (mAP) is calculated at an Intersection over Union threshold of 0.5. Inference times are measured in milliseconds (ms) per frame. b Normalized confusion matrices for the proposed model. c The baseline YOLOv7. d The baseline YOLOv5s. Diagonal elements represent the true positive rate (recall) for each class. The figure highlights our model’s superior CPU efficiency and its enhanced classification accuracy for common classes (e.g., Sedan car), alongside a more semantically consistent error pattern for challenging classes (e.g., 5 A truck). While these aggregate metrics provide a high-level summary, a micro-level analysis of the confusion matrices reveals crucial differences in the models’ behavior (Fig. 1b–d). For the high-frequency sedan car class, our model achieves a true positive rate of 0.97, outperforming both baselines and reducing misclassifications of background as sedans. More tellingly, the models’ responses to the challenging, low-frequency 5 A truck class reveal fundamental differences in their underlying architecture. As shown in Fig. 1c, d, the failure modes of the baseline models are scattered and less coherent: YOLOv7 misattributes instances across multiple classes (e.g., 0.31 to 4 A truck, 0.23 to 6 A truck), while YOLOv5 not only scatters predictions but critically misclassifies 14% of 5 A trucks as background, indicating a failure to even recognize the object’s presence. Our model displays a systematic and semantically coherent error pattern, consistently classifying all 5 A truck instances as the visually analogous 4 A truck. We attribute this structured and predictable failure mode to the architectural integration of the SimAM. As a parameter-free mechanism, SimAM enables the network to discern the importance of individual neurons and prioritize the most discriminative features without increasing model complexity. This appears to help preserve the core semantic features of truck-like vehicles. Consequently, when faced with an unfamiliar 5 A truck, the model defaults to the most semantically plausible category (4 A truck) rather than failing catastrophically. This behavior signifies a form of graceful degradation, which is highly desirable for safety-critical applications. Overall, our model provides a validated solution that simultaneously overcomes two critical barriers in dynamic bridge assessment: it is compact and fast enough for on-device, real-time deployment, and its architecturally ingrained robustness leads to more interpretable and reliable performance, even in cases of classification uncertainty. This work thus demonstrates a practical pathway toward developing highly efficient yet dependable systems for real-time bridge load monitoring. Performance analysis for dynamic traffic spatiotemporal information recognition To validate the system’s performance in recognizing dynamic traffic information, we conducted a multi-stage evaluation encompassing geometric accuracy, vehicle counting, and speed estimation. The foundation of our method rests upon establishing a precise mapping between the 3D bridge coordinates and the 2D image plane. We computed a projection transformation matrix from four calibration points (P1, P2, P5, P6) and independently verified its accuracy against two test points (P3, P4), as depicted in Fig. 2a. The analysis revealed a maximum spatial error of only 0.222 m (Fig. 2c). Relative to the ~15 m dimensions of the test area, this corresponds to a mapping error of just ~1.5%. This sub-meter geometric fidelity is not merely a calibration result but a critical prerequisite, as it establishes a highly accurate spatial framework essential for all subsequent dynamic measurements.Fig. 2Performance validation of the dynamic traffic information recognition method.a Schematic of the practical coordinate system on the bridge deck and its mapping to the camera’s pixel coordinate system. Points P1, P2, P5, and P6 were used for calibration, while P3 and P4 served as independent test points. b Vehicle counting and lane assignment accuracy assessment during a high-traffic morning period (Scenario 1, 06:30-06:40) and a low-traffic night period (Scenario 2, 22:30-22:40). c Calculated coordinate errors for the independent test points P3 and P4, demonstrating the sub-meter accuracy of the geometric mapping. d Configuration of the vision-based speed detection system, illustrated through real-world, 3D, and top-down views showing the defined detection lines (L1, L2). e An example frame illustrating real-time vehicle detection, classification (sedan car), and tracking with confidence scores. f Probability distribution of the speed measurement error, comparing the vision-based system against a ground-truth radar. The sharp peak at zero with a narrow spread, highlighted in the inset, indicates high precision and low bias. g Cumulative frequency of speed errors, quantifying that 82.03% of measurements have an error of 1% or less. Together, these panels provide a comprehensive validation of the system’s high fidelity, from foundational geometric calibration to final speed estimation. Performance validation of the dynamic traffic information recognition method. a Schematic of the practical coordinate system on the bridge deck and its mapping to the camera’s pixel coordinate system. Points P1, P2, P5, and P6 were used for calibration, while P3 and P4 served as independent test points. b Vehicle counting and lane assignment accuracy assessment during a high-traffic morning period (Scenario 1, 06:30-06:40) and a low-traffic night period (Scenario 2, 22:30-22:40). c Calculated coordinate errors for the independent test points P3 and P4, demonstrating the sub-meter accuracy of the geometric mapping. d Configuration of the vision-based speed detection system, illustrated through real-world, 3D, and top-down views showing the defined detection lines (L1, L2). e An example frame illustrating real-time vehicle detection, classification (sedan car), and tracking with confidence scores. f Probability distribution of the speed measurement error, comparing the vision-based system against a ground-truth radar. The sharp peak at zero with a narrow spread, highlighted in the inset, indicates high precision and low bias. g Cumulative frequency of speed errors, quantifying that 82.03% of measurements have an error of 1% or less. Together, these panels provide a comprehensive validation of the system’s high fidelity, from foundational geometric calibration to final speed estimation. Building upon this precise geometric foundation, we evaluated the vehicle counting and lane identification performance under contrasting conditions: a high-density morning peak (06:30-06:40) and a sparse, challenging night period (22:30-22:40). The system demonstrated perfect performance at night, achieving 100% accuracy in both counting (215/215 vehicles) and lane assignment across all eight lanes (Fig. 2b). This resilience to low-light conditions and headlight glare is attributed to the robustness of the underlying deep-learning-based detector. During the morning peak, while lane identification remained perfect, a minor undercount of 4 out of 320 vehicles occurred (98.75% accuracy). Notably, these discrepancies were isolated to lanes 3 and 4, the two most congested lanes which carried 49% of the total traffic, strongly corroborating transient vehicle occlusion as the primary cause. Overall, the system achieved a 99.25% counting accuracy across 535 vehicle events, affirming its reliability in real-world scenarios. Finally, we evaluated the core function of vehicle speed recognition. Utilizing a dataset of 69,543 vehicle samples collected on March 20, 2023, we compared our vision-based speed measurements against a co-located roadside radar system (measurement error: 0.1 m/s), which served as the ground truth. The algorithm involves establishing detection lines (L₁ and L₂, 40 m apart) and tracking vehicle trajectories between them (Fig. 2d, e). The resulting error analysis reveals exceptional performance. The probability distribution of speed errors is characterized by a sharp, unimodal peak centered at zero, indicating a lack of systemic bias (Fig. 2f). Critically, 82.03% of all speed measurements exhibited an error of ≤1%, and 95.19% of measurements were within a ≤7% error margin (Fig. 2g). The rare instances of larger errors, with a maximum observed error of 15%, were confirmed through visual inspection to be caused by severe vehicle occlusion events. These results collectively demonstrate that the proposed system can deliver highly accurate and reliable vehicle speed data, making it a robust tool for real-time traffic monitoring. Performance analysis of the BPMoses algorithm for dynamic load inverse estimation To evaluate the performance and robustness of the proposed BPMoses algorithm, we conducted field experiments on an instrumented bridge. This section presents a comparative analysis of our method against the conventional Moses algorithm under four increasingly complex traffic scenarios: (1) single-vehicle, (2) two-vehicle following, (3) two-vehicle side-by-side, and (4) a three-vehicle mixed configuration (Fig. 3). Our analysis focuses on systematically probing the algorithms’ ability to accurately estimate dynamic axle and Gross Vehicle Weights (GVW) from strain gauge data sampled at 500 Hz. The core challenge lies in resolving individual axle loads, particularly in multi-vehicle events where dynamic responses are superimposed.Fig. 3Comparative performance of the Moses and Back-Propagation Moses (BPMoses) algorithms for inverse estimation of dynamic vehicle weights.The figure compares the conventional Moses algorithm with the proposed BPMoses algorithm across four traffic scenarios of estimation complexity. a Schematic of the single-vehicle baseline test configuration. b Axle weight identification errors for Vehicle 2 using the conventional Moses algorithm, showing large fluctuations. c Axle weight identification errors for Vehicle 2 using the BPMoses algorithm, showing stable and accurate estimates. d Axle weight identification errors for Vehicle 4 using the conventional Moses algorithm, showing instability and large deviations. e Axle weight identification errors for Vehicle 4 using the BPMoses algorithm, showing consistently accurate results. f Schematic of the two-vehicle following configuration. g Convergence of axle weight estimates using the Moses algorithm, showing slow convergence and large residual errors. h Convergence of Gross Vehicle Weight (GVW) estimates using the Moses algorithm, showing persistent error. i Convergence of axle weight estimates using the BPMoses algorithm, showing rapid and stable convergence. j Convergence of GVW estimates using the BPMoses algorithm, showing stable accuracy across all frames. k Schematic of the two-vehicle side-by-side configuration. l Axle weight estimation errors using the Moses algorithm, showing high inaccuracies. m GVW convergence using the Moses algorithm, showing slow stabilization. n Axle weight estimation errors using the BPMoses algorithm, showing improved accuracy. o GVW convergence using the BPMoses algorithm, showing rapid and stable performance. p Schematic of the three-vehicle combined scenario. q Axle weight estimation errors using the Moses algorithm, showing extreme errors exceeding 400%. r GVW estimation using the Moses algorithm, showing inconsistency and unreliability. s Axle weight estimation errors using the BPMoses algorithm, showing controlled and stable results. t GVW estimation using the BPMoses algorithm, showing reliable and accurate performance under complex conditions. Comparative performance of the Moses and Back-Propagation Moses (BPMoses) algorithms for inverse estimation of dynamic vehicle weights. The figure compares the conventional Moses algorithm with the proposed BPMoses algorithm across four traffic scenarios of estimation complexity. a Schematic of the single-vehicle baseline test configuration. b Axle weight identification errors for Vehicle 2 using the conventional Moses algorithm, showing large fluctuations. c Axle weight identification errors for Vehicle 2 using the BPMoses algorithm, showing stable and accurate estimates. d Axle weight identification errors for Vehicle 4 using the conventional Moses algorithm, showing instability and large deviations. e Axle weight identification errors for Vehicle 4 using the BPMoses algorithm, showing consistently accurate results. f Schematic of the two-vehicle following configuration. g Convergence of axle weight estimates using the Moses algorithm, showing slow convergence and large residual errors. h Convergence of Gross Vehicle Weight (GVW) estimates using the Moses algorithm, showing persistent error. i Convergence of axle weight estimates using the BPMoses algorithm, showing rapid and stable convergence. j Convergence of GVW estimates using the BPMoses algorithm, showing stable accuracy across all frames. k Schematic of the two-vehicle side-by-side configuration. l Axle weight estimation errors using the Moses algorithm, showing high inaccuracies. m GVW convergence using the Moses algorithm, showing slow stabilization. n Axle weight estimation errors using the BPMoses algorithm, showing improved accuracy. o GVW convergence using the BPMoses algorithm, showing rapid and stable performance. p Schematic of the three-vehicle combined scenario. q Axle weight estimation errors using the Moses algorithm, showing extreme errors exceeding 400%. r GVW estimation using the Moses algorithm, showing inconsistency and unreliability. s Axle weight estimation errors using the BPMoses algorithm, showing controlled and stable results. t GVW estimation using the BPMoses algorithm, showing reliable and accurate performance under complex conditions. We first established a performance baseline with single-truck crossings at speeds from 20 to 80 km/h. These tests immediately exposed a critical limitation of the conventional Moses algorithm. While it controlled GVW estimation error to within a respectable 3%, the underlying axle weight estimates were fundamentally unreliable, with errors for the first two axles fluctuating around 50% and the third axle around 15% (Fig. 3b, d). Such poor axle-level accuracy renders the method inadequate for detailed load assessment. In contrast, the BPMoses algorithm demonstrated superior accuracy and stability. It constrained GVW error to within 2% and, crucially, restricted individual axle weight errors to within a 15% margin across all speeds (Fig. 3c, e). This highlights its enhanced robustness and its capacity for reliable axle load estimation even in the simplest case. Next, we challenged the algorithms with multi-vehicle scenarios where superimposed dynamic responses create ill-conditioned systems. In a two-vehicle following test (50 km/h), the Moses algorithm’s vulnerability became evident. In brief temporal stages with sparse data (e.g., Stage 4 with 65 frames), the underlying influence matrix became severely ill-conditioned, causing the direct inversion to catastrophically amplify noise and yield physically impossible axle weights (e.g., −26.97 t for a 2.72 t axle, Table 1). The algorithm only converged to a reasonable GVW estimation in the data-rich Stage 6 (1105 frames), and even then required over 900 frames for the GVW error to stabilize below 2% (Fig. 3g, h). BPMoses, however, was impervious to data sparsity. Its iterative gradient-descent approach, moderated by a learning rate, effectively regularizes the solution path and bypasses the matrix inversion entirely. Consequently, it delivered stable and accurate axle weights even with <100 data frames (Table 1) and achieved immediate, stable convergence for both axle and GVW in all stages (Fig. 3i, j).Table 1Time-history results for two-vehicle following scenarioMethodAxle IDActual weight (t)Stage (number of data frames)65(2) 90(3) 566(4) 65(5) 90(6) 1105(7) 65(8) 90(9) 564(10) 65(11) 90MosesTruck2, Axle12.728.145.166.53−26.9717.033.41Truck2, Axle23.815.143.8431.91-6.463.173.04Truck2, Axle310.074.9516.543.9810.0713.357.71Truck3, Axle12.4515.13-7.503.805.496.96−1.19Truck3, Axle23.711.842.056.05-0.615.341.05Truck3, Axle38.979.494.889.6310.313.013.77BPMosesTruck2, Axle12.722.012.032.462.212.332.33Truck2, Axle23.814.014.384.184.284.273.99Truck2, Axle310.0710.2610.1510.2310.209.979.99Truck3, Axle12.452.012.031.921.821.851.80Truck3, Axle23.714.023.743.793.833.764.00Truck3, Axle38.979.569.739.799.7110.0110.01 Table 1Time-history results for two-vehicle following scenarioMethodAxle IDActual weight (t)Stage (number of data frames)65(2) 90(3) 566(4) 65(5) 90(6) 1105(7) 65(8) 90(9) 564(10) 65(11) 90MosesTruck2, Axle12.728.145.166.53−26.9717.033.41Truck2, Axle23.815.143.8431.91-6.463.173.04Truck2, Axle310.074.9516.543.9810.0713.357.71Truck3, Axle12.4515.13-7.503.805.496.96−1.19Truck3, Axle23.711.842.056.05-0.615.341.05Truck3, Axle38.979.494.889.6310.313.013.77BPMosesTruck2, Axle12.722.012.032.462.212.332.33Truck2, Axle23.814.014.384.184.284.273.99Truck2, Axle310.0710.2610.1510.2310.209.979.99Truck3, Axle12.452.012.031.921.821.851.80Truck3, Axle23.714.023.743.793.833.764.00Truck3, Axle38.979.569.739.799.7110.0110.01 Time-history results for two-vehicle following scenario In the two-vehicle side-by-side configuration scenario, two trucks (Vehicle 3 and Vehicle 4) traveled in adjacent lanes at 50 km/h. During the primary analysis period (Stage 3, 1824 frames), the Moses algorithm identified GVW with reasonable errors of −1.49% and 1.1% (Table 2). However, its inefficiency was again apparent in the convergence analysis (Fig. 3l, m), which showed it required approximately 1200 frames to bring GVW error within 5% and axle weight errors within a very broad 50% range. BPMoses again excelled. It achieved comparable GVW accuracy (2.01% and −0.83%) but with vastly superior axle weight estimation, with errors largely contained between −20% and 40% regardless of sample size (Table 2). Its GVW error converged smoothly and rapidly to within ±5% (Fig. 3n, o), confirming its ability to effectively resolve spatially overlapping loads, rendering it fully capable for real-time traffic applications.Table 2Time-history results for two-vehicle side-by-side scenarioMethodAxle IDActual weight (t)Stage (number of data frames)(1) 65(2) 90(3) 1824(4) 65(5) 90MosesTruck3, Axle12.45-1.83-1.142.21Truck3, Axle23.712.423.753.27Truck3, Axle38.978.5710.779.37Truck4, Axle12.697.706.850.89Truck4, Axle24.776.348.403.41Truck4, Axle312.6511.6712.0813.81BPMosesTruck3, Axle12.452.002.032.50Truck3, Axle23.714.024.344.01Truck3, Axle38.978.6110.0310.01Truck4, Axle12.692.002.032.67Truck4, Axle24.774.024.644.01 Table 2Time-history results for two-vehicle side-by-side scenarioMethodAxle IDActual weight (t)Stage (number of data frames)(1) 65(2) 90(3) 1824(4) 65(5) 90MosesTruck3, Axle12.45-1.83-1.142.21Truck3, Axle23.712.423.753.27Truck3, Axle38.978.5710.779.37Truck4, Axle12.697.706.850.89Truck4, Axle24.776.348.403.41Truck4, Axle312.6511.6712.0813.81BPMosesTruck3, Axle12.452.002.032.50Truck3, Axle23.714.024.344.01Truck3, Axle38.978.6110.0310.01Truck4, Axle12.692.002.032.67Truck4, Axle24.774.024.644.01 Time-history results for two-vehicle side-by-side scenario To assess performance at the limit, we designed a highly complex but realistic scenario involving three trucks in a mixed side-by-side and following configuration. The conventional Moses algorithm was completely overwhelmed. In data-sparse stages, the errors were divergent; for instance, in Stage 5, the calculated weight for one axle exceeded −13,000 t, demonstrating a total failure of the method (Table 3). Even in the ideal, data-rich Stage 6, while GVW errors were contained within 10%. However, in the subsequent short, data sparse stages (e.g. Stage 8 and Stage 9), individual axle-weight errors again diverged to several hundred percent (Fig. 3q, r). Remarkably, the BPMoses algorithm remained robust and effective, providing stable and physically plausible results across all 11 stages (Table 3). This demonstrates an inherent resilience to both data sparsity and signal complexity. In Stage 6, it successfully identified the GVW for all three vehicles, with errors converging to within a 10% margin—a feat the Moses algorithm could not reliably achieve (Fig. 3s, t).Table 3Time-history results for three-vehicle mixed scenarioMethodAxle IDActual weight (t)Stage (number of data frames)65(2) 90(3) 566(4) 65(5) 90(6) 1105(7) 65(8) 90(9) 564(10) 65(11) 90MosesTruck2, Axle12.72−6.69−7.38−3.141116−13,7443.21Truck2, Axle23.810.3513.30−148322,3683.204.14Truck2, Axle310.078.035724−26,1129.07−21.179.01Truck3, Axle12.4512.3513.1628.57−112859,264−4.60Truck3, Axle23.718.33−36.654224−147,71214.723.47Truck3, Axle38.9720.81−9424116,0326.47144.4315.56Truck4, Axle12.69−1102−5916.18−465.61182.68−8.96Truck4, Axle24.77850−0.15304.44−399.7720.812.80Truck4, Axle312.6514.14113.11226.667.8316.2917.44BPMosesTruck2, Axle12.722.002.012.072.022.091.89Truck2, Axle23.814.014.054.024.083.924.04Truck2, Axle310.0710.0210.0210.0610.0110.1010.05Truck3, Axle12.452.002.012.082.022.071.99Truck3, Axle23.714.014.064.024.064.074.03Truck3, Axle38.9710.0310.0110.0510.2010.1110.05Truck4, Axle12.692.002.013.242.642.682.95Truck4, Axle24.774.015.294.704.765.154.01Truck4, Axle312.6511.2910.7810.8611.4310.0310.01 Table 3Time-history results for three-vehicle mixed scenarioMethodAxle IDActual weight (t)Stage (number of data frames)65(2) 90(3) 566(4) 65(5) 90(6) 1105(7) 65(8) 90(9) 564(10) 65(11) 90MosesTruck2, Axle12.72−6.69−7.38−3.141116−13,7443.21Truck2, Axle23.810.3513.30−148322,3683.204.14Truck2, Axle310.078.035724−26,1129.07−21.179.01Truck3, Axle12.4512.3513.1628.57−112859,264−4.60Truck3, Axle23.718.33−36.654224−147,71214.723.47Truck3, Axle38.9720.81−9424116,0326.47144.4315.56Truck4, Axle12.69−1102−5916.18−465.61182.68−8.96Truck4, Axle24.77850−0.15304.44−399.7720.812.80Truck4, Axle312.6514.14113.11226.667.8316.2917.44BPMosesTruck2, Axle12.722.002.012.072.022.091.89Truck2, Axle23.814.014.054.024.083.924.04Truck2, Axle310.0710.0210.0210.0610.0110.1010.05Truck3, Axle12.452.002.012.082.022.071.99Truck3, Axle23.714.014.064.024.064.074.03Truck3, Axle38.9710.0310.0110.0510.2010.1110.05Truck4, Axle12.692.002.013.242.642.682.95Truck4, Axle24.774.015.294.704.765.154.01Truck4, Axle312.6511.2910.7810.8611.4310.0310.01 Time-history results for three-vehicle mixed scenario Our experiments confirm that the performance gap between the algorithms represents a paradigm shift. The conventional Moses method, reliant on a single, high-risk matrix inversion, is fundamentally unsuited for the ill-posed problems inherent in real-world vehicle weight recognition. The BPMoses algorithm, by recasting the problem as a constrained, iterative optimization, is not merely an incremental improvement but a structurally robust method that remains effective even when confronted with the severe data sparsity and signal complexity that cause traditional methods to fail. This marks it as a reliable, dynamic load estimation method in complex, real-world bridge structure monitoring. Application to real-time load monitoring of a real-world bridge Having validated the individual components, we deployed the integrated framework for continuous monitoring to assess its end-to-end performance and derive real-world load characteristics. First, to confirm the stability and precision of the full, integrated system, we compared the final weight outputs derived from two independent sensor types (A and B) within the deployed system. As shown in Fig. 4a, the Calculation Error, representing the absolute difference between the two concurrent, end-to-end measurements, remained consistently low (maximum of 0.95 t). This high degree of agreement does not re-validate accuracy (which was established in the previous section) but rather demonstrates the practical stability and robustness of the integrated framework as a whole. It confirms that all system components—from initial vehicle identification to final weight estimation—work together seamlessly and reliably in a demanding, real-world engineering environment, making it suitable for long-term structural health assessment.Fig. 4Real-world application and validation.a Consistency and stability assessment of the inversely estimated single-vehicle weight algorithm. Results for 11 representative bus weights using two independent sensor types (A and B) show that absolute calculation errors are consistently below 1.0 t. b Time-dependent dynamic lane loads over a 24-h period. The analysis shows that observed operational loads (colored lines) remain below the statutory design limit of 10.5 kN/m39. c Empirically determined strain influence lines for each of the eight lanes. These plots represent the unique structural response functions at each sensor location, which form the foundational basis of the load calculation model. d System-level validation comparing the average hourly measured strain (red circles) with the theoretical strain (blue lines) predicted by the model. The close correlation across all lanes confirms the high fidelity of the integrated monitoring and analysis framework. Real-world application and validation. a Consistency and stability assessment of the inversely estimated single-vehicle weight algorithm. Results for 11 representative bus weights using two independent sensor types (A and B) show that absolute calculation errors are consistently below 1.0 t. b Time-dependent dynamic lane loads over a 24-h period. The analysis shows that observed operational loads (colored lines) remain below the statutory design limit of 10.5 kN/m39. c Empirically determined strain influence lines for each of the eight lanes. These plots represent the unique structural response functions at each sensor location, which form the foundational basis of the load calculation model. d System-level validation comparing the average hourly measured strain (red circles) with the theoretical strain (blue lines) predicted by the model. The close correlation across all lanes confirms the high fidelity of the integrated monitoring and analysis framework. With the monitoring framework validated, we analyzed the vehicle-induced dynamic loads on a real-world bridge for a representative 24-h period (Fig. 4b). A key finding is that the operational loads are well within the bridge’s design capacity. The maximum dynamic load effect observed in any lane was ~6.0 kN/m, which is only 57% of the 10.5 kN/m uniformly distributed load specified by the national design code. The temporal distribution of the load reveals a clear diurnal cycle correlated with urban traffic patterns. The bridge enters a high-load state from ~06:00 to 20:00. Within this period, distinct peaks are evident, corresponding to the morning (c. 07:00), midday (c. 12:00), and evening (c. 17:00) traffic rushes, with a smaller peak during late-evening activity (c. 21:00). Spatially, the load distribution across the eight lanes is non-uniform, reflecting distinct lane utilization patterns by heavy vehicles. The analysis of Fig. 4b indicates that the inner traffic lanes (e.g., Lanes 3 and 6) and the outer slow lane in one direction (Lane 7) consistently sustain high traffic loads. Notably, Lane 8, also a designated slow lane, experiences load peaks that are among the highest on the bridge. In stark contrast, Lane 1, the slow lane for the opposing traffic direction, exhibits the lowest load levels. This asymmetric loading of the slow lanes suggests that heavy vehicles, primarily buses in this urban setting, may preferentially use one side of the bridge or follow specific routing patterns over the structure. This detailed spatiotemporal load profile, made possible by the validated monitoring system, is critical for refined fatigue assessments and optimized bridge maintenance strategies. To validate the methodology across the entire traffic spectrum, we compared the theoretical and measured structural responses over a continuous 24-h period. The theoretical average hourly strain for each lane was calculated by applying the full set of inverted vehicle weights and their measured trajectories to the empirically derived influence lines for each lane (Fig. 4c). These influence lines represent the unique structural response characteristics at each measurement point and form the foundation for the subsequent vehicle load inversion using the strain area method. As shown in Fig. 4d, the calculated theoretical strains exhibit a remarkable correspondence with the average strains measured directly by the sensors across all eight lanes. The diurnal patterns are replicated with high fidelity, and the quantitative error remains within about 10%. This strong agreement on an aggregate scale provides robust verification for the entire inverted vehicle load dataset, confirming its reliability for analyzing the bridge’s operational load characteristics. Conclusions To address the current limitations of bridge dynamic weighing systems in achieving rapid synchronization and real-time dynamic tracking of vehicle characteristics, spatiotemporal information, and vehicle weight, as well as the high-investment challenges posed by pavement weigh-in-motion systems in urban environments, this study utilizes surveillance footage to extract both static and dynamic vehicle information. Based on the dynamic strain of the main girder, vehicle load estimation, and inversion are performed, thereby realizing dynamic bridge traffic load estimation based on video-strain data. This approach has been applied to an actual steel truss bridge. The main research conclusions are as follows:The lightweight YOLOv7-MobileNetV3-SimAM model is optimized for efficient, on-device deployment in resource-constrained engineering environments. Compared to the standard YOLOv7, it reduces model parameters by 82.3% and accelerates CPU-based inference time by 41.9%, with only a marginal 1.2% drop in mean average precision (mAP@0.5).The video-based tracking system’s dynamic tracking module achieves exceptional spatiotemporal accuracy. It is founded on a precise geometric calibration with a maximum spatial error of just 0.222 m. In real-world testing, the system demonstrated 99.25% vehicle counting accuracy and 100% lane identification. Speed estimation is highly precise, with 82.03% of measurements registering an error of ≤1% when benchmarked against a ground-truth radar system.The proposed BPMoses algorithm for vehicle weight inversion marks a paradigm shift in robustness over conventional methods. For single-vehicle crossings, it consistently maintains GVW error within 2% and individual axle weight errors within 15%. More importantly, in complex multi-vehicle scenarios where traditional algorithms fail catastrophically, BPMoses remains stable, controlling GVW and axle weight errors within 10% and 40%, respectively.The integrated framework was validated in a continuous 24-h deployment, producing a high-fidelity spatiotemporal load profile of the bridge. The maximum dynamic loads were only 57% of the statutory design limit. The strong correlation between the model’s predicted strain and measured field data confirms the system’s overall reliability. The lightweight YOLOv7-MobileNetV3-SimAM model is optimized for efficient, on-device deployment in resource-constrained engineering environments. Compared to the standard YOLOv7, it reduces model parameters by 82.3% and accelerates CPU-based inference time by 41.9%, with only a marginal 1.2% drop in mean average precision (mAP@0.5). The lightweight YOLOv7-MobileNetV3-SimAM model is optimized for efficient, on-device deployment in resource-constrained engineering environments. Compared to the standard YOLOv7, it reduces model parameters by 82.3% and accelerates CPU-based inference time by 41.9%, with only a marginal 1.2% drop in mean average precision (mAP@0.5). The video-based tracking system’s dynamic tracking module achieves exceptional spatiotemporal accuracy. It is founded on a precise geometric calibration with a maximum spatial error of just 0.222 m. In real-world testing, the system demonstrated 99.25% vehicle counting accuracy and 100% lane identification. Speed estimation is highly precise, with 82.03% of measurements registering an error of ≤1% when benchmarked against a ground-truth radar system. The video-based tracking system’s dynamic tracking module achieves exceptional spatiotemporal accuracy. It is founded on a precise geometric calibration with a maximum spatial error of just 0.222 m. In real-world testing, the system demonstrated 99.25% vehicle counting accuracy and 100% lane identification. Speed estimation is highly precise, with 82.03% of measurements registering an error of ≤1% when benchmarked against a ground-truth radar system. The proposed BPMoses algorithm for vehicle weight inversion marks a paradigm shift in robustness over conventional methods. For single-vehicle crossings, it consistently maintains GVW error within 2% and individual axle weight errors within 15%. More importantly, in complex multi-vehicle scenarios where traditional algorithms fail catastrophically, BPMoses remains stable, controlling GVW and axle weight errors within 10% and 40%, respectively. The proposed BPMoses algorithm for vehicle weight inversion marks a paradigm shift in robustness over conventional methods. For single-vehicle crossings, it consistently maintains GVW error within 2% and individual axle weight errors within 15%. More importantly, in complex multi-vehicle scenarios where traditional algorithms fail catastrophically, BPMoses remains stable, controlling GVW and axle weight errors within 10% and 40%, respectively. The integrated framework was validated in a continuous 24-h deployment, producing a high-fidelity spatiotemporal load profile of the bridge. The maximum dynamic loads were only 57% of the statutory design limit. The strong correlation between the model’s predicted strain and measured field data confirms the system’s overall reliability. The integrated framework was validated in a continuous 24-h deployment, producing a high-fidelity spatiotemporal load profile of the bridge. The maximum dynamic loads were only 57% of the statutory design limit. The strong correlation between the model’s predicted strain and measured field data confirms the system’s overall reliability. Method Our methodology establishes an end-to-end, non-intrusive framework for the real-time estimation of vehicular loads on bridges. The framework integrates three sequential modules: (1) a lightweight vision-based network for vehicle classification and localization; (2) a dynamic tracking algorithm to reconstruct spatiotemporal vehicle trajectories; and (3) an iterative algorithm, termed BPMoses, for accurate axle weight estimation. The entire system is validated through a field experiment on a long-span steel truss bridge. Lightweight model for vehicle information recognition To enable practical deployment on edge-computing devices with limited power and computational resources, we developed a lightweight yet accurate object detection model. The widely-used YOLOv7 model31, while powerful, presents a computational burden (37.2 million parameters), compared to the earlier YOLOv5 version32. Our objective was to drastically reduce this overhead while preserving detection accuracy. To achieve this, we designed a custom model, YOLOv7-MobileNetV3-SimAM. The core modifications are twofold. First, we replaced the original feature-extraction backbone of YOLOv7 with the highly efficient MobileNetV3 network. MobileNetV3 is specifically designed for mobile CPUs and utilizes an inventive Bneck structure with depth-wise separable convolutions, which substantially reduces parameter count and computational cost33. Second, to compensate for any potential accuracy loss from the lightweight backbone, we incorporated the SimAM into the network’s head. Inspired by neuroscience principles of spatial suppression, SimAM is a parameter-free attention mechanism that computes 3D attention weights34. It enhances feature representation by assigning higher importance to neurons exhibiting strong differentiation from their neighbors, without adding computational overhead34. The SimAM module was strategically inserted between the W-ELAN and REP modules in the head of the network (see proposed architecture in Fig. 5a).Fig. 5The proposed computer vision and dynamic strain fusion approach for urban bridge weigh-in-motion.a YOLOv7-MobileNetV3-SimAM model for static vehicle flow feature information recognition. b Vehicle trajectory tracking algorithm process framework for traffic spatiotemporal information dynamic recognition. c Transformation between bridge coordinate system and pixel coordinate system. d Flowchart of proposed BPMoses algorithm for continuous dynamic inverse estimation of bridge vehicle load. The proposed computer vision and dynamic strain fusion approach for urban bridge weigh-in-motion. a YOLOv7-MobileNetV3-SimAM model for static vehicle flow feature information recognition. b Vehicle trajectory tracking algorithm process framework for traffic spatiotemporal information dynamic recognition. c Transformation between bridge coordinate system and pixel coordinate system. d Flowchart of proposed BPMoses algorithm for continuous dynamic inverse estimation of bridge vehicle load. Each input image, after preprocessing, is resized to a 640 × 640 RGB image in YOLOv7, then is processed by the Backbone network for feature extraction. The three outputs from the backbone network are used to generate three different sizes of feature maps in the head layer. These feature maps are fed into the REP blocks and CBM convolution layers to predict three outputs for object detection: object categorization, foreground/background differentiation, and bounding box localization. In the final step, the YOLOv7 network combines these prediction results and outputs the final object detection results. Dynamic traffic spatiotemporal information recognition method Following detection, we employ a tracking-by-detection paradigm to associate vehicles across frames and generate continuous trajectories. This process follows a predict-associate-update cycle. Having identified vehicles in individual frames, we reconstruct their continuous spatiotemporal trajectories using the DeepSORT algorithm35. It is chosen for its robustness in handling occlusions and identity switches, which are common in dense traffic scenes. It integrates motion information, predicted by a Kalman filter, with appearance features extracted by a deep neural network. The state of each vehicle is represented by an eight-dimensional vector \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\bf{X}}}=(u,{v},\,\gamma ,\,h,\,\dot{u},\overline{{v}},\,\dot{\gamma },\,\dot{h})$$\end{document}X=(u,v,γ,h,u°,v¯,γ°,h°), where u and v denote the x and y coordinates of the bounding box center, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma$$\end{document}γ and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$h$$\end{document}h correspond to the aspect ratio and height of the detection box, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\dot{u},\overline{{v}},\,\dot{\gamma },\,\dot{h})$$\end{document}(u°,v¯,γ°,h°) represent the respective changes in position and size in the image coordinate system. We set a linear constant velocity model. The core of the tracking is associating the set of existing tracks T = {T1, T2,…, Tn} with the current set of detections D. This is framed as an assignment problem solved by the Hungarian algorithm. The algorithm minimizes a cost matrix C, where each element Cij represents the dissimilarity between the i-th track and the j-th detection. To ensure robustness against occlusions, the cost is a weighted combination of motion and appearance metrics, as in the DeepSORT framework. The motion cost is the squared Mahalanobis distance:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{{ij}}^{(1)}={\left({x}_{i}-\,{y}_{j}\right)}^{T}{{{\bf{S}}}}_{j}^{-1}({x}_{i}-\,{y}_{j})$$\end{document}dij(1)=xi−yjTSj−1(xi−yj)where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{{ij}}^{(1)}$$\end{document}dij(1) represents the Mahalanobis distance between detection box i and predicted box j, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{i}$$\end{document}xi and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${y}_{j}$$\end{document}yj denote the center points of the individual boxes, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{{\bf{S}}}}_{j}$$\end{document}Sj is the state covariance matrix of the Kalman prediction for box j, reflecting the uncertainty in trajectory prediction. Using a 95% confidence interval of the chi-squared distribution as a threshold \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t=9.4877$$\end{document}t=9.4877, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{{ij}}^{(1)} < {{{\rm{t}}}}^{(1)}$$\end{document}dij(1)