A Full-Chain Dataset for Intelligent New Energy Vehicles based on High-fidelity Digital Twin Platform Authors: Zhengxian Chen, Yuyang Pan, Tianyang Gong, Chenggang Zhang, Yunwei Li, Jieyu Wang, Huanan Wang, Wenfeng Guo, Mingqiang Wang, Chaosheng Huang, Jun Li Published: 2026 ABSTRACT AbstractWith the rapid development of intelligent new energy vehicles, there is an increasing demand for datasets that can jointly characterize traffic environments, vehicle control behaviors, and powertrain energy responses. Existing public datasets often focus on perception tasks or isolated research directions, making them insufficient for system-level closed-loop studies. To address this gap, we present 3ITVP (Intelligent Transportation, Intelligent Vehicle, Intelligent Power), a full-chain dataset built on a high-fidelity digital twin platform. The dataset approaches from the perspectives of five types of vehicles: ordinary cars, including both electric and fuel vehicles; light trucks; semi-trailers; logistics vehicles and buses, exploring their driving conditions and behaviors. Rigorously time-synchronized multimodal data is provided, and systematically covers key aspects of intelligent transportation, autonomous driving and new energy technologies. In addition to long-horizon continuous driving behaviors, 3ITVP includes a wide range of special event cases, supporting research on perception, decision-making, planning and control, energy management, and Sim-to-Real transfer. Background & Summary In recent years, the rapid development of intelligent connected vehicle technologies and new energy vehicles has been profoundly reshaping the design philosophy of automotive control systems, driving intelligent new energy vehicle control systems to gradually evolve from traditional rule-driven approaches toward advanced data-driven paradigms1–3. Early autonomous driving control systems mainly relied on manually designed rules. However, in open and complex road environments, they quickly encountered combinatorial explosion due to highly variable scenarios, complex interactions, and long-tail cases that are difficult to enumerate exhaustively, thereby exhibiting evident limitations4–9. Therefore, in order to achieve higher levels of automated driving capability (Level 3 and above), the research focus in recent years has irreversibly shifted toward learning-based paradigms, such as deep learning, reinforcement learning, end-to-end control, and large language models. The core objective of these approaches is to enable integrated vehicle control systems to continuously improve their decision-making, planning, and control performance in unknown environments through large-scale data accumulation and the optimization of complex models. Consequently, high-quality data has become a central supporting element. Whether in the decision-making processes of autonomous driving control systems or in the development of energy management strategies for new energy power systems, system performance is highly dependent on comprehensive and trustworthy data foundations. At present, the field of autonomous driving has established a relatively systematic data foundation to support research on perception, prediction, driving behavior understanding, and partial decision modeling. Representative autonomous driving datasets, such as KITTI10, nuScenes11, the Waymo Open Dataset12, and Argoverse13, characterize real-world road environments through synchronized multi-sensor data collection and have played an important role in tasks including object detection, multi-object tracking, and trajectory prediction2,14. Based on these datasets, research efforts have continuously advanced multimodal perception fusion and scene understanding capabilities, while gradually exploring extensions from perception toward driving behavior understanding and decision modeling. Some studies leverage large-scale video or multimodal inputs to construct end-to-end driving models, in which perception, decision-making, and control processes are jointly modeled within a unified framework15. Meanwhile, datasets such as ROAD16, Drive&Act17, AIDE18, and RoadText-1K19 provide finer-grained characterizations of the driving process from the perspectives of events, behaviors, and semantics. In addition, works such as Vehicle-Rear20, unified longitudinal trajectory datasets21, and V2V4Real22 complement traffic participant relationship modeling from the perspective of vehicle interactions and multi-vehicle cooperative behaviors. With the development of intelligent connected vehicle technologies, datasets including Mixed Signals23, V2X-Seq24, and V2X-Sim25 further introduce vehicle–infrastructure cooperation and Vehicle-to-Everything(V2X) scenarios, characterizing information sharing and collaborative perception processes under multi-agent conditions. Overall, these datasets still primarily focus on collaboration at the perception and information levels, with research emphasis placed on achieving more comprehensive perception and more effective information sharing, while relatively little attention is given to low-level control signals and vehicle energy system modeling. From a system-level perspective, their data organization and task design remain largely centered on perception and prediction tasks, exhibiting a strong task-oriented characteristic. Most datasets take object detection, semantic segmentation, or trajectory prediction as core application scenarios, implicitly embedding assumptions about downstream tasks during the data collection and annotation stages. While this perception-centric data paradigm improves the performance of individual modules, it also limits, to some extent, systematic modeling of the complete closed-loop driving decision and control process. The absence of certain information, such as powertrain-related data and detailed vehicle operating states, further constrains the overall generalization capability of these datasets. Meanwhile, in the research direction of intelligent new energy vehicles, data-driven methods have also been widely adopted. Related datasets typically focus on the vehicle’s internal operating states, battery parameters, power output characteristics, and energy consumption processes, and are extensively used in studies on energy consumption prediction, energy management, and driving behavior analysis26–28. Building upon these datasets, some studies further explicitly incorporate energy consumption factors into route planning and speed planning problems, exploring the design of autonomous driving strategies under energy efficiency constraints29–31. However, such new energy vehicle datasets are predominantly ego-vehicle centered, emphasizing internal energy and power system characteristics while providing relatively limited representations of external information in complex traffic environments. Information such as vehicle–infrastructure cooperative perception, V2X interaction signals, and overall traffic system behavior characteristics is often not jointly represented with vehicle energy, power, and control signals within a unified data framework. As a result, cross-domain research on energy-aware decision-making, cooperative control, and predictive energy management in complex traffic environments still lacks direct data support. Although the autonomous driving field has accumulated various datasets oriented toward cooperative perception, effective coupling between these datasets and new energy vehicle energy systems has not yet been established, leaving the information layer and the energy layer relatively fragmented. Furthermore, from the perspective of scenario structure, existing public datasets still exhibit a certain degree of separation between “comprehensive driving” scenarios and “special driving event” scenarios. On the one hand, Comprehensive Driving Behavior datasets are large in scale and continuously distributed, but provide limited coverage of rare or extreme events. On the other hand, datasets focusing on specific or extreme situations can capture such events but often lack sufficient long-term temporal context, making it difficult to reflect the complete evolution process from steady operation to extreme operating conditions32.This scenario-level fragmentation, to some extent, constrains the overall learning capability of algorithms with respect to complex driving behaviors and system stability, and also affects model generalization and robustness in real-world traffic environments33. To address the structural issues widely observed in existing public autonomous driving and new energy vehicle datasets—namely, the lack of effective coordination and coupling among intelligent connected information, ego-driver characteristics, and vehicle powertrain information, as well as the separation between comprehensive driving and special events, this study constructs “3ITVP” (Intelligent Transportation, Intelligent Vehicle, Intelligent Power), an intelligent new energy vehicle full-chain dataset based on the 51sim high-fidelity digital twin platform. Built upon a unified simulation system, the dataset systematically records the complete operational process of 5 types of vehicles (ordinary passenger cars, buses, light trucks, semi-trailers, logistics vehicles) in complex traffic environments, covering key stages including environment perception, intent recognition, behavior decision-making, motion execution, and energy response. It enables continuous and synchronized observation of driving decision formation mechanisms and their corresponding energy response processes. Through a strict time-alignment mechanism, 3ITVP jointly collects environmental semantic information, ego-vehicle control inputs, vehicle dynamic states, and energy system responses within a unified framework. This includes dynamic changes in obstacles and lane structures; fine-grained operation inputs such as steering wheel and pedal actions; low-level feedback such as wheel speed and yaw rate; as well as energy-related parameters, particularly the released Demand_Power derived from vehicle longitudinal dynamics. This variable represents wheel-end mechanical power; during deceleration, its negative values indicate the theoretical upper bound of mechanically available braking power, while electrical-side regenerative braking modeling is left to users according to their own powertrain assumptions. In addition, system-level environmental conditions—such as pedestrians, weather, lighting, wind fields, and road friction coefficients—are further integrated. Based on this multimodal and highly synchronized data organization, 3ITVP is able to reconstruct, within a unified data space, the complete evolution process from driving intent to vehicle motion and energy variation. This allows researchers to systematically analyze the behavioral characteristics of decision models under energy constraints, as well as the mechanisms by which control strategies influence power distribution and energy flow. Compared with data formats that describe only environmental information or vehicle-side information in isolation, this dataset provides a more comprehensive observational perspective for understanding the coordination between high-level planning strategies and low-level execution mechanisms, thereby offering data support for interpretable and energy-efficient intelligent driving model research. Under real-world road conditions, some complex or rare driving events are difficult to collect systematically through large-scale on-road testing due to their low occurrence probability, high data acquisition cost, or strict safety constraints. Leveraging the 51sim high-fidelity digital twin platform, the 3ITVP dataset can construct, under safe and controllable conditions, multiple categories of special driving events that are difficult to obtain systematically in real environments. These include sudden cut-in maneuvers by traffic participants, emergency evasive actions, complex meteorological conditions, and low-adhesion road surfaces. Such events can be accurately reproduced in the simulation environment and support controlled adjustment of individual or multiple environmental factors, for example, modifying wind field conditions while maintaining the same traffic flow structure, or adjusting road friction characteristics under fixed road geometry. The vehicle dynamics models and energy constraint models integrated within the simulation platform ensure physical consistency of the data at both the kinematic and energy levels, enabling researchers to conduct parameter sensitivity analysis, strategy comparison, and system robustness evaluation under reproducible experimental conditions. Moreover, this simulation framework is inherently suitable for the Sim-to-Real research paradigm: researchers can analyze algorithm performance under idealized conditions and subsequently introduce varying levels of noise or disturbances to systematically evaluate model stability and generalization capability in complex environments, providing reliable data support and validation foundations for subsequent real-world deployment. Overall, the core objective of this study is to construct 3ITVP, a comprehensive and multi-level dataset that deeply integrates environment perception, driving behavior, vehicle control, energy state, and powertrain systems, in order to fill long-standing data gaps in autonomous driving and new energy vehicle research and to address the lack of coupling between new energy systems and autonomous driving in intelligent vehicle studies. We expect this dataset to support the development of multiple frontier research directions and to provide a systematic foundation for understanding and optimizing the full-chain mechanisms of intelligent driving systems: 3ITVP provides unprecedented research conditions for cross-layer decision modeling. Through high-precision synchronized recording of vehicle operation inputs, kinematic states, and environmental semantic information, researchers can deeply analyze how high-level behavioral decisions (e.g., lane changes and overtaking) influence low-level control signals (e.g., fine steering adjustments and pedal depth variations) across temporal scales, and further investigate the causal relationships and strategy coupling structures between them. 3ITVP provides unprecedented research conditions for cross-layer decision modeling. Through high-precision synchronized recording of vehicle operation inputs, kinematic states, and environmental semantic information, researchers can deeply analyze how high-level behavioral decisions (e.g., lane changes and overtaking) influence low-level control signals (e.g., fine steering adjustments and pedal depth variations) across temporal scales, and further investigate the causal relationships and strategy coupling structures between them. The autonomous driving information contained in the dataset supports the exploration of related research directions, such as trajectory prediction. By integrating real traffic dynamics, vehicle energy efficiency models, and driving intent into a unified data source, researchers can not only conduct traditional studies in autonomous driving control and decision-making, but also explore the design of integrated control strategies that jointly consider safety, traffic efficiency, and energy efficiency. The autonomous driving information contained in the dataset supports the exploration of related research directions, such as trajectory prediction. By integrating real traffic dynamics, vehicle energy efficiency models, and driving intent into a unified data source, researchers can not only conduct traditional studies in autonomous driving control and decision-making, but also explore the design of integrated control strategies that jointly consider safety, traffic efficiency, and energy efficiency. Benefiting from comprehensive coverage of energy flows, dynamic characteristics, and control signals, 3ITVP naturally serves as a validation platform for intelligent energy management and energy-saving algorithms in new energy vehicles. Whether for predictive control, eco-driving assistance, energy recovery optimization, or adaptive energy management strategies, the dataset provides standardized, reproducible benchmarks and rich training data support. Benefiting from comprehensive coverage of energy flows, dynamic characteristics, and control signals, 3ITVP naturally serves as a validation platform for intelligent energy management and energy-saving algorithms in new energy vehicles. Whether for predictive control, eco-driving assistance, energy recovery optimization, or adaptive energy management strategies, the dataset provides standardized, reproducible benchmarks and rich training data support. 3ITVP systematically supports algorithm validation. Researchers can construct scenarios that are difficult to obtain on real roads through the dataset, enabling safe testing, causal analysis, and evaluation of cross-domain transfer capability (Sim-to-Real). 3ITVP systematically supports algorithm validation. Researchers can construct scenarios that are difficult to obtain on real roads through the dataset, enabling safe testing, causal analysis, and evaluation of cross-domain transfer capability (Sim-to-Real). Methods Simulation platform and ego vehicle specifications Experimental platform Distinct from traditional datasets collected using physical sensors, this study constructed the experimental environment based on the 51World high-fidelity digital twin platform. The platform integrated a high-precision physics engine capable of simulating complex vehicle dynamic characteristics and diverse human–vehicle–road interaction environments. A key advantage of the platform in this study was that it provided an integration of traditional simulators such as CARLA and SUMO, and dynamic simulators such as CarSim, allowing its simulations to inherit the benefits of all these platforms. Meanwhile, perceptions were set to be ideal, which enabled a complete decoupling between perception noise and the evaluation of decision-making and planning algorithms, and allowed the simulation of data that were difficult to collect in real-world conditions. As illustrated in Fig. 1, the 51sim simulator interface included a mini-map, a sensor device panel, ego-vehicle input information such as steering wheel angle and accelerator pedal position, as well as basic vehicle states including vehicle speed and engine rotational speed. The Simulator Screenshot - Workbench Interface. The simulator provided multiple viewpoints that could be used to enhance interpretability. As shown in Fig. 2, in addition to the conventional rear-view perspective of the vehicle, users could also select a bird’s-eye view and a driver’s-eye view. This allowed validations to be carried out using different viewpoints to understand the driver’s intent, enhancing the accuracy of validation. No third-party observational dataset or data-use agreement was involved in generating the released records. All released data were produced from the configured simulation scenarios and the post-processing scripts described in this section. Different View. (a) Bird’s-eye view; (b) Driver’s-eye view. Ego vehicle configuration The ego vehicle was configured as a car with full dynamic response characteristics. To support coupled studies of energy management systems (EMS) and vehicle control, precise mass, inertia, and geometric parameters were incorporated into the physics engine. The research conducted the simulation based on five different types of vehicles’ points of view, including ordinary cars (fuel and electric), buses, light trucks, semi-trailers, and logistics vehicles. The key dynamics parameters of some common two-axle vehicles were listed in Table 1. These parameters defined the vehicle’s physical responses to control inputs (throttle, braking, and steering) as well as to external forces (aerodynamic drag and road grade resistance). Table 1Dynamics parameters of some common two-axle vehicles.ParameterLarge BusLight TruckPassenger CarWheelbase (cm)500336.5291.9Center of Gravity (CG) longitudinal offset from front axle (cm)350200134.3CG height (cm)150117.552.5CG lateral offset (cm)000Chassis height (cm)292325140Chassis width (cm)260254180Distance from front bumper to front axle (cm)24094100Moment of inertia Ix (kg · m²)169402153.1618.9Moment of inertia Iy (kg · m²)6776033376.32621.2Moment of inertia Iz (kg · m²)6776032810.92858.5Vehicle mass (kg)152604600.31696Distance from rear axle to rear bumper (cm)235109.588 Dynamics parameters of some common two-axle vehicles. Meanwhile, for ordinary cars, this study provided both fuel-powered cars and electric vehicles as two different power types. The powertrain specifications were listed in Tables 2, 3. Table 2Electric Vehicle Parameters.CategoryName of ParameterValue of ParameterCommentBasic StructureType of PowerElectricDrive FormFour Wheel DriveFront Left/Front Right/Back Left/Back RightCore PowerSingle Motor Peak Torque300 Nm0–5000 rpmMaximum Total System Torque749.7 NmStarting with full ThrottleSingle Motor Peak RPM288,000 r/minDrivetrain Natural Frequency9 HzDamping Ratio 0.9Motor’s Moment of Inertia0.01 kg*m2 Electric Vehicle Parameters. Table 3Fuel Vehicle Parameters.CategoryName of ParameterValue of ParameterCommentBasic StructureType of PowerFuelDrive FormFront Wheel DriveCore PowerEngine Peak RP7000RPMEngine Idle RP740RPMEngine’s Moment of Inertia0.2 kg*m2Drivetrain Natural Frequency9 HzDamping Ratio 0.9 Fuel Vehicle Parameters. To obtain global semantic information of the environment, the ego vehicle was equipped with an ideal sensor suite based on target-level sensing. Unlike physical sensors, which require complex offline calibration procedures, these virtual sensors directly extracted data from the simulation engine, thereby eliminating spatiotemporal synchronization errors. The primary perception sensor, perfectPerception1, operated at a frequency of 30 Hz, provided a 360° horizontal field of view, and had a detection range of 80 m. It outputted the precise positions, velocities, and heading angles of all surrounding traffic participants. In addition, a 64-beam target-level LiDAR simulator was configured to generate auxiliary point cloud features, ensuring that the autonomous driving system operated smoothly. It should be noted that although this dataset was framed around intelligent new energy vehicles, the current release did not explicitly include architecture-specific Hybrid Electric Vehicle (HEV) or Plug-in Hybrid Electric Vehicle (PHEV) configurations. This choice was intentional. HEV and PHEV modeling requires power-split logic, battery state-of-charge dynamics, motor and engine efficiency maps, regenerative braking limits, and supervisory energy-management strategies, which are highly architecture-dependent and may vary substantially across vehicle platforms. Instead of prescribing one specific HEV/PHEV architecture, the dataset provided two baseline powertrain configurations: a fuel vehicle as a conventional reference and a pure electric vehicle as a representative new-energy configuration. More importantly, the released Demand_Power variable was defined as the wheel-end mechanical power demand derived from vehicle longitudinal dynamics. This variable is independent of a specific powertrain topology and can therefore serve as a common input for user-defined EV, HEV, PHEV, or other customized energy-management models. In this sense, the dataset remains consistent with the stated intelligent new energy vehicle framing, because its main purpose is to provide synchronized traffic, driving-behavior, vehicle-control, and wheel-end power-demand data for energy-aware intelligent vehicle studies, rather than to prescribe a closed set of powertrain architectures. Scenario configurations Data collection adopted a dual-stream data acquisition scheme to balance long-term continuous driving and short-term microscopic scene data, ultimately forming two sub-data sets: Comprehensive Driving Behavior Dataset and Special Driving Events Dataset. Comprehensive driving behavior This subset included five types of vehicles, among which the ordinary cars provided comprehensive driving behavior data for two types of power systems. To support long-horizon dynamic analysis, approximately 6,800 seconds (about 110 minutes) of comprehensive driving behavior data of ordinary passenger cars were collected on a high-fidelity digital twin map of Frankfurt, Germany. This portion of the data was seamlessly segmented into 40 temporally continuous clips, fully recording car-following, cruising, and interaction behaviors of vehicles within urban traffic flow. Each segment contained at least one accelerating from stationary phase and one decelerating until stationary phase. The ego vehicle stopped at a crossing with a traffic light when one segment terminated, as these points usually served as natural breakpoints for vehicles. This ensured that each sequence could itself be considered a complete driving scenario. The time lengths of segments varied from 3 minutes to 10 minutes; this was to ensure they contained enough data to carry out training for light-weight models, while not being verbose when used. In addition to the ordinary passenger cars, light trucks and buses were also included in the proposed dataset, as Fig. 3 shown. Owing to their substantially larger collision volumes, turning radii, and center-of-gravity locations compared with ordinary passenger cars, a separate comprehensive driving behavior dataset exceeding 10 minutes was generated for each of these two vehicle types. Given their large turning radii, road segments composed of relatively gentle curves were selected. The dataset also included cases using two special types of vehicles, including semi-trailers and logistics vehicles, to allow users to carry out specific tasks. Special vehicle types. (a) Light truck in simulator; (b) Bus in simulator. Special driving events and environmental matrix In order to support interactive research on the safety and energy-saving aspects of autonomous driving, key risk scenarios were constructed in accordance with the principles of Safety of the Intended Functionality (SOTIF), including pedestrian dart-out events, unprotected intersection crossings, and aggressive cut-in maneuvers. To quantify algorithm robustness, these events were orthogonally combined with an environmental condition matrix. By systematically adjusting illumination conditions (bright sunlight and low-light scenarios), weather conditions (rain, snow, and fog), and physical parameters (wind speed ranging from 0 to 10 m/s and road friction coefficients from 0.45 to 0.8), a strictly controlled design-of-experiments framework was established. The combinations of fine-grained temporal information collected under these varying environmental conditions constituted the microscopic special-event subset of the dataset. The power data provided in the dataset was also designed for users who were willing to study the safety-energy trade-off or how to optimize the control of the power system through the SOTIF system. This study categorized the primary weather conditions into five types: foggy, rainy, snowy, dusty, and clear weather. Each weather condition was configured based on the corresponding meteorological characteristics and prior knowledge. The 51sim platform accurately reproduced the impacts of different weather scenarios on the surrounding environment. Figure 4 below shows the scenarios under different weather conditions. Primary weather conditions. (a) Foggy weather; (b) Rainy weather; (c) Dusty weather; (d) Snowy weather; (e) Sunny weather. (a) Rainy scenario: Rainfall leads to the formation of a water film on the road surface, significantly reducing the friction coefficient between tires and the road, which increases braking distance and raises the risk of lateral skidding during steering maneuvers. Falling raindrops in the air can cause image blurring, glare, or noise interference in optical sensors such as cameras, and may also affect the signal penetration and reflection quality of LiDAR and millimeter-wave radar, thereby reducing the accuracy of object detection and tracking. (b) Snowy scenario: Snow accumulation further decreases road adhesion while obscuring critical ground markings such as lane lines and curbs, increasing the difficulty of localization and path planning. Falling snowflakes introduce dynamic occlusion and scattering within sensor fields of view, producing substantial noise in both LiDAR point clouds and camera images. This scenario simulates the combined challenges of reduced visibility and degraded sensor performance. (c) Dust storm scenario: Compared with rain and snow, sand accumulation on the road surface has a relatively smaller impact on friction; however, the high concentration of suspended particles in the air causes severe degradation across multimodal sensors. Visible-light and infrared camera imaging is obscured, LiDAR generates ghost points due to multiple scattering by particles, and millimeter-wave radar signals may also experience attenuation. Overall visibility decreases sharply, forcing autonomous driving systems to rely on more robust perception fusion and prediction strategies. (d) Foggy scenario: Fog consists of suspended water droplets or ice crystals in the air, leading to intensified light scattering and a significant reduction in environmental visibility. Camera images exhibit reduced contrast, color distortion, and loss of edge information; LiDAR signals are attenuated by absorption and scattering from fog particles, resulting in lower point cloud density and increased noise. In addition, the uniformly distributed nature of fog may induce backscatter interference in some active illumination systems (e.g., auxiliary lighting), further increasing the difficulty of object detection and semantic segmentation. (e) Clear-weather scenario: As a baseline condition, clear weather provides ideal visibility and road adhesion. However, strong sunlight may cause overexposure, high-contrast regions, or lens flare in camera images, particularly under low-angle direct sunlight. High ambient temperatures may also increase sensor thermal noise or induce image distortion due to heat haze effects above the road surface. This scenario is primarily used to evaluate system performance under ideal conditions and to serve as a reference baseline for performance degradation under adverse weather scenarios. In addition, five types of special scenarios are provided: (a) Opposing-direction conflict: A vehicle from the opposite lane partially crosses the lane boundary and moves toward the ego vehicle. (b) Turning conflict: During the ego vehicle’s turning maneuver, another vehicle overtakes from the left side and cuts in to turn ahead of the ego vehicle. (c) Sudden appearance: While the ego vehicle is traveling straight, another vehicle merges from a different lane into the through lane and suddenly appears in front of the ego vehicle. (d) Lateral front cut-in: A vehicle located at the front-left side changes lanes into the lane occupied by the ego vehicle. (e) Straight-through at an intersection: Starting from a standstill, the ego vehicle proceeds straight through a relatively congested intersection and then gradually decelerates until coming to a complete stop. One can find a clear presentation of the five special events in Fig. 5. Special events. (a) Opposing-direction conflict; (b) Turning conflict; (c) Sudden appearance; (d) Lateral front cut-in; (e) Straight-through at an intersection. These special scenarios collectively constituted the special-event subset of our dataset, enabling the integration of comprehensive driving behavior data and special-event data, two components that are often difficult to coexist within a single dataset. High-order feature derivation and annotation To transform raw data into high-level features with physical and semantic interpretability, this study developed a post-processing algorithm, which was provided as part of the Reference Code, that served as an automatic annotation mechanism for the dataset. Geometric feature extraction Road grade was a key parameter for energy modeling. Unlike simple trigonometric approximations, this study derived the instantaneous road grade computationally by utilizing the changes in the vehicle’s three-dimensional coordinates between consecutive time steps. The atan2 function was used to distinguish uphill and downhill conditions and to avoid numerical instability. Physics-based power estimation To address the lack of energy-related information in existing datasets, this study derived the instantaneous demanded power (Pdemand) based on standard vehicle longitudinal dynamics equations. The demanded power was computed as the sum of the power required to overcome rolling resistance, aerodynamic drag, grade resistance, and acceleration resistance. These calculations were systematically executed across the continuous dataset using the provided Power Demand Calculation.py script. Here, Demand_Power denotes the wheel-end mechanical power required by the driving condition. Positive values indicate traction power demand, whereas negative values indicate mechanical braking power at the wheel end during deceleration. Electrical-side regenerative braking, including drivetrain efficiency, motor efficiency, regenerative braking limits, and battery acceptance constraints, is not directly modeled in the released dataset. This part can be customized with the appropriate model according to the user’s own needs. Semantic driving state classification Semantic annotation of driving behaviors To support fine-grained driving intent prediction and behavior analysis, this dataset established an automated semantic annotation framework based on vehicle kinematic states. This component of the data was generated through a post-processing pipeline that mapped continuous physical trajectories into discrete driving action primitives. All behavior labels were processed with temporal consistency smoothing. Lateral driving behaviors were used to describe the straight-driving, lane-changing, and turning states of the ego vehicle during continuous driving. To improve label reliability, this study developed a dual-source fusion annotation method driven by the IMU-based heading angle and GPS-based absolute position information. The IMU signal provided the vehicle yaw angle Ψ(t), which describes instantaneous attitude variation, while the GPS position coordinates x(t) and y(t) were used to reconstruct the vehicle trajectory and further derive both a position-based heading angle and pure lateral-offset features. For the IMU-based heading method, the yaw-angle sequence was first unwrapped to avoid misclassification caused by discontinuities between −π and π. A 5 s short window was then used to identify local lateral oscillations, while a 12 s long window was used to capture sustained steering trends. The net heading-angle variation within the long window was defined as where TL=12s, where |ΔψLt|>25∘, the state was preferentially classified as turning34,35. For lane-change detection, the peak-to-peak heading-angle variation and the net heading-angle variation within the 5 s short window were further calculated. If the peak-to-peak variation was no less than 5° and the net variation is smaller than 3∘, the state was classified as a lane-changing candidate; otherwise, it was preliminarily classified as straight driving. For GPS position information, two types of features were extracted. The first was the position-derived heading angle, calculated from the vehicle trajectory as This feature reflects the actual motion direction of the vehicle and complements the IMU-based heading angle. The second was the pure lateral-offset feature. This method did not use the IMU heading angle, nor did it use the position-derived heading angle as a decision variable. Instead, a smoothed reference trajectory p®t was first constructed, and the lateral offset of the vehicle relative to this trajectory was then calculated as Where pt=xt,ytT, and nt is the local normal unit vector of the smoothed trajectory. In the implementation, an 8 s smoothing window was used for the position sequence, a 10 s long window was used for turning detection, and a 5 s short window was used for lane-change detection. When the lateral-offset ratio exceeded 0.032, or when the trajectory curvature remained above 0.07 m^{−1}, the state was classified as a turning candidate. When the peak-to-peak lateral offset within the 5 s window was no less than 1.5 m and the net lateral offset was smaller than 0.8 m, the state was classified as a lane-changing candidate. Because this method used physical quantities different from those used in the IMU-based heading method, it served as an independent consistency-check method. The validation results showed that the independent outputs of the IMU-based heading method and the pure lateral-offset method had an agreement rate of 86.45%, indicating that a single information source still produced discrepancies in local transition stages, small-amplitude oscillations, and complex curved road segments, as shown in Fig. 6. The agreement rate between the position-derived heading method and the IMU-based heading method was 89.37%, indicating that the two heading cues were generally consistent but still showed local differences, as shown in Fig. 7. Therefore, the final lateral behavior labels were generated by fusing the IMU-based heading angle and the position-derived heading angle. As shown in Fig. 8, the fused labels achieved 95.5% agreement with the original IMU-based heading method and 94.0% agreement with the position-derived heading method. In addition, Fig. 9 shows that the complete comprehensive passenger-car dataset was manually annotated and used as the reference. The fused method achieved an accuracy of 97.0% against the manual annotations, demonstrating its reliability in identifying straight driving, lane changing, and turning behaviors. Users can use the check.py provided in the dataset for analysis. Agreement rate of IMU-based heading method and the pure lateral-offset method. Agreement rate of IMU-based heading method and the position-derived heading method. Fused labels agreement. (A is IMU-based heading method; B is position-derived heading method). Fused labels agreement with manual labels. (A is IMU-based heading method; B is position-derived heading method). Longitudinal behavior dynamics identification: Longitudinal behavior recognition adopted a velocity trend regression analysis based on a sliding time window (with an empirically optimal duration of 5 seconds). By performing linear fitting on the velocity sequence within the window, the rate of speed change (slope) was extracted as the decision criterion, enabling vehicle states to be subdivided into four typical conditions: stationary, cruising, acceleration, and deceleration. This trend-based logic filtered out high-frequency sensor noise and captured more macroscopic driving intent. A temporal smoothing algorithm was applied, enforcing minimum duration constraints for specific behaviors to ensure continuity. Behavior composition and operating condition statistics: The comprehensive driving behavior subset further provided a joint distribution matrix of lateral and longitudinal behaviors. Through the Cartesian product of “lateral–longitudinal” behavior combinations, researchers could efficiently index complex composite driving scenarios. The generated statistical reports recorded the frequency of each behavior combination and the proportion of time duration. Semantic reconstruction and local coordinate transformation To address the environmental perception requirements of the comprehensive driving behavior dataset, this study further implemented a semantic reconstruction algorithm that transformed raw discrete world-coordinate data into traffic flow features with high-level driving semantics. To accommodate curved roads and dynamic traffic environments, the algorithm first established an ego-centric local coordinate frame. For each data frame, the ego vehicle pose was treated as the origin, and the absolute world coordinates of all traffic participants were projected into relative longitudinal distances (dx) and lateral offsets (dy) with respect to the ego vehicle. This transformation effectively “unfolded” globally curved roads into locally straight coordinate representations. Adaptive lane width estimation In the absence of high-definition map priors, this study adopted a data-driven approach to estimate lane geometric properties. By performing statistical analysis on the lateral distribution of a large number of vehicle trajectories in the comprehensive driving behavior dataset, the algorithm computed a globally robust median of lateral distances for vehicles within a specified longitudinal range (e.g., 10 m= 0;Time(MS) >= 0Actor_0_PosX/Y/Z(M)Ego Vehicle’s global coordinatem1 Hz/60 Hzfloat64Finite map-coordinate values, [−1000,1000]Actor_0_RotX/Y/Z(R)Euler angle of ego vehicle in 3D space (roll/pitch/yaw)rad1 Hz/60 Hzfloat64RotX/RotY: finite small-angle values; RotZ: [-pi, pi]Actor_0_Speed(M/S)Ego Vehicle’s absolute speedm/s1 Hz/60 Hzfloat64[0, 40]Actor_0_Throttle(P)Driver Acceleration Pedal Opening%1 Hz/60 Hzfloat64[0, 1]Actor_0_Brake(P)Driver Brake Pedal Opening%1 Hz/60 Hzfloat64[0, 1]Actor_0_Steering(P)Driver Steering Angledeg1 Hz/60 Hzfloat64[−545, 545]Actor_0_RPMCurrent Number of Rotations Per Minute of Engine/Motorrpm1 Hz/60 Hzfloat64[0,3000]Actor_0_GearThe Current GearN/A1 Hz/60 HzintNon-negative [0,6]Actor_0_LinearVelocity_X/Y/ZThe Linear Speed Component of Ego Vehicle on the Three Axlesm/s1 Hz/60 Hzfloat64X/Y [−30, 30], Z [−0.1, 0.1].Actor_0_LinearAcceleration_X/Y/ZThe Linear Acceleration Component of Ego Vehicle on the Three Axlesm/s^21 Hz/60 Hzfloat64X/Y [−5, 5], Z [−0.1, 0.1].Actor_0_AngularVelocity_X/Y/ZThe Angular Speed Component of Ego Vehicle on the Three Axlesrad/s1 Hz/60 Hzfloat64X/Y [−0.1, 0.1], Z [−1, 1].Slope(rad)Current Slope Derived from 3D Displacementrad1 Hz/60 Hzfloat64[−0.08, 0.08]Calculated Power(W) / Demand_PowerCurrent Power Demand Derived from Longitudinal Dynamic ModelW1 Hz/60 Hzfloat64[−400, 200]Lane_same/left1/right1_countVehicle Counts of Ego Vehicle’s Current Lane and Neighbor Lanescount1 Hz/60 Hzint / float64>= 0Pedestrian_countPedestrians Count of Current Perception Rangecount1 Hz/60 Hzint64 / float64>= 0FrontDist_1/2/3…Relative Longitudinal Distance of Multi-Lane Target Vehicles in Frontm1 Hz/60 Hzfloat64>= 0 or NaNNeighborDist_1/2/3…Aggregated Information on the Relative Distance of Adjacent Vehicles on the Sidem1 Hz/60 Hzfloat64>= 0 or NaNPedestrianNearestDistThe Euclidean Distance from Nearest Pedestrian from Ego Vehiclem1 Hz/60 HzFloat>= 0 or NaN Variable descriptions of ego-vehicle. Frame >= 0; Time(MS) >= 0 Table 6Variable descriptions of other actors.VariableDescriptionUnitSampling RateData TypeValid RangeActor_N_IDUnique Identifier for Nth surrounding traffic participantN/A60 HzIntPositive integer or NaNActor_N_PosX/Y/ZAbsolute spatial position of the Nth surrounding traffic participantm60 HzFloatFinite map-coordinate values, [−1000, 1000]Actor_N_SpeedAbsolute speed of the Nth surrounding traffic participantm/s60 HzFloat[0, 40] or NaNActor_N_RotZSpatial yaw angle of the Nth surrounding traffic participantrad60 HzFloat[-pi, pi] or NaNActor_N_dxThe longitudinal distance of the Nth surrounding participant to the ego vehiclem60 HzFloat[−200, 200] or NaNActor_N_dyThe lateral distance of the Nth surrounding participant to the ego vehiclem60 HzFloat[−200, 200] or NaN Variable descriptions of other actors. Actor IDs are dynamically sorted in real-time based on their Euclidean proximity to the ego vehicle. If a scenario contains fewer than 20 actors, the remaining unpopulated actor columns are padded with NaN values. There are strictly no missing values (NaN) for the ego-vehicle dynamic states. Technical Validation Comparison with other datasets The data files consist of two main components: a comprehensive driving behavior dataset and a special driving event dataset. The comparison between our dataset and several publicly available datasets is shown in Table 7: Table 7Comparison With Other Datasets.NameFocusType of InformationV2X dataContinuous DrivingSpecial EventsPower data3ITVP(Ours)Autonomous Driving & Power InteractionsTarget-Level Semantic, Drivers’Input, Power StatusyesyesyesyesNGSIM37Micro Traffic FlowTraffic surveillance video,2D Vehicle Trajectories,Dynamic StatusyesnononohighD38Naturalistic Driving BehaviorDrone Top-down Videos, High-Fidelity Trajectories,Bounding BoxesnonononoNREL Fleet DNA39Energy Efficiency of Commercial VehiclesEnergy Consumption/Emissions Logs GPSnoyesnoyesVED28Energy Consumption of VehiclesGPS Trajectories, Vehicle Speed, Battery/fuel Consumption Logs.noyesnoyesWaymo Open40Driving PerceptionsCamera Images,LiDAR/Radar Point Cloud, 3D Bounding Boxesyesnoyesno Comparison With Other Datasets. Target-Level Semantic, Drivers’ Input, Power Status Traffic surveillance video, 2D Vehicle Trajectories, Dynamic Status Drone Top-down Videos, High-Fidelity Trajectories, Bounding Boxes Energy Consumption/ Emissions Logs GPS Camera Images, LiDAR/Radar Point Cloud, 3D Bounding Boxes To ensure the high fidelity, physical realism, and machine-readability of the dataset, a series of checks and analytical validations were systematically conducted. Temporal and kinematic consistency check To verify the temporal synchronization between driver operations and vehicle response, variable correlation analyses were performed across the continuous driving subset. As visually validated in Figs. 10 and 11, the extracted time-series plots reveal strict synchronous patterns. An increase in throttle input immediately triggers corresponding, undelayed rises in engine RPM, vehicle acceleration, and demanded power. Similarly, governed by the coordinate sign convention, every peak in the steering wheel angle is instantly followed by a proportional trough in the vehicle’s angular velocity. These tightly coupled relationships confirm that the multi-dimensional variables within the dataset are perfectly time-synchronized and faithfully reflect the kinematic laws of real-world driving without temporal misalignment. Temporal coupling between driver control inputs and vehicle dynamic responses. (a) Time against throttle; (b) Time against vehicle speed; (c) Time against rotations per minute; (d) Time against linear acceleration; (e) Time against power demand; (f) Time against brake position. Temporal coupling between steering input and angular velocity. (a) Time against steering angle; (b) Time against angular velocity. As Fig. 12 shows, the gap between the performances of an electric vehicle(EV) and fuel vehicle are not significant in most cases, especially during times when the vehicles are undergoing a long term of uninterrupted and steady driving. Meanwhile, the EVs tend to have larger peaks, which is due to its higher power peaks, along with the shorter reaction time required upon demand. These features end up with that the EVs are more likely to present a more fluctuating speed curve, which also means they usually tend to take larger acceleration and deceleration actions. The comparison between electric/fuel vehicles used in the proposed dataset. (a) Comparison between electric vehicle and fuel vehicle’s power demand; (b) Comparison between electric vehicle and fuel vehicle’s speed. What is worth noting is that, given that the surrounding vehicles, pedestrians and bicycle actors during simulation are generated under a random basis, even if the vehicles are performing the same way, there are still chances in which their trajectories are randomly affected by the actors, which finally leads to unforeseeable results. Physical boundary and data integrity verification A data quality check was carried out across all scenario files to examine unexpected missing values (NaNs) and physical anomalies. The validation confirmed zero missing values for all core ego vehicle states throughout the dataset. Furthermore, over 99.99% of the dynamic data stably fall within normal daily driving physical boundaries. A negligible fraction of extreme values was identified and physically verified. For instance, there are some cases in which the brake and deceleration come out to be extremely high, however, during the data quality check, it was found that the situation was that the ego vehicle was driving through a complex section with other vehicles trying to cut-in in front of it, which validate the high braking. Deliberately retaining these outliers preserves the authenticity and structural robustness of the simulated outputs. Statistical distribution validation The macroscopic realism of the dataset was validated through fundamental statistical distribution checks. Figure 13 presents the statistical analyses of vehicle velocity and demanded power. The data is predominantly concentrated in the 4–8 m/s mid-to-low speed range (accounting for nearly 1,500 seconds) and the 0–15 kW positive driving power interval (exceeding 3,000 seconds). This highly centralized distribution perfectly aligns with the physical expectations of normal urban driving behaviors, reflecting common states such as steady cruising and car-following. Furthermore, the statistical distributions of the meteorological environment data (Fig. 14) span a comprehensive and logical spectrum from routine daily weather to extreme adverse conditions (e.g., heavy rain and snow). The specific distribution is shown in Tables 8 and 9. These consistent and expected statistical patterns confirm that the generated scenarios accurately and reliably mirror the most frequent operating domains of real-world urban traffic. Statistical distributions of vehicle speed and demanded power in the comprehensive driving behavior dataset. (a) Vehicle speed’s distribution; (b) Vehicle power demand’s distribution. Hierarchical distribution of weather conditions and wind characteristics. Table 8Weather Property and times (Rain, Snow, Fog).TypeRainSnowFogPropertyLightModerateHeavyLightModerateHeavyLightModerateHeavyTimes1671616716313Total39397 Weather Property and times (Rain, Snow, Fog). Table 9Weather Property and times (Sandstorm and Wind).TypeSandstormWind SpeedWind DirectionPropertyLightModerateHeavyUnwindLv1Lv2Lv3Lv4BeamHeadFollowingTimes33316324327191919Total97357 Weather Property and times (Sandstorm and Wind). Usage Note Intended use Decision-making and Planning By providing unbiased ground-truth data, this dataset can be used to train and validate behavioral decision-making, trajectory planning, and driving strategies that consider energy consumption constraints for autonomous driving. Users can introduce appropriate error values and evaluate the results by comparing the decision-making outcomes with the ground truth. EMS Validation Because it contains continuous dynamic physical quantities and explicit driving intention labels, this dataset serves as an ideal benchmark for developing energy management, energy recovery optimization, and energy distribution algorithms. Edge scenario testing It is recommended to utilize the rich meteorological matrices and extreme conflict scenarios provided by this dataset (e.g., passing through a densely congested intersection when the road is slippery, adhesion is low, there are strong crosswinds, and visibility is poor) to conduct stress tests on the physical boundaries and safety of control algorithms. Time-series prediction (Vehicle speed prediction, power prediction, etc.): This dataset contains rich kinematic information, GPS data, and other content, as well as surrounding vehicle information and driver input information not found in many other datasets, supporting users in conducting deep and comprehensive explorations into time-series prediction tasks. Data limitations Sensor realism and measurement uncertainty The dataset uses ideal target-level perception to provide clean and time-aligned ground-truth information. This design helps isolate decision-making, planning, and energy-related tasks from perception errors. However, it also means that the dataset does not contain raw sensor noise, calibration errors, occlusion-induced uncertainty, detection failures, or measurement drift. Therefore, the dataset is not intended for perception-level benchmarking or raw sensor-fusion evaluation. Users who study Sim-to-Real transfer, robust planning, or uncertainty-aware control should introduce task-specific noise models or perception error models before deployment-oriented evaluation. Map fidelity The dataset is generated on high-fidelity digital twin maps, which provide consistent road geometry, lane structure, and traffic context. Nevertheless, the maps cannot fully reproduce all small-scale deviations that may occur in real roads, such as local pavement deformation, construction-zone changes, lane-marking degradation, curb irregularities, and temporary traffic-control modifications. These differences may influence localization, road-grade estimation, and local trajectory planning in real-world applications. Therefore, the dataset should be regarded as a physically consistent and reproducible simulation benchmark rather than a complete replacement for real-road map validation. Actuator delays and vehicle execution dynamics The simulated vehicle dynamics provide coherent responses to steering, throttle, and braking commands. However, real vehicles may exhibit additional actuator delays, hydraulic or electric braking lag, steering-system backlash, drivetrain response delay, and controller-area-network communication latency. These effects are not explicitly parameterized in the current dataset. As a result, algorithms trained only on this dataset may overestimate closed-loop tracking accuracy or response speed when transferred to physical vehicles. For hardware-in-the-loop or real-vehicle deployment studies, users should add actuator delay models and vehicle-specific execution constraints. Traffic-participant rationality Although the dataset includes diverse traffic interactions and special events, the behavior of surrounding traffic participants is still generated within the logic of the simulation platform. Real human drivers may show more heterogeneous, irrational, or socially interactive behaviors, including hesitation, aggressive negotiation, rule violations, and non-smooth maneuvers. This limitation should be considered when the dataset is used for behavior prediction, risk assessment, or interaction-aware planning. Energy model simplifications The provided power-related variable represents theoretical wheel-end mechanical power demand derived from vehicle longitudinal dynamics. Negative power values indicate the theoretical upper bound of recoverable mechanical energy during deceleration. The dataset does not directly model battery SOC, battery temperature and other energy storage related model. Therefore, the dataset supports power-demand prediction, eco-driving analysis, and energy-management studies, but detailed electrical-side energy consumption or regenerative braking analysis requires users to couple the dataset with their own powertrain, battery, and efficiency models. Recommended split Because the dataset contains systematically generated scenarios and multiple processed versions of the same driving records, users should construct train/validation/test splits at the file-group or scenario-group level rather than at the frame level. Random frame-level splitting is not recommended, because adjacent samples from the same trajectory, the same Frankfurt map segment, or the same special-event template may share nearly identical road geometry, traffic context, and environmental factors. For the comprehensive driving behavior subset, the segmented files should be treated as the minimum splitting units. Data derived from the same original trajectory, including the 60 Hz records, the 1 Hz downsampled records, and the reduced 1800 s driving-cycle file, should be assigned to the same split whenever they overlap in time. If the research goal is temporal generalization, users may split the data by non-overlapping continuous time blocks or segment IDs. If the goal is spatial generalization, complete map regions or route segments should be held out from training and used only for validation or testing. For the special driving event subset, files should be grouped according to event type and environmental factors encoded in the file names. Users should avoid placing near-duplicate cases with the same event type, wind direction, wind intensity, and weather condition into different splits. For scenario generalization, a complete event category, such as lateral front cut-in or opposing-direction conflict, can be held out for testing. For environmental generalization, one or more weather types, wind levels, or friction conditions can be excluded from training and used as validation or test cases. Overall, the recommended principle is to split the dataset by independent sources of variation, including scenario type, vehicle type, weather condition, wind setting, friction condition, time block, and map region. This practice reduces near-duplicate leakage and provides a more reliable estimate of model generalization. Author contributions Zhengxian Chen, Yuyang Pan, Tianyang Gong, Chenggang Zhang prepared the manuscript. Zhengxian Chen, Yuyang Pan built the simulation platform. Zhengxian Chen, Jieyu Wang, Yunwei Li, Huanan Wang managed data. Zhengxian Chen, Yuyang Pan, Wenfeng Guo, Mingqiang Wang, Tianyang Gong performed technical validations. Zhengxian Chen, Chaosheng Huang, Jun Li devised, planned and supervised the project. All authors reviewed the manuscript. Funding The research is funded by the National Natural Science Foundation of China (T2241003). Data availability All the data mentioned in the manuscript are derived from the 3ITVP dataset: https://doi.org/10.5281/zenodo.20416386. Code availability The relational codes mentioned in this study and a brief description (readme.md) have been uploaded in Zenodo, same as the dataset, https://doi.org/10.5281/zenodo.20416386. Or Please contact the first author with any further queries regarding code availability. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Citations Data Availability Statement All the data mentioned in the manuscript are derived from the 3ITVP dataset: https://doi.org/10.5281/zenodo.20416386. The relational codes mentioned in this study and a brief description (readme.md) have been uploaded in Zenodo, same as the dataset, https://doi.org/10.5281/zenodo.20416386. Or Please contact the first author with any further queries regarding code availability.