Estimation of Kinematic Motion from Dashcam Footage
Evelyn Zhang, Alex Richardson, Jonathan Sprinkle · 2025
Read Estimation of Kinematic Motion from Dashcam Footage by Evelyn Zhang, Alex Richardson, Jonathan Sprinkle (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01104.txt.
Introduction
Evelyn Zhang, Alex Richardson, and Jonathan Sprinkle test whether dashcam video can measure a car's motion in a 2025 paper. Their question is practical: how accurately can footage from a dashboard camera predict the true kinematic motion of a car-like vehicle?
To answer it, they pair video with ground truth from the car's own data stream, sent over its controller area network. The camera is time-synchronized with that stream and mounted on a consumer-grade vehicle. The study covers many hours of footage and driving. Reliable video-based motion estimates could help crash analysis and driver monitoring where no other sensors exist.
Catalog record
| CREATOR | Evelyn Zhang, Alex Richardson, Jonathan Sprinkle |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01104
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Estimation of Kinematic Motion from Dashcam Footage Evelyn Zhang, Alex Richardson, Jonathan Sprinkle Abstract The goal of this paper is to explore the accuracy of dashcam footage to predict the actual kinematic motion of a car-like vehicle. Our approach uses ground truth information from the vehicle’s on-board data stream, through the controller area network, and a time-synchronized dashboard camera, mounted to a consumer-grade vehicle, for 18 hours of footage and driving. The contributions of the paper include neural network models that allow us to quantify the accuracy of predicting the vehicle speed and yaw, as well as the presence of a lead vehicle, and its relative distance and speed. In addition, the paper describes how other researchers can gather their own data to perform similar experiments, using open-source tools and off-the-shelf technology. I Introduction and Related Works One application of the extraction of kinematic data from dashcam videos is the ability to detect specific events that occur while driving from just the footage, such as turns, lane changes, and passes. Others have employed different methods for accomplishing this, such as training videos to descriptions of the videos. However, this relies on having existing descriptions of the events in the videos. Some use a combination of GPS/IMU data and footage with marked objects, but again, this relies on having objects in videos marked by hand correctly. Finally, others seek to mark less complex driving events such as a car being stationary, slow, normal, or turning, which are events that can be marked if kinematic data can be derived from footage [ 8 , 10 , 12 , 3 ] . In order to train neural networks to produce predictions that are reliable enough to use for other applications, a large dataset is necessary. Currently, BDD100k [ 14 ] is among the largest available datasets [ 2 ] with over a thousand hours of footage. Although there is an ample amount of footage, speed is the only reliable kinematics data available, with regard to the kinematics data important to this study. The BDD100k also had accelerometer data, but due to inconsistent collection of data causing roughly half the values to be negative, the data cannot be trained on. Our dataset is collected from a single vehicle driven primarily in Arizona. The data include arterial and city roads, freeways, night and day, and multiple types of weather. We collect CAN data directly from the car, which is synchronized with video data from dashcam footage. This allows us to train and predict a more diverse set of kinematics. For this experiment, we use a total of 18 hours of footage, with 15.5 hours acting as our training set, and 2.5 hours used as a validation dataset to verify our results. Other datasets exist such as KITTI [ 5 ] , ApolloScape [ 7 ] , Waymo Open [ 11 ] , etc. However, many are smaller than 18 hours, and others do not contain the data we want to train on, such as different times of day and different types of weather or the various kinematics we use. The data we have collected is from the controller area network (CAN) bus, which is collected directly from the vehicle. In order to understand the collected CAN messages, we use the Strym tool [ 1 ] to decode the relevant CAN messages into specific kinematic data. This allowed us to extract and convert specific data points into usable tensors for training. II Problem Formulation We describe our approach to designing our final experiment, and the motivations behind certain choices. We also describe the preliminary testing performed to collect context for final test parameters and design. II-A Desired Attribute Prediction One application that we are hoping to use the results of this study for is the ability to automatically mark specific events in footage, such as turns, overtaking a car, lane switches, etc. Marking these events in footage will help us better understand the contents of the videos and allow us to perform further analysis. In order to determine these events from just footage, we train the models on speed, yaw, the presence of a lead car, the distance of a lead car, and the relative speed of a lead car. Once these have been predicted by the model, we can automatically find events by designing a set of conditions to be met by the predicted values (e.g., yaw exceeding a certain number indicates a right turn). Additionally, focusing on using models to predict this set of attributes allows us to add events that we have not considered yet without requiring more training, as opposed to directly training on marked events. II-B BDD100k Preliminary Testing Preliminary testing was performed on the BDD100k (Berkeley DeepDrive 100k) dataset in order to ensure that some level of learning was achievable, and to learn what reasonable paramters were for experiments of this type and scale. The BDD100k dataset is split into 40 second chunks of video footage collected primarily in New York, San Francisco, and Berkeley, totaling to about 1000 hours. The dataset is comprehensive in comparison to others previously mentioned, containing footage taken in the rain, at night, during cloudy days, and on highways. For our testing, we primarily focused on the speed and yaw data, which was collected using phones positioned on the dashboard of the cars. The dataset needed to be modified in order to be used for our training purposes. The videos were stretched to form a square and the videos were set to 5 frames per second. With too many frames per second, training takes too long, but with not enough, information is lost. We leave to future work the exploration of frame rate as a hyperparameter. The first round of testing we performed was using the U-NET autoencoder [ 9 ] . The BDD100k dataset includes one frame per video which contains annotations of objects, and the autoencoder was trained on these frames. Every video was processed into 512 latents using this autoencoder, and a simple grid search was conducted for speed. There were four models: one non-recurrent baseline neural network, two recurrent neural networks of GRU [ 4 ] and LSTM [ 6 ] , and a transformer [ 13 ] . The results of our testing are shown in Table I . This shows the results of three learning rates, with each model running for 50 epochs, and a batch size of 32, two layers in the neural network, and the RNNs used a hidden size of 512. LR Baseline GRU LSTM Transformer 1\text{\times}{10}^{-1} 25.64 52.32 50.57 44.96 1\text{\times}{10}^{-3} 19.77 8.97 10.07 14.63 1\text{\times}{10}^{-5} 44.23 12.88 14.23 15.36 TABLE I : Validation Set MSE (m/s) of Models after 50 epochs The use of an autoencoder can be effective if the output targets can be fine tuned for ground truth information. However, in the context of this work, the use of an autoencoder will have limitations, which motivates the use of a convolutional neural network (CNN) which allows us to retrain the algorithms on the BDD dataset, to establish a baseline for comparison against our own dataset in the future sections. The best results of our CNN had the setup of [3, 32, 64,