Benchmarking EfficientTAM on FMO datasets

Senem Aktas, Charles Markham, John McDonald & Rozenn Dahyot · 2025

Read Benchmarking EfficientTAM on FMO datasets by Senem Aktas, Charles Markham, John McDonald & Rozenn Dahyot (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06536.txt.

Introduction

Tracking objects that streak across an image is benchmarked by Senem Aktas and three co-authors in a 2025 computer vision paper. Fast moving objects blur into streaks, which makes them hard to follow. Several open datasets record such sequences, but their notes about each frame were scattered and incomplete.

The team first writes a shared metadata file in JSON format for four public datasets. They then extend it into a richer file, called FMOX, that adds ground-truth object sizes for the sequences. With that file they test EfficientTAM, a recent foundation model for tracking, meaning a large model meant for general use. Its scores, measured by how well predicted and true paths overlap, compare well with pipelines built specially for these datasets. All code and data files are shared openly for other groups.

Catalog record

CREATORSenem Aktas, Charles Markham, John McDonald & Rozenn Dahyot
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06536

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Benchmarking EfficientTAM on FMO datasets Senem Aktas, Charles Markham, John McDonald & Rozenn Dahyot Abstract Fast and tiny object tracking remains a challenge in computer vision and in this paper we first introduce a JSON metadata file associated with four open source datasets of Fast Moving Objects (FMOs) image sequences. In addition, we extend the description of the FMOs datasets with additional ground truth information in JSON format (called FMOX) with object size information. Finally we use our FMOX file to test a recently proposed foundational model for tracking (called EfficientTAM) showing that its performance compares well with the pipelines originally taylored for these FMO datasets. Our comparison of these state-of-the-art techniques on FMOX is provided with Trajectory Intersection of Union (TIoU) scores. The code and JSON is shared open source allowing FMOX to be accessible and usable for other machine learning pipelines aiming to process FMO datasets. Keywords: Fast Moving Object, FMO Dataset, Labeling, EfficientTAM 1 Introduction The tracking and detection of small and/or fast-moving objects (FMOs) remains relatively underexplored, particularly in comparison to research on general objects [ Zhang et al., 2022 ] . Specific challenges occur in such scenarios including motion blur, which can distort the appearance of objects and complicate the performance of general object detectors and trackers. Moreover, data annotation becomes more difficult due to the size and motion blur of FMOs, leading to a scarcity of available annotated datasets [ Zhang et al., 2022 , Yu et al., 2020 , Rozumnyi et al., 2017 ] . Despite these issues, several public FMO datasets, including Falling Object [ Kotera et al., 2020 ] , FMOv2 [ Rozumnyi et al., 2017 ] , TbD [ Kotera et al., 2019 ] , and TbD-3D [ Rozumnyi et al., 2020 ] are now available (see Section 2 ). Unlike larger objects, small objects often suffer from reduced visibility and lower image cover rate, leading to fewer appearance cues and increased background interference. Considering both issues, fast and small object tracking, are therefore still challenging for modern tracking techniques [ Zhang et al., 2022 , Haalck et al., 2024 , Yu et al., 2020 ] . In this work, we define and characterize the terms small and fast in the context of object tracking in videos. Second, we provide a combined and extended dataset, which we term FMO eXtended (FMOX), combining the four datasets mentioned above with a more informative metadata encoding. This encoding is provided through a simplified JSON format, allowing the development of Machine Learning dataloaders and pipelines that operate over the entire unified dataset (Section 3 ). Finally, we assess EfficientTAM (Efficient Track Anything Model) [ Xiong et al., 2024 ] , a recent state-of-the-art technique for tracking, on these datasets, benchmarking its performance against leading FMO specific techniques from the literature. Code and FMOX JSON are made available at https://github.com/CVMLmu/FMOX/ . 2 Fast-Moving Object Datasets The following datasets 1 1 1 available at https://cmp.felk.cvut.cz/fmo/ are considered here to create FMOX. Falling Objects [ Kotera et al., 2020 ] . The dataset comprises static image frames from video sequences recorded at 25/30 Frame Per Second (FPS). It comprises a collection of 6 different objects, including box, cell, key, and rubber, which are dropped from a table, as shown in Figure 2 . It includes two sets of images as high-speed and low-speed sequences, with ground truth provided in text format. FMOv2 [ Rozumnyi et al., 2017 ] . This dataset includes 19 sports sequences featuring a variety of small objects, primarily balls, such as volleyballs and tennis balls, but also objects such as darts and frisbees, with PNG images of varying lengths. The ground truths are provided in Matlab for each sequence, with object mask PNG format and its run length encoding compressed text file. In the dataset, FMO displacement is evenly spread between 0-150 pixels while bounding boxes between consecutive frames have nearly zero intersection each time. Figure 2 illustrates samples from the dataset, with the FMOv2 extension sequence samples highlighted in red. Figure 1: Falling Objects [ Kotera et al., 2020 ] samples. Figure 2: FMOv2 [ Rozumnyi et al., 2017 ] samples. TbD [ Kotera et al., 2019 ] . This dataset comprises striking golf, tennis, volleyball and badminton balls as well as a falling cube and coin. 14 image sequences are provided with ground-truth trajectories including 12 sequences from sports videos with mostly spherical objects and almost no appearance changes over time. These 30 FPS low-speed videos are obtained from 240 FPS high-speed videos using temporal averaging. Semi-manual annotation are applied for creating ground truth by labeling the first frame and applying a tracker, and then correcting the annotations. TbD-3D [ Rozumnyi et al., 2020 ] . Similar to TbD, the TbD-3D dataset includes 10 PNG image sequences, focusing on objects moving in 3D that undergo significant changes in appearance within a single low-speed videos. 9 of these sequences depict a spherical object, specifically a ball, while the last sequence features a circular object. The 6D poses (3D position and rotation) of FMOs are provided as ground truth with manually annotated 3D object location (2D position and radius) and estimated 3D object rotation. Videos were captured in raw format with a high-speed camera operating at 240 FPS. Both high-speed and low-speed versions of the image sequences are provided. Ground truths are available in both MATLAB format and text format as trajectories for the low-speed image sequences. Figure 4 shows a sample frame from each sequence. Figure 3: TbD [ Kotera et al., 2019 ] samples. Figure 4: TbD-3D [ Rozumnyi et al., 2020 ] samples. 3 FMOX In this section, we define FMOX, a JSON structured annotation format that simplifies the handling of the four public FMO datasets described in Section 2 ). It is designed to be easily interpretable and compatible, enabling researchers and practitioners to quickly understand the structure and content of the data, allowing them to use the datasets with minimal effort. We also provide object size categories as additional labeling for the datasets. By considering size as a key factor in the annotations, we aim to enhance the relevance of the dataset for various research applications, such as small object detection and tracking. Object Size. In the computer vision community, the term "small objects" characterizes as covering an area of 32\times 32 pixels or less [ Tong et al., 2020 , Lin et al., 2014 , Zhang et al., 2022 ] . For instance, the Microsoft COCO benchmark [ Lin et al., 2014 ] categorizes objects into three size categories for evaluation as small , medium , and large based on the dimensions of the bounding boxes that encapsulate the objects. Object size categorization facilitates the analysis of detector and model performance across various object sizes [ Zhu et al., 2016 ] . Table 1 presents object size categories as defined in the literature. Table 2 presents our object size categories used in FMOX. Table 3 summarizes the FMOX statistics, primarily focusing on object size information. Additionally, the listing 1 displays the structure of FMOX for the four analyzed FMO datasets. Study Resolution Categorization Traffic sign [ Zhu et al., 2016 ] 2048\times 2048 small (0,32] , medium

Read the complete text