Consistent multi-animal pose estimation in cattle using dynamic Kalman filter based tracking
Maarten Perneel* (ORCID 0000-0002-4831-5357), Ines Adriaens (ORCID 0000-0001-9768-2308), Ben Aernouts † \dagger (ORCID 0000-0001-6266-3019), Jan Verwaeren † \dagger (ORCID 0000-0001-9978-501X) · 2025
Read Consistent multi-animal pose estimation in cattle using dynamic Kalman filter based tracking by Maarten Perneel* (ORCID 0000-0002-4831-5357), Ines Adriaens (ORCID 0000-0001-9768-2308), Ben Aernouts † \dagger (ORCID 0000-0001-6266-3019), Jan Verwaeren † \dagger (ORCID 0000-0001-9978-501X) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10450.txt.
Introduction
Watching cattle behavior by camera is cheaper than human observers, but most software is built for one question at a time. Maarten Perneel, Ines Adriaens, Ben Aernouts, and Jan Verwaeren present a pose-tracking method in 2025 that records each animal's keypoints across time for reuse.
Their tracker, KeySORT, pairs pose estimates with an adaptive Kalman filter that builds tracks without leaning on bounding boxes, which steadies keypoints that flicker frame to frame. Tests found up to eighty percent of true keypoints at high accuracy. The authors focus on cattle but stress the method carries to other species. A stable, higher-level record of movement, they argue, lets one data set answer many behavior questions at once.
Catalog record
| CREATOR | Maarten Perneel* (ORCID 0000-0002-4831-5357), Ines Adriaens (ORCID 0000-0001-9768-2308), Ben Aernouts † \dagger (ORCID 0000-0001-6266-3019), Jan Verwaeren † \dagger (ORCID 0000-0001-9978-501X) |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 95 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-10450
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Consistent multi-animal pose estimation in cattle using dynamic Kalman filter based tracking Maarten Perneel* (ORCID 0000-0002-4831-5357), Ines Adriaens (ORCID 0000-0001-9768-2308), Ben Aernouts † \dagger (ORCID 0000-0001-6266-3019), Jan Verwaeren † \dagger (ORCID 0000-0001-9978-501X) Abstract Abstract Over the past decade, studying animal behaviour with the help of computer vision has become more popular. Replacing human observers by computer vision lowers the cost of data collection and therefore allows to collect more extensive datasets. However, the majority of available computer vision algorithms to study animal behaviour is highly tailored towards a single research objective, limiting possibilities for data reuse. In this perspective, pose-estimation in combination with animal tracking offers opportunities to yield a higher level representation capturing both the spatial and temporal component of animal behaviour. Such a higher level representation allows to answer a wide variety of research questions simultaneously, without the need to develop repeatedly tailored computer vision algorithms. In this paper, we therefore first cope with several weaknesses of current pose-estimation algorithms and thereafter introduce KeySORT (Keypoint Simple and Online Realtime Tracking). KeySORT deploys an adaptive Kalman filter to construct tracklets in a bounding-box free manner, significantly improving the temporal consistency of detected keypoints. In this paper, we focus on pose estimation in cattle, but our methodology can easily be generalised to any other animal species. Our test results indicate our algorithm is able to detect up to 80% of the ground truth keypoints with high accuracy, with only a limited drop in performance when daylight recordings are compared to nightvision recordings. Moreover, by using KeySORT to construct skeletons, the temporal consistency of generated keypoint coordinates was largely improved, offering opportunities with regard to automated behaviour monitoring of animals. Keywords : pose estimation, pose tracking, adaptive Kalman filter, behaviour monitoring 1 Introduction Dairy cattle show an extensive behavioural repertoire. The behaviour of an individual or group of animals is often used as an indicator for animal health and welfare, and can be linked to productivity [ 1 ] . However, the manual observation of animal behaviour is a laborious task and the observer can be concentrated for only a limited timespan [ 2 ] . Recent work has shown that, for some behaviours such as mounting and drinking, the human observer can successfully be replaced by a computer vision system [ 3 , 4 ] . However, for other behaviours, especially social behaviours involving several animals simultaneously, it proves difficult to replace the human observer completely by a computer vision system. Moreover, the majority of computer vision algorithms developed in the past is highly tailored towards a single research objective. For example, [ 4 ] studied the influence of heat stress on the drinking behaviour of cattle, while [ 5 ] monitored the feeding behaviour of individual animals. These tailored approaches, however, limit the possibilities for data reuse. Alternatively, video data could first be converted into a generic higher level representation. If designed thoughtful, such a representation can retain most of the relevant information, while having a lower memory requirement [ 6 , 7 , 8 ] . Moreover, if the representation space is constructed with meaningful features, subsequent extraction of behavioural information of interest can be performed easily and data reuse will be facilitated. In the case of dairy cattle, such a higher level representation space could be defined by the location of several behaviour-determining keypoints, such as the nose, withers and tail implant. To capture both the spatial and temporal behaviour patterns, these keypoints should not only be detected on separate video frames, but also be tracked. However, current research considers pose estimation and tracking independently from each other [ 9 , 10 ] , whereas the integration of both of them offers remarkable opportunities to improve the accuracy and consistency of the obtained keypoint locations. In this work, we propose an algorithm which integrates pose-estimation and keypoint-tracking. Our algorithm generates for each animal in a pen a pre-defined set of keypoints in a reliable, consistent and computationally efficient manner. As such, the problem we solve is an instance of the more general multiple-object pose estimation and tracking problem. We optimised the algorithm for group-housed dairy cattle, but it can easily be generalised to different animal species or object categories. In barn environments, several animals are present in most frames, requiring multi-animal pose estimation (MAPE) algorithms that allow estimation of the pose of each animal per frame. MAPE involves both the detection of keypoints and grouping of the detected keypoints per animal. Classical top-down methods originating from multi-person pose estimation [ 11 , 12 ] handle this problem by first detecting individuals in a frame, often using bounding box detectors, after which keypoint detection is performed for each detected bounding box. On the other hand, most recently published MAPE algorithms such as Deeplabcut [ 9 ] and SLEAP [ 10 ] use bottom-up methods to perform MAPE. Bottom-up methods first detect keypoints in a frame and associate keypoints per animal in a second step to construct a single skeleton per animal. For the keypoint association step, generally part affinity fields are used, which were introduced by [ 13 ] , but were already used before under a slightly different form by [ 14 ] . According to the implementation of [ 14 ] , for two connected keypoints A and B, such as the tail implant and the shoulders of a pig, part affinity fields indicate, conditionally on the presence of a keypoint of category A, where a corresponding keypoint of category B could be located and vice versa. When compared to top-down implementations of MAPE, bottom-up approaches have two advantages that are important for our application: they allow faster inference and provide more accurate results in dense scenes [ 15 ] . Especially the last aspect is highly relevant when studying socially interacting animals. To study social behaviours and make a distinction between and agonistic and affiliative behaviours, not only the spatial component of animal behaviour, but also the temporal component should be analysed [ 16 , 17 ] . Therefore, tracking of detected skeletons is necessary. Moreover, since some social behaviours take place on the sub-second to seconds scale, temporal frequency of the tracklets should be relatively high while keeping the computational cost of constructing them feasible. A popular approach to track objects is tracking by re-identification [ 18 , 19 , 20 ] . However, most re-identification algorithms are based on neural networks and require the availability of segments or object detections of the individuals of interest, which would nullify any computational advantage of using bottom-up MAPE. Alternatively, computationally less intensive approaches could be used, such as the Kalman-filter based SORT (Simple and Online Realtime Tracking) [ 21 ] . However, all published Kalman-filter based tracking algorithms still require bounding boxes generated by object detection, which would again neutralise the computational advantages of using bottom-up MAPE. We therefore introduce KeySORT (Keypoint Simple and Online Realtime Tracking), a dynamic Kalman filter based tracking algorithm inspired by SORT, but able to track directly (in)complete skeletons constituted of several related keypoints. The advantages of tracking individual keypoints are twofold: (1) no additional bounding box detectors or re-identification steps are needed, and (2) Kalman filtering has a stabilising effect