A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments
Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, Rudolf Mester · 2025
Read A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments by Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, Rudolf Mester (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25969.txt.
Introduction
Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, and Rudolf Mester built a tracking system in 2025 to watch the welfare of farmed salmon. Underwater scenes are hard for cameras: fish block each other, look alike, and move alike. Past tools handled one welfare sign at a time and cost much effort to run.
Their system finds each fish and its body parts, then tracks the parts in fine detail. Special steps handle the loss of identity when fish crowd or turn. Welfare signs, such as tail beat patterns, are then read from the tracks. The method beat a leading tracker built for people on both new salmon test sets.
Catalog record
| CREATOR | Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, Rudolf Mester |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-25969
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, Rudolf Mester Abstract Computer Vision (CV)-based continuous, automated and precise salmon welfare monitoring is a key step toward reduced salmon mortality and improved salmon welfare in industrial aquaculture net pens. Available CV methods for determining welfare indicators focus on single indicators and rely on object detectors and trackers from other application areas to aid their welfare indicator calculation algorithm. This comes with a high resource demand for real-world applications, since each indicator must be calculated separately. In addition, the methods are vulnerable to difficulties in underwater salmon scenes, such as object occlusion, similar object appearance, and similar object motion. To address these challenges, we propose a flexible tracking framework that uses a pose estimation network to extract bounding boxes around salmon and their corresponding body parts, and exploits information about the body parts, through specialized modules, to tackle challenges specific to underwater salmon scenes. Subsequently, the high-detail body part tracks are employed to calculate welfare indicators. We construct two novel datasets assessing two salmon tracking challenges: salmon ID transfers in crowded scenes and salmon ID switches during turning. Our method outperforms the current state-of-the-art pedestrian tracker, BoostTrack, for both salmon tracking challenges. Additionally, we create a dataset for calculating salmon tail beat wavelength, demonstrating that our body part tracking method is well-suited for automated welfare monitoring based on tail beat analysis. Datasets and code are available at https://github.com/espenbh/BoostCompTrack. 1 Introduction Enhancing the welfare of salmon in industrial net pens necessitates a detailed understanding of the health status of the fish. This information has traditionally been extracted by manual observation of sampled fish, which limits the quantity and quality of the data obtained. In recent years, it has been demonstrated that computer vision methods can be used to extract several welfare indicators [ 7 , 30 , 12 , 5 ] , offering new, more precise, and automated methods for salmon monitoring. Although promising, a number of challenges limit the real-world applicability of the proposed methods. Figure 1 : A visualization of our proposed pipeline BoostCompTrack . Bounding boxes around salmon and salmon body parts are detected by a top-down pose model, before each salmon component is tracked separately and fused in a way that prevents certain challenges in underwater salmon tracking. The resulting tracks of salmon and salmon body parts can be leveraged for a diverse set of tasks, including segmentation mask generation with Segment Anything [ 10 ] (as demonstrated in the middle image on the right) and tail beat wavelength estimation (as visualized in the top image on the right). Firstly, the demonstrated algorithms are often developed with ”ideal” data from controlled experimental setups. In commercial aquaculture facilities, the data is considerably more challenging. Complete and partial object occlusion is common and objects (fish in this case) move and appear very similar. Typical underwater scenes show significant illumination variability, motion blur, marine snow, and color-cast effects. Moreover, labeled data are a scarce resource, and the non-rigid camera mounting introduces motion, while the uniform background renders conventional camera motion compensation methods ineffective. Secondly, each welfare indicator (or small group of indicators) is calculated by its own dedicated pipeline, often consisting of multiple deep neural networks. This results in excessive computational load, especially when considering multiple indicators concurrently to achieve a more comprehensive welfare estimate. Finally, extracting welfare indicators is usually heavily dependent on object localization and object tracking, yet available publications on salmon tracking and welfare monitoring tend to use object detectors and trackers from other application areas with little to no adaptation [ 15 , 31 , 18 , 7 ] . This leads to two main issues. Firstly, these trackers consider a point or a bounding box for each object, which is often not sufficiently informative to separate fish in dense crowds. Secondly, any welfare indicator that is associated with specific locations on the salmon body requires an additional module to extract these locations. This module is challenging to construct, as it needs to both delineate a body part, and distinguish which body part belongs to the salmon of interest and which belongs to nearby fish occluding that particular salmon. Several animal tracking methods operate with object keypoints [ 11 , 16 ] , which are sufficiently informative for some welfare indicators [ 7 ] , while others benefit from bounding boxes [ 20 , 5 ] or segmentation masks [ 32 , 2 ] . Another challenge with pose estimation models is that they rely on distinguishable object keypoint locations for accurate pose inference. Furthermore, for tracking association, keypoints are often less robust than bounding boxes, which provide a more stable representation for matching objects across frames. To address the challenges posed by tracking salmon in real-world underwater scenes, we propose a flexible object localization and tracking framework that explicitly copes with the difficulties specific to underwater salmon scenes and provides detailed object information that can be directly used to determine welfare indicators. To achieve this, we use a top-down pose estimation network that first detects bounding boxes enclosing each clearly visible salmon and subsequently determines finer bounding boxes around specific body parts. A state-of-the-art bounding box tracker is utilized to track detected salmon components separately, before specialized modules are employed to merge the associations at the two different scales (salmon and body parts) together to individual object associations. This fusion step addresses task-specific challenges in salmon tracking, including often observed track ID transfers in crowded scenes and track ID switches when salmon are turning. Following the tracking, the temporal evolution of the tail is determined to compute the salmon tail beat wavelength, demonstrating also the applicability of our tracker for salmon welfare monitoring. Additionally, the body parts can be prompted with foundational models (such as Segment Anything [ 10 ] ) to obtain tracks of segmented fish, which are displayed in Fig. 1 . To evaluate our tracking method, we assembled two challenging salmon tracking datasets covering two underwater application cases. The first targets automatic data collection for training individual fish appearance models, requiring the tracker to maintain identity consistency and avoid switching between individuals (i.e., prevent fish ID transfer). Furthermore, it benefits from having no appearance model, as such a model could introduce bias into the collected dataset by favoring certain fish tracks based on the feature embeddings of this particular model. Learning individual appearance models is crucial for long-term salmon re-identification and, consequently, for monitoring individual salmon welfare. To assess this task, we compiled the CrowdedSalmon (CS) dataset, containing densely distributed salmon schooling in front of the camera ( Fig. 2 ). The second application case addresses the challenge of associating observations of a salmon’s left and right sides, captured at different times as the fish turns, into a single individual identity. This enables improved surveillance and individual monitoring. With a single camera, this can only be achieved by maintaining a salmon track during turning, which is challenging since the turning fish changes appearance and bounding box shape. To support this use case, we developed the TurningSalmon (TS) dataset, which features more dynamic scenes with multiple salmon turning simultaneously ( Fig. 2 ). Additionally, we