Parameter-Efficient Fine-Tuning of Vision Foundation Model for Forest Floor Segmentation from UAV Imagery
Mohammad Wasil 1,2,* , Ahmad Drak 1,3 , Brennan Penfold 1,3 , Ludovico Scarton 1,2,3 , Maximilian Johenneken 1,3 , Alexander Asteroth 1,3 , Sebastian Houben 1,2,3 · 2025
Read Parameter-Efficient Fine-Tuning of Vision Foundation Model for Forest Floor Segmentation from UAV Imagery by Mohammad Wasil 1,2,* , Ahmad Drak 1,3 , Brennan Penfold 1,3 , Ludovico Scarton 1,2,3 , Maximilian Johenneken 1,3 , Alexander Asteroth 1,3 , Sebastian Houben 1,2,3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08932.txt.
Introduction
Seven authors adapt a large vision model to map the forest floor from drones in this 2025 paper. Drones help with reforestation and forest checks, but the ground layer is messy, fast changing, and hard to label, with stumps, plants, and woody debris mixed together.
The team fine tuned only a small set of added parameters in the Segment Anything Model, leaving the main weights fixed. They also adjusted its mask decoder to output their ground categories without hand prompts. An adapter method gave the best overlap scores, while a low rank method used fewer parameters and suits small drones. The work supports cheap forest surveys from the air.
Catalog record
| CREATOR | Mohammad Wasil 1,2,* , Ahmad Drak 1,3 , Brennan Penfold 1,3 , Ludovico Scarton 1,2,3 , Maximilian Johenneken 1,3 , Alexander Asteroth 1,3 , Sebastian Houben 1,2,3 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08932
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Parameter-Efficient Fine-Tuning of Vision Foundation Model for Forest Floor Segmentation from UAV Imagery Thanks: * Corresponding author Mohammad Wasil 1,2,* , Ahmad Drak 1,3 , Brennan Penfold 1,3 , Ludovico Scarton 1,2,3 , Maximilian Johenneken 1,3 , Alexander Asteroth 1,3 , Sebastian Houben 1,2,3 Abstract Unmanned Aerial Vehicles (UAVs) are increasingly used for reforestation and forest monitoring, including seed dispersal in hard-to-reach terrains. However, a detailed understanding of the forest floor remains a challenge due to high natural variability, quickly changing environmental parameters, and ambiguous annotations due to unclear definitions. To address this issue, we adapt the Segment Anything Model (SAM), a vision foundation model with strong generalization capabilities, to segment forest floor objects such as tree stumps, vegetation, and woody debris. To this end, we employ parameter-efficient fine-tuning (PEFT) to fine-tune a small subset of additional model parameters while keeping the original weights fixed. We adjust SAM’s mask decoder to generate masks corresponding to our dataset categories, allowing for automatic segmentation without manual prompting. Our results show that the adapter-based PEFT method achieves the highest mean intersection over union (mIoU), while Low-rank Adaptation (LoRA), with fewer parameters, offers a lightweight alternative for resource-constrained UAV platforms. I Introduction Unmanned Aerial Vehicles (UAVs) are increasingly utilized for reforestation and forest monitoring efforts [ 1 ] . For instance, UAV-assisted seeding can mitigate biological challenges by optimizing water availability through precise depth placement and controlled seed spacing [ 2 ] . Additionally, UAVs can efficiently disperse seeds in hard-to-reach terrain, which can potentially support large-scale reforestation efforts. However, identifying suitable seed locations presents an additional challenge, as factors such as water content, plant competition, and physical obstacles like woody debris, logs, and tree stumps must be considered. In this work, we focus on obstacle detection using object segmentation methods. Pre-trained large vision models (LVMs) or vision foundation models become more available for different tasks such as object recognition [ 3 ] , semantic segmentation (e.g., via the Segment Anything Model (SAM)) [ 4 ] , and Language-Image Understanding [ 5 ] . These models were trained on large amounts of images, videos or texts datasets such as ImageNet, CIFAR-100, or WebImageText [ 5 ] . Nevertheless, the generalization of these models to remote sensing data from UAVs is more challenging due to the complexity of backgrounds, the diversity of scenes [ 6 ] , and fewer and smaller datasets. Fine-tuning the whole parameters of a foundation model is computationally expensive. Moreover, in [ 7 ] , Hu et al. claim that fine-tuning a portion of parameters results in comparable performance to that of fine-tuning the whole model. The knowledge transfer to downstream tasks is typically done through parameter-efficient fine-tuning (PEFT) methods that aim at introducing small numbers of trainable parameters, known as PEFT modules, to the original and frozen foundation model parameters [ 8 ] , [ 9 ] , [ 10 ] , [ 7 ] . In this work, we focus on applying parameter-efficient fine-tuning (PEFT) methods for segmenting forest floor data. We use the Segment Anything model (SAM) [ 4 ] , a semantic segmentation model, as the foundation model for segmenting forest floor objects. Despite strong zero-shot generalization, SAM segmentation performance can degrade on specific dataset [ 11 ] . For example, in [ 11 ] , Mazurowski et al. claim that SAM performs better on bigger medical objects. Furthermore, SAM only provides masks without category information. To address these limitations, we apply parameter-efficient fine-tuning (PEFT) techniques to adapt SAM to our dataset and modify SAM mask decoder such that the number of segmented masks corresponds to the number of categories in our dataset. II Related Work In this section, we review parameter-efficient fine-tuning methods, such as adapters and Low-Rank Adaptation (LoRA), which can be employed to adapt the Segment Anything Model (SAM). II-A Segment Anything Model The Segment Anything model (SAM) is a large, pre-trained image segmentation model trained on more than 11 million images and 1 billion+ masks and used as the foundation model for several downstream tasks such as medical image analysis [ 10 ] , crack segmentation in construction [ 12 ] , and remote sensing [ 6 ] . SAM consists of three components, namely an image encoder, a prompt encoder and a mask decoder. The image encoder is based on the Vision Transformer (ViT) model [ 3 ] . SAM necessitates prompts to guide the model to output masks given an image input. It supports two types of prompts: 1) sparse prompts with points, box and text; 2) dense prompts with masks. II-B Parameter-Efficient Fine-Tuning Methods Parameter-efficient fine-tuning (PEFT) techniques allow us to adapt pre-trained foundation models to specific tasks efficiently by adding and training only a small number of additional parameters. The parameters of the foundation model are kept frozen during this process, which leads to a significant reduction in both memory usage and computational expense. Examples of PEFT methods include Adapter [ 8 ] , [ 9 ] , [ 10 ] and Low-rank Adaptation [ 7 ] . II-B 1 Adapter An adapter tuning method injects new neural network modules into the original foundation model. In [ 8 ] , Houlsby et al. proposed an adapter method for transfer learning in NLP where two adapter layers are injected into each Transformer layer: one adapter is injected after the multi-head attention and two feed-forward layers as shown in Fig. 1 . The adapter module is a bottleneck neural network architecture where the d -dimensional features are mapped into a smaller dimension and then projected back to d -dimensional output features. Fig. 1: Adapter injects two serial adapters into each transformer block [ 8 ] . One adapter is injected after the Multi-Head Attention module, and the other is injected after the MLP module. Fig. 2: AdaptFormer introduces a bottleneck adapter alongside with a MLP layer named AdaptMLP [ 9 ] . In [ 9 ] , Chen et al. introduced a more efficient version of adapter layers by introducing a single residual adapter layer per transformer block. The residual adapter module is inserted alongside the MLP module as depicted in Fig. 2 . The module is called AdaptMLP that also introduces a scale factor s to the residual connection. The authors claim that the parallel design of adapters in AdaptMLP gives better performance than serial adapters. In [ 10 ] , Wu et al. combined the original adapter with AdaptMLP. The first adapter is added after the multi-head attention layer and before the layer norm and the second adapter is added in parallel to the MLP layer. II-B 2 Low-Rank Adaptation In [ 7 ] , Hu et al. proposed the Low-Rank Adaptation (LoRA) method, which fine-tunes low-rank matrices for downstream tasks. Their approach is based on the hypothesis that full-rank matrices are not required for downstream tasks, making it unnecessary to fine-tune the entire foundation model. Fig. 3: LoRA architecture with a trainable matrix product BA [ 7 ] , where the original weight matrix remains frozen. The final embedding h is obtained by element-wise summation of the outputs from the pre-trained weights and the LoRA module. LoRA introduces two additional trainable matrices in parallel to the foundation model