BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing
Fangxun Liu, S M Rayeed, Samuel Stevens, Alyson East, Cheng Hsuan Chiang, Colin Lee, Daniel Yi, Junke Yang, Tejas Naik, Ziyi Wang, Connor Kilrain, Elijah H. Buckwalter, Jiacheng Hou, Saul Ibaven Bueno, Shuheng Wang, Xinyue Ma, Yifan Liu, Zhiyuan Tao, Ziheng Zhang, Eric Sokol, Michael Belitz, Sydne R · 2025
Read BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing by Fangxun Liu, S M Rayeed, Samuel Stevens, Alyson East, Cheng Hsuan Chiang, Colin Lee, Daniel Yi, Junke Yang, Tejas Naik, Ziyi Wang, Connor Kilrain, Elijah H. Buckwalter, Jiacheng Hou, Saul Ibaven Bueno, Shuheng Wang, Xinyue Ma, Yifan Liu, Zhiyuan Tao, Ziheng Zhang, Eric Sokol, Michael Belitz, Sydne R (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00255.txt.
Introduction
Biologists sort beetles by laying them on trays and photographing each tray, which leaves thousands of images to process by hand. Fangxun Liu and a large team built BeetleFlow, a three-stage pipeline that handles the pile automatically. Stage one finds each beetle on a tray, stage two sorts and crops each insect, and stage three outlines body parts in fine detail.
Detection pairs a transformer-based finder with a vision-language model in repeated rounds. For the outlining stage, the team hand-labeled hundreds of beetle images and tuned two segmentation models. The pipeline turns a slow manual chore into a fast step that can speed up ecology research.
Catalog record
| CREATOR | Fangxun Liu, S M Rayeed, Samuel Stevens, Alyson East, Cheng Hsuan Chiang, Colin Lee, Daniel Yi, Junke Yang, Tejas Naik, Ziyi Wang, Connor Kilrain, Elijah H. Buckwalter, Jiacheng Hou, Saul Ibaven Bueno, Shuheng Wang, Xinyue Ma, Yifan Liu, Zhiyuan Tao, Ziheng Zhang, Eric Sokol, Michael Belitz, Sydne R |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00255
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing Fangxun Liu, S M Rayeed, Samuel Stevens, Alyson East, Cheng Hsuan Chiang, Colin Lee, Daniel Yi, Junke Yang, Tejas Naik, Ziyi Wang, Connor Kilrain, Elijah H. Buckwalter, Jiacheng Hou, Saul Ibaven Bueno, Shuheng Wang, Xinyue Ma, Yifan Liu, Zhiyuan Tao, Ziheng Zhang, Eric Sokol, Michael Belitz, Sydne Record, Charles V. Stewart, Wei-Lun Chao, [.5em] The Ohio State University Abstract In entomology and ecology research, biologists often need to collect a large number of insects, among which beetles are the most common species. A common practice for biologists to organize beetles is to place them on trays and take a picture of each tray. Given the images of thousands of such trays, it is important to have an automated pipeline to process the large-scale data for further research. Therefore, we develop a 3-stage pipeline to detect all the beetles on each tray, sort and crop the image of each beetle, and do morphological segmentation on the cropped beetles. For detection, we design an iterative process utilizing a transformer-based open-vocabulary object detector and a vision-language model. For segmentation, we manually labeled 670 beetle images and fine-tuned two variants of a transformer-based segmentation model to achieve fine-grained segmentation of beetles with relatively high accuracy. The pipeline integrates multiple deep learning methods and is specialized for beetle image processing, which can greatly improve the efficiency to process large-scale beetle data and accelerate biological research. 1 Introduction In entomology and ecology research, biologists often need to collect a large number of insects to study, among which beetles are one of the most common species. Beetles account for around 25% of all known species in the world [ 1 ] , therefore, they have significant research value in a variety of fields such as evolution and biodiversity, for their species richness and widespread distribution. In practice, biologists often use pins to mount beetles collected at the same site and time on a tray. A tray contains from several to over 60 beetle specimens. After collection, biologists can have up to thousands of trays and take a picture of each tray to digitize. This procedure brings in a large amount of beetle data organized by trays, but how to process them for further study becomes a new question. Given the importance of beetle study and the challenges, we develop a 3-stage deep learning pipeline to process large-scale beetle images. In the first stage, we utilize an open-vocabulary detector, Grounding DINO [ 2 ] , for beetle detection and a vision-language model, LLaVA-NeXT [ 3 ] , for final verification. This approach achieves a high accuracy of 97.81%. In the second stage, we crop each detected beetle from the tray image and save it as a single image, with optional sorting and metadata matching. In the third stage, we leverage a transformer-based model, Mask2Former [ 4 ] , to segment each beetle into 5 or 9 morphological parts. The model achieves a mean Intersection over Union (mIOU) of 85.11% for 5 class segmentation and 77.38% for 9 class segmentation. Multiple downstream research on beetles can be conducted based on the high-throughput data processed by our pipeline. Moreover, the pipeline has the potential to generalize to more biological data processing cases, as we have observed a similar detect-and-segment pattern in other biological pipelines, such as QuPath [ 5 ] for cells and PlantCV [ 6 ] for plants. This pattern is applicable to a variety of biological data processing workflows, and our work on beetles sets an example for insects, which are one of the most numerous groups of organisms in the world. 2 Related Work While traditional object detectors like the R-CNN family [ 7 ] and YOLO [ 8 ] are limited by predefined datasets, open-vocabulary detection methods [ 9 , 10 ] now leverage language prompts to detect arbitrary objects. Similarly, Vision-Language Models (VLMs) [ 11 , 12 ] extend the reasoning of large language models [ 13 , 14 , 15 ] to the multimodal domain for joint text-image reasoning. In parallel, the state-of-the-art in semantic segmentation has shifted from Convolutional Neural Networks (CNNs) like U-Net [ 16 ] to transformer-based models [ 17 , 18 , 19 ] . These models leverage self-attention for global context, which is critical for distinguishing morphologically similar parts in biological imaging. These advancements are relevant to beetle studies, where machine learning, supported by large-scale beetle sampling programs like NEON [ 20 ] , is already applied for identification and classification. Existing works range from traditional algorithms on extracted features [ 21 ] to deep learning methods, such as CNNs for identification [ 22 ] and deep vision models for fine-grained taxonomic classification [ 23 , 24 ] . 3 Beetle Image Processing Pipeline The input to our pipeline is a series of images of trays containing multiple beetles. Each tray image undergoes a 3-stage processing (Figure 1 ). The code and data are available at https://github.com/Imageomics/BeetleFlow.git . Figure 1: An overview of the 3-stage pipeline: individual detection, cropping/metadata matching, and body-part segmentation stages. 3.1 Iterative Beetle Detection The first stage is iterative beetle detection. In one iteration, the tray image and the text prompt “a beetle” are sent as input to Grounding DINO, which then outputs the bounding box coordinates for the detected beetles. Next, white masks are placed over all detected beetles based on the bounding boxes, leaving the undetected ones in the tray image. The resulting modified image then proceeds to the next iteration for another round of detection and masking. When no detection is reported, the iteration stops. The final modified image is then sent to LLaVA-NeXT with the text prompt “Do you see beetles in this image?”. We constrain the model to output “YES” or “NO” as the final word for automated check. If it answers “YES”, a message is sent to the user to check manually. If it answers “NO”, the detection is successful and the bounding box coordinates are sent to the next stage. 3.2 Beetle Image Cropping The second stage takes the bounding box coordinates as input and outputs individual cropped beetle images, with optional functions of sorting and metadata matching. Given the bounding box coordinates, the pipeline crops the beetles out of the original tray image and saves them as individual images. A specific order can be applied when saving the beetle images. In practice, the beetles on a tray are arranged in regular rows and columns. If digitized metadata for the beetles are available, they are typically provided in a left-to-right, top-to-bottom order by the biologists. The pipeline sorts the beetle images according to the top-left bounding box coordinate and associates the metadata with each beetle. The metadata matched to the beetles are saved in a CSV file for each tray. 3.3 Fine-grained Beetle Segmentation The third stage takes individual beetle images as input and outputs the morphological segmentation results for each beetle. For this task, we fine-tune two variants of Mask2Former, based on granularity: a 5-class model (segmenting head, pronotum, elytra, legs, and antennas) for basic morphological analysis, and a 9-class model (additionally eyes, mouthparts, tail, and pin). Both models inherently separate the beetle from the background. For