SAMJ: fast image annotation on ImageJ/Fiji via segment anything model Authors: Carlos García-López-de-Haro, Caterina Fuster-Barceló, Curtis T Rueden, Jónathan Heras, Vladimír Ulman, Daniel Franco-Barranco, Adrián Inés, Kevin W Eliceiri, Jean-Christophe Olivo-Marin, Jean-Yves Tinevez, Daniel Sage, Arrate Muñoz-Barrutia Published: 2026 ABSTRACT AbstractAccurate image annotation is essential for training artificial intelligence (AI) systems in biomedical image analysis, enabling tasks such as cell detection, tissue quantification, and disease characterization. However, creating pixel-level annotations is a time-consuming and labor-intensive process that requires expert input, limiting the development and adoption of AI methods. Recent advances in foundation models, such as the Segment Anything Model (SAM), enable interactive object segmentation from simple user prompts, but their integration into widely used bioimage analysis platforms remains limited and often requires technical expertise. Here we show that SAMJ, a user-friendly plugin for ImageJ/Fiji, enables fast, interactive, and accurate image annotation on standard computers without requiring programming skills or specialized hardware. SAMJ integrates efficient SAM variants into a familiar graphical interface, allowing users to delineate objects in large scientific images in real time using simple clicks or bounding boxes. This approach significantly reduces annotation effort, accelerates dataset creation, and broadens access to advanced AI-assisted annotation tools for the biomedical research community.Subject terms: Image processing, Machine learning Introduction Access to high-quality annotated images is key to training and developing new methods for biomedical image analysis and object segmentation. However, creating large datasets of annotated images is a time-consuming, labor-intensive, and subjective process1. The demand becomes even more pronounced when training supervised Deep Learning (DL) models, typically requiring hundreds to thousands of annotated images2. Therefore, there is a critical need for efficient annotation tools capable of running on commonly available computers. Such tools must reduce IT-related barriers, ensuring broader accessibility for domain-specific experts such as biologists. These solutions should also be compatible with widely used bioimage software platforms, such as ImageJ/Fiji3, and must be fast enough to accelerate the annotation of dozens of objects in large images, which are typical in biological research and essential for training machine learning models. Recently, Meta Research released the Segment Anything Model (SAM)4, a foundational DL framework designed for one-shot and promptable image segmentation. SAM represents a significant breakthrough, capable of delineating complex objects and enabling semi-automated, interactive annotation across diverse image types. In its standard mode of operation, SAM requires user-defined prompts, such as points, sets of points, or bounding boxes, to efficiently isolate target objects. SAM has been made available in commonly used bioimage software packages, such as QuPath5 and Napari6 (see section 16), providing interactive annotation, but has not yet been incorporated into the ImageJ ecosystem7. Other proprietary software, such as Matlab, ArcGIS, Kili Technology, or Unitlab Magic Touch have incorporated SAM as an add-on tool to enhance their segmentation capabilities. Similarly, Segment Anything Annotator, an open-source Python-based UI application, uses SAM for pixel-level annotation. Although many annotation tools are both user-friendly and widely accessible, they often face challenges when dealing with uneven illumination, overlapping structures, or subtle intensity gradients. Several plugins have introduced more advanced methods: AnnotatorJ8, a popular Fiji plugin, supports semi-automatic cellular annotation by combining manual input with deep learning; Labkit9 leverages machine learning for pixel-wise classification; and Thomas et al.10 propose a streamlined approach to systematic manual annotation by assigning predefined categories to images or regions. However, none of these plugins yet benefit from the advantages that SAM provides. In this manuscript, we present SAMJ, a plugin that brings the capabilities of SAM into the Fiji ecosystem, both as a standalone Fiji plugin and as an extension for Labkit9. We designed SAMJ for one-click installation. It features a no-code interface, and runs on standard computers without requiring specialized hardware. By lowering technical barriers, it broadens access to cutting-edge AI tools within the life sciences community. A key contribution of our work is the tailored adaptation of SAMJ to efficiently handle scientific images. By harnessing the powerful segmentation capabilities of SAM, this tool has the potential to significantly accelerate the annotation process, enabling users to instantly generate regions of interest around target objects with a single point click or bounding box. Results Design and architecture of the SAMJ plugin It is important to note that SAMJ does not introduce a novel model for bioimage segmentation. Instead, it provides a software tool that facilitates the use of an existing foundational model (SAM), which we believe can be highly valuable for bioimage annotation, especially within Fiji where its environment, usability, and macro language make it an ideal setting for high-throughput annotation and automation. SAMJ leverages the core architecture of SAM, which is built upon three key components: an image encoder, a prompt encoder, and a mask decoder. The image encoder is based on a Visual Transformer (ViT)11 that encodes the input image, offering high representational power at the expense of significant computational resources. The prompt encoder is another transformer to encode the prompt given by the user to point or hint to the location of the object of interest. Finally, the encoded image and prompt are used by the mask decoder, which combines elements of both a transformer and a Convolutional Neural Network, to generate the mask of interest. Accessibility and usability The SAMJ plugin is installed as any standard Fiji extension, eliminating the technical IT barriers often associated with Python packages and making advanced image segmentation accessible to a wider audience. Unlike most Python-based libraries that require command-line interfaces for environment configuration and dependency management, SAMJ simplifies the process with a seamless, one-click installation, enabling effortless deployment. In addition, SAMJ provides access to five SAM variants: SAM-212 (Tiny, Small, and Large), EfficientSAM13, and EfficientViTSAM-L214. All variants perform the same segmentation task and are trained on comparable generalist datasets, but they differ substantially in computational requirements and performance characteristics (see Supplementary Information section 1). This diversity enables users to select a model that fits both their hardware and application needs, ensuring that at least one option remains practical to run on any virtual-machine, workstation or mid-range laptop. For instance, users may prioritize speed and efficiency on less powerful machines or higher annotation accuracy on complex images. Integration with the Fiji ecosystem and Java–Python interoperability Beyond its ease of use, SAMJ is designed for adaptability and seamless integration. Powered by SAM, it can enhance annotation workflows across a range of Java-based platforms, for example, in addition to Labkit, we have also integrated it into BigDataViewer15. Moreover, to support broader adoption, SAMJ provides a well-documented API, enabling developers to easily incorporate SAM functionality into their preferred Java software effortlessly. Furthermore, it includes a software-agnostic Java GUI, facilitating straightforward integration into any Java-based environment. A key strength of SAMJ is its integration of Python methods into Java environments. This is achieved through Appose16 (see section 16), a Java package that allows Java and Python to run as separate processes yet communicate in real time. Leveraging Micromamba17, Appose also automates Python environment setup, totally eliminating manual configuration via the command line. As a result, SAMJ is remarkably user-friendly, granting effortless access to advanced methods. Interactive annotation workflow in Fiji The annotation process in the SAMJ plugin mirrors SAM’s workflow (Fig. 1). Each image is encoded once by the image encoder, the most computationally intensive step. Afterwards, prompts are processed and masks generated almost instantly, allowing users to interactively produce multiple annotations with real-time responsiveness. SAMJ integrates seamlessly with Fiji’s existing tools, allowing users to create prompts such as points and rectangles directly through Fiji’s familiar toolbar, thereby minimizing the learning curve for new users. SAMJ User Interface and Workflow with estimated time. This figure illustrates the typical workflow of SAMJ for annotating objects in an image, including the time required for each step on standard workstations. a Model Installation: The most time-consuming step, where SAMJ installs the selected model and, if necessary, sets up the environment, taking approximately 1000 s. b Image Encoding: When the user clicks “Go” in the plugin, the image is processed to generate an embedding, which takes around 10 s. c Object Annotation: Since no re-embedding is required, this step is immediate, with each annotation generated in approximately 0.1 s per user click. The annotation process can be repeated multiple times as desired for different objects, offering rapid and interactive segmentation. The SAMJ plugin offers two annotation modes: Live and Batch mode (called BatchSAMize). In Live mode, prompts are drawn directly on the encoded image, and annotations are generated interactively, one at a time. In Batch mode, users can define multiple seed points to serve as prompts for segmenting multiple objects in a raw image. These seeds can be generated automatically using traditional Fiji commands (e.g., Find Maxima, Watershed) within an ImageJ macro, followed by a call to SAMJ’s Batch mode to segment the whole image. Additionally, users can enhance annotations by providing seeds from other segmentation methods as prompts, enabling iterative refinement. Prompts for BatchSAMize can be supplied either manually or automatically. Thanks to SAMJ’s integration with Fiji, users can combine bounding boxes or point prompts generated by deep learning models, macros, or files imported through the ROI Manager. Users can even use the macros to annotate batches of images with SAMJ, as illustrated in the Supplementary Materials. This flexibility enhances both the reproducibility and the automation of image annotation. Representative use cases To illustrate the power of SAMJ and its seamless integration with the Fiji ecosystem, we present four representative use cases highlighting its capabilities (Fig. 2). These examples illustrate SAMJ’s ability to adapt to diverse image annotation tasks while taking advantage of Fiji’s powerful image processing tools. In all cases, the targets are compact and well-defined–conditions under which SAMJ performs particularly well. Elongated or branched structures may pose more of a challenge; However, users can refine prompts, adjust scale settings, or combine SAMJ with complementary methods. Experimentation is encouraged, as SAMJ often proves effective even under suboptimal imaging conditions (see section 16). Use cases demonstrating the versatility of SAMJ for bioimage analysis. a Nuclei Segmentation using Fiji’s capabilities and BatchSAMize: An image from the CellPose dataset21 is used to demonstrate the segmentation of nuclei in the red channel. Pixel intensity maxima are identified in Fiji to generate single-point prompts for each nucleus. These prompts are processed in batch mode using SAMJ, resulting in semantic segmentation of individual nuclei. b Tumor Area Quantification and Nuclei Analysis with SAMJ and StarDist plugins for Fiji: Breast cancer TMA (Tissue Microarray) images, stained with H& E (Hematoxylin and Eosin) provided by the British Columbia Cancer Agency (BCCA)28, are used to quantify tumoral regions and their nuclei. Tumoral areas are annotated with SAMJ's rectangle prompt, generating masks that are combined with the original image in Fiji using an AND operation to obtain the intersection. StarDist is then applied through deepImageJ to segment individual nuclei in the tumoral areas. c Accelerating annotation of Bacterial Motility: SAMJ is applied to annotate motile bacteria using a single rectangle annotation. When compared to Ground Truth, SAMJ's annotations achieve comparable or superior precision, showcasing its efficiency in handling complex shapes and streamlining high-throughput workflows. Each use case highlights SAMJ's integration with Fiji, combining SAM's advanced annotation capabilities with Fiji’s extensive image processing tools. d Efficient annotation of 3D Electron Microscopy images of mitochondria29 using SAMJ and Labkit: SAMJ is integrated into Labkit to support 3D and multi-label annotation. Users can annotate structures on arbitrarily oriented and scaled slices, improving visibility and accuracy of objects with complex spatial orientation. The multi-class labeling capability of Labkit allows the annotation of several distinct structures within the same volume. This workflow reduces annotation effort and ensures spatial consistency across slices. In the first use case, we illustrate nuclei segmentation in a fluorescence image using the BatchSAMize mode of SAMJ. Classical Fiji commands are used to detect nuclei, which serve as single-point prompts for batch processing in SAMJ. This workflow highlights the synergy between the preprocessing capabilities of Fiji and the segmentation power of SAMJ, streamlining large-scale annotation tasks. The second use case intended for tumor area quantification focuses on breast cancer TMA images stained with H&E. The task involved quantifying tumoral areas and the number of nuclei within these regions. Using SAMJ’s rectangle annotation feature, tumoral regions were outlined, generating masks for these areas. Subsequently, individual nuclei were segmented with StarDist18 using the deepImageJ7,19 plugin. This workflow demonstrates SAMJ’s role as a flexible and efficient annotation tool that seamlessly integrates with Fiji, enabling efficient annotation and precise quantification for tumor analysis. The third use case highlights SAMJ’s efficiency in annotating complex shapes20. Bacteria from motility studies were annotated using a single rectangle prompt per image, significantly reducing manual effort. When compared to manually generated ground truth, SAMJ’s annotations are highly consistent with the reference labels, as supported by the IoU results reported in Table 4. This use case demonstrates how SAMJ accelerates annotation tasks for intricate biological structures, making it a powerful tool for high-throughput studies. Table 4IoU of annotations produced by each model across datasetsSoftwareModel2122232425252627CCBCNCMMSAMJSAM2 Tiny*0.640.350.360.570.660.450.790.69SAM2 Tiny†0.770.740.740.840.810.720.880.82SAM2 Small*0.630.350.360.570.670.420.820.71SAM2 Small†0.770.730.740.840.820.730.880.82SAM2 Large*0.660.370.420.600.700.430.820.73SAM2 Large†0.780.740.730.840.810.720.880.82EfficientSAM*0.630.400.390.550.680.420.690.70EfficientSAM†0.740.690.680.810.770.620.870.79EfficientViTSAM-L2*0.690.500.410.610.710.470.820.73EfficientViTSAM-L2†0.800.770.760.860.850.760.900.86Micro-SAMMobileSAM LM*0.690.600.600.770.750.660.330.65MobileSAM LM†0.820.800.760.850.840.800.770.84MobileSAM EM*0.480.470.400.600.670.560.390.76MobileSAM EM†0.620.710.620.780.870.770.790.87SAM Base LM*0.740.690.700.820.740.700.460.69SAM Base LM†0.850.830.820.890.840.820.790.84SAM Base EM*0.580.490.380.580.640.530.530.81SAM Base EM†0.770.710.650.760.840.780.800.88SAM Large LM*0.770.710.740.830.750.710.490.69SAM Large LM†0.860.830.830.890.840.830.780.84SAM Large EM*0.570.500.410.570.620.530.590.82SAM Large EM†0.760.690.630.750.840.780.780.89SAM Huge*0.640.280.300.550.640.350.440.70SAM Huge†0.760.690.620.750.790.590.710.82QuPath–SAMSAM2 Tiny*0.570.290.320.570.280.360.410.75SAM2 Tiny†0.740.710.650.780.720.620.650.84SAM2 Small*0.560.320.320.550.250.340.420.70SAM2 Small†0.760.680.640.770.750.650.650.84SAM2 Base+*0.590.400.310.550.250.380.420.59SAM2 Base+†0.750.700.630.760.710.640.650.84MobileSAM*0.470.250.310.540.200.270.250.51MobileSAM†0.710.620.610.730.680.570.640.77SAM Huge LM*0.660.690.700.770.450.530.290.67SAM Huge LM†0.750.800.810.870.680.640.690.73SAM Huge EM*0.590.450.460.560.480.510.460.81SAM Huge EM†0.740.650.700.720.630.680.710.80For each software tool and model variant, we report the IoU (Intersection over Union) between the predicted masks and the ground-truth annotations on the following benchmark datasets: Cellpose21, LiveCell22, DeepBacs23, NeurIPS Cell Segmentation24, TissueNet Nuclei25, TissueNet Cells25, MitoEM26 and CEM-MitoLab27. Importantly, this evaluation measures annotation performance, not full-image segmentation: prompts were automatically derived from the ground-truth labels and each tool generated a corresponding mask, which we compared directly to the ground truth. For conciseness, in Micro-SAM and QuPath-SAM we evaluated only the fine-tuned SAM variants and SAM Huge. The fine-tuned models are expected to perform best on cellular data, and SAM Huge is not available in SAMJ.*3-point multipoint prompt, †bounding box prompt, C cells, B bacteria, N nuclei, M mitochondria, LM fine-tuned on light microscopy images, EM fine-tuned on electron microscopy images. IoU of annotations produced by each model across datasets For each software tool and model variant, we report the IoU (Intersection over Union) between the predicted masks and the ground-truth annotations on the following benchmark datasets: Cellpose21, LiveCell22, DeepBacs23, NeurIPS Cell Segmentation24, TissueNet Nuclei25, TissueNet Cells25, MitoEM26 and CEM-MitoLab27. Importantly, this evaluation measures annotation performance, not full-image segmentation: prompts were automatically derived from the ground-truth labels and each tool generated a corresponding mask, which we compared directly to the ground truth. For conciseness, in Micro-SAM and QuPath-SAM we evaluated only the fine-tuned SAM variants and SAM Huge. The fine-tuned models are expected to perform best on cellular data, and SAM Huge is not available in SAMJ. *3-point multipoint prompt, †bounding box prompt, C cells, B bacteria, N nuclei, M mitochondria, LM fine-tuned on light microscopy images, EM fine-tuned on electron microscopy images. The final use case demonstrates SAMJ’s capability for efficient annotation of mitochondria instances in 3D electron microscopy volumes through its integration with Labkit. Annotating objects in 3D is inherently challenging due to the difficulty of maintaining spatial coherence across slices and the substantial effort required for manual annotation. SAMJ streamlines this process, and its integration into Labkit–designed for interactive 3D visualization–enables users to annotate structures on arbitrarily oriented slices, ensuring optimal views of the object of interest. Moreover, Labkit support for multi-class labeling allows users to assign distinct labels to multiple structures within the same volume. This combination significantly reduces annotation effort while improving both accuracy and consistency, highlighting the power of integrating SAMJ with complementary Fiji tools for complex 3D bioimage annotation tasks. In this example, every slice of the volume was annotated with SAMJ, effectively following a pragmatic 2.5 D strategy. Thanks to SAMJ’s rapid interactive prompting, this approach still provided a substantial gain in speed compared to fully manual slice-by-slice annotation. Together, these use cases highlight the versatility and strength of SAMJ integration within the Fiji ecosystem. By combining the advanced segmentation capabilities of SAM with the rich set of image processing tools of Fiji, SAMJ enables more efficient, precise, and scalable annotation workflows across a wide range of biological imaging tasks. This integration lowers technical barriers and supports diverse use cases, from 2D fluorescence images to complex 3D volumes. Consequently, it empowers the broader life science community–particularly biologists who primarily rely on GUI software platforms–with powerful deep-learning tools that would otherwise remain accessible only to a few, accelerating and improving their bioimage analysis. Annotating complex objects with multi-step prompting The Segment Anything Model was originally trained with a one-prompt/one-object paradigm on a wide variety of objects, mostly from natural images. As a result, biological structures with intricate morphologies, such as neurons, are often difficult to annotate with a single prompt, regardless of prompt placement. However, thanks to SAMJ interactivity, users can overcome this limitation by enabling a multi-step, compositional annotation workflow. While SAM may struggle to segment a branched structure globally, it remains highly effective at detecting local boundaries. SAMJ leverages this strength by allowing users to segment distinct sub-structures (e.g., soma, axons, and dendrites) individually. These partial masks can then be merged into a single, coherent object. This piecewise strategy yields significantly more accurate annotations for elongated or highly branched structures than single-prompt inference (see Fig. 3). Multi-step annotation of a single object using different prompts. SAMJ's interactive workflow allows users to segment multiple parts of an object that the model recognizes and subsequently merge them. The figure shows a fluorescent neuron30 with two ramifications. Using a single box prompt (middle image), the model captures only one ramification. By annotating the second ramification with additional point prompts (purple dots on the right image), the user obtains two separate labels whose union yields the final mask of the complex neuron. Discussion Accurate annotation is fundamental to deep learning-based image analysis, playing a critical role in model training, validation, and fine-tuning. To address the limitations posed by the high embedding latency of SAM, particularly in large microscopy images, we developed SAMJ, an intuitive and interactive annotation tool tailored for the life sciences. SAMJ integrates efficient SAM2 models optimized to run on mid-range computers and includes a Java-Python bridge to eliminate complex installation steps. As an ImageJ/Fiji plugin, it remains fully compatible with standard Fiji tools such as ROI selection, the ROI manager, and macros. Through this seamless integration, SAMJ enables efficient, high-quality annotations and supports the creation of curated datasets, thus contributing to the development of more accurate and robust models. In conclusion, SAMJ provides a user-friendly and accessible way for biologists and bioimage analysts to adopt cutting-edge AI methods, facilitating faster and more precise image annotation and ultimately accelerating scientific discovery. Methods Ethics statement This work describes a software tool and did not involve the recruitment of human participants, animal experimentation, or the collection of new identifiable personal data. All experiments were performed on publicly available datasets, used in accordance with their terms of use and licenses. Therefore, no additional institutional ethics committee/IRB approval was required for this study. This study complies with all relevant ethical regulations. Statistics and reproducibility This study is primarily a software/methods report, and the main text is descriptive. Quantitative analyses were performed only in the comparative benchmarking experiments described in the Methods Section, in which SAMJ and two comparable tools were run on all images across the selected public datasets, and annotation performance (including IoU) were computed (Table 4). Sample sizes were determined by dataset availability and relevance. No statistical method was used to predetermine sample size. No data were excluded from the analyses. The experiments were not randomized. The Investigators were not blinded to allocation during experiments and outcome assessment. Reproducibility is supported by the use of publicly available datasets, reporting of software versions and benchmarking settings in the Methods Section, and release of the benchmarking code (https://github.com/segment-anything-models-java/benchmarking). Review of annotation tools for image segmentation We provide a comprehensive overview of the existing annotation tools for bioimage analysis, summarizing key features, such as their release dates, supported annotation types, and integration with the Segment Anything Model (SAM) or any other AI assistant (Table 1). This table highlights the diversity of tools available across platforms like Fiji, QuPath, and Napari, as well as their varying levels of SAM integration. Notably, while tools such as QuPath and Napari have embraced SAM, Fiji-based plugins like AnnotatorJ lack direct SAM support. SAMJ addresses this gap by bringing SAM’s advanced annotation capabilities to the Fiji environment, combining flexibility with ease of use. This comparative analysis underscores SAMJ’s unique contribution to enabling SAM-powered annotation within the Fiji ecosystem. Table 1Summary of tools integrating annotation capabilities in bioimage analysis platformsTool/PluginPlatformReleaseLast UpdateAI assistanceAnnotationProgramming languageMicroSAMNapari202303/2025Yes (SAM fine-tune)Polygons BBox PointsPythonQuPath Ext. SAMQuPath202309/2024Yes (SAM2)AutoMask BBox PoitnsPythonNapari Plugin of SAMNapari202304/2023Yes (SAM1)BBox PointsPythonNapari Plugin of SAM2Napari202409/2024Yes (SAM2)BBox PointsPythonAnnotatorJFiji202010/2020Yes (CNN)Instance Semantic BBoxJavaLabkitFiji202210/2024Yes (Random Forest SAM1,2)Pixel ClassificationJavaQualitative AnnotationsFiji202004/2021Yes (CNN)One or Multiple ROIPythonSAMJFiji202504/2025Yes (SAM1,2)BBox PointsJavaThe table highlights the main tools and plugins used for annotation in various bioimage analysis platforms, detailing their release (YYYY) and last update dates (MM/YYYY, Last accessed: 20 MAY 2025), annotation features, integration with the SAM, and the programming languages used for their development. SAM integration is specified with its supported annotation types, including bounding boxes (BBox), points, polygons, and more, showcasing the variety of annotation modes available. Summary of tools integrating annotation capabilities in bioimage analysis platforms The table highlights the main tools and plugins used for annotation in various bioimage analysis platforms, detailing their release (YYYY) and last update dates (MM/YYYY, Last accessed: 20 MAY 2025), annotation features, integration with the SAM, and the programming languages used for their development. SAM integration is specified with its supported annotation types, including bounding boxes (BBox), points, polygons, and more, showcasing the variety of annotation modes available. Review of SAM-based annotation tools for bioimaging We have benchmarked the most widely used SAM-based bioimage annotation tools: SAMJ, Micro-SAM6, and the SAM extension for QuPath5. Each of these three tools serves a distinct niche, and their suitability depends on the user’s requirements. Tables 2, 3 summarize aspects related to ease of use (available models, CPU-only compatibility, optional GPU acceleration, model-loading time, prompt-response speed, and one-click installation). Table 4 reports the intersection-over-union (IoU) of annotations generated by each tool across multiple datasets. Table 2Comparison of features across SAM-based annotation toolsSoftwareOne-ClickCPUGPUModelsInstallationSupportSupportAvailableSAMJYESYESNOSAM2 TinyYESYESNOSAM2 SmallYESYESNOSAM2 LargeYESYESNOEfficientSAMYESYESNOEfficientViTSAM-L2Micro-SAMNOSlowYESSAM BaseNOSlowYESSAM Base LightNOSlowYESSAM Base LM fine-tunedNOSlowYESSAM Large LM fine-tunedNOSlowYESSAM Large EM fine-tunedNOYESYESMobileSAMNOYESYESMobileSAM LM fine-tunedNOYESYESMobileSAM EM fine-tunedQuPath–SAMNONOYESSAM Huge LM fine-tunedNONOYESSAM Huge EM fine-tunedNOSlowYESSAM HugeNOSlowYESSAM LargeNOSlowYESSAM BaseNOYESYESMobileSAMNOYESYESSAM2 TinyNOYESYESSAM2 SmallNOYESYESSAM2 Base PlusNOYESYESSAM2 LargeLM Light Microscopy, EM Electron Microscopy. Comparison of features across SAM-based annotation tools LM Light Microscopy, EM Electron Microscopy. Table 3CPU-only model loading and annotation times across software toolsSoftwareModelLoad time (s)Annotation time (s)SAMJSAM2 Tiny1.910.06SAM2 Small2.000.07SAM2 Large3.560.07EfficientSAM2.690.08EfficientViTSAM-L21.370.08Micro–SAMMobileSAM LM2.270.04MobileSAM EM1.980.03SAM Base LM8.650.05SAM Base EM8.380.07SAM Large LM26.030.06SAM Large EM25.030.05SAM Huge39.390.04QuPathSAMSAM2 TinyNA0.31SAM2 Small LMNA0.32SAM2 Base Plus EMNA0.58MobileSAM LMNA0.25SAM Huge EMNAOnly GPUSAM Huge LMNAOnly GPULoad time denotes the time required to load a model and encode the image on the CPU before annotation begins. For QuPathSAM, this metric is not applicable because the software does not pre-encode the image; instead, it encodes only the region around the user-provided prompt. If subsequent prompts fall within an already encoded region, no additional computation is required; otherwise, QuPathSAM performs a new local encoding step. CPU-only model loading and annotation times across software tools Load time denotes the time required to load a model and encode the image on the CPU before annotation begins. For QuPathSAM, this metric is not applicable because the software does not pre-encode the image; instead, it encodes only the region around the user-provided prompt. If subsequent prompts fall within an already encoded region, no additional computation is required; otherwise, QuPathSAM performs a new local encoding step. Importantly, our benchmarking targets annotation performance, not full image segmentation. Annotation requires the tool to correctly identify and delineate an object given a prompt–that is, the user (or, in our evaluation, the ground truth) indicates where the object is, and the tool produces the corresponding mask. To evaluate annotation accuracy, we generated prompts directly from the ground-truth labels, produced annotations in each software using those prompts, and then compared the resulting masks with the ground-truth masks. As a final remark, we note that QuPath-SAM and Micro-SAM also provide access to the original SAM models trained on natural images. To keep the comparison focused and the tables concise, we restricted our evaluation to the fine-tuned variants, which are expected to perform more reliably on cellular datasets. We additionally include SAM Huge in our benchmarking, as this model is available on Micro-SAM but not in SAMJ. Enabling seamless integration of SAM with Fiji: the role of appose and SAMJ The integration of SAM into the Fiji ecosystem via SAMJ is facilitated by Appose16 (see Fig. 4), a software framework designed to streamline inter-process communication. Within SAMJ, Appose orchestrates communication between the Java environment–where Fiji and SAMJ reside–and the Python environment, which runs SAM’s inference via PyTorch. Overview of the integration between SAMJ and SAM via Appose. On the left, the Fiji/ImageJ environment hosts the SAMJ plugin, which acts as the interface for users within Java-based image processing tools. On the right, the Python with Pytorch runs the SAM model for segmentation tasks. Appose bridges these two environments, enabling seamless inter-process communication between Java and Python. It facilitates variable exchange via pipes and shared memory, ensuring efficient handling of images/tensors while avoiding memory duplication. This architecture encapsulates processes, preventing conflicts, and allows for the concurrent execution of multiple models. Appose employs process pipes to transmit variables between Java and Python and shared memory to exchange images and tensors efficiently, thus avoiding unnecessary data duplication. This architecture neatly segregates computational tasks, mitigating the risk of memory conflicts and enabling multiple models to be run concurrently without interference. Once a model is deselected, the associated processes are automatically terminated, ensuring a clean and self-contained execution environment. In addition to managing communication, Appose automatically handles the installation and setup of required software environments. Using Micromamba, it can create and maintain isolated environments with all necessary dependencies without requiring user intervention. This fully automated process ensures a seamless user experience, simplifying the deployment and use of SAM within Fiji. Appose creates a robust foundation for SAMJ to function as a powerful Java-to-Python bridge. This allows users to integrate SAM’s advanced segmentation capabilities, enabling the annotation of images in the Fiji ecosystem, benefiting from all tools, plugins, and capabilities of Java. Appose and SAMJ mark a significant checkpoint in the integration of novel, cutting-edge technologies developed in Python into the user-friendly world of Java. Reporting summary Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. Supplementary information Supplementary Information Reporting Summary Transparent Peer Review file Source data Source Data Acknowledgements Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. This work was partially funded by the project CNRS France 2030 Recherche à Risque (RI2) ProteoVir (J-C.O.-M. and C.G.-L.-H.). This work was partially supported by the European Union’s Horizon Europe research and program under grant agreement number 101057970 (AI4Life project) awarded to A.M-B. and the Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación, MCIN/AEI/10.13039/501100011033/, under grants PID2023-152631OB-I00 and AIA2025-164165-C41, co-financed by European Regional Development Fund (ERDF), “A way of making Europe”. This work has been funded in part by the Agence Nationale de la Recherche through the programs PR[AI]RIE-PSAI (ANR-23-IACL-0008) and France-BioImaging infrastructure (ANR-10-INBS-04) (J-C.O-M. and J-Y.T.). This project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 945358 (BIGPICTURE). This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation program and EFPIA (J-C.O-M.). J-C.O-M and J-Y.T were also supported by the French National Research Agency (France BioImaging, ANR-24-INBS-0005 FBI BIOGEN). V.U. was supported by grant number 2024-342803 from the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation, and by the core facilities CBIA and CELLIM supported by MEYS CR (LM2023050 Czech-BioImaging). C.T.R. and K.W.E were funded by the National Institutes of Health, grant number NIH P41 GM135019. J. H. and A. I. were supported by Agencia de Desarrollo Económico de La Rioja ADER 2022-I-IDI-00015, by project AFIANZA 2024/01 granted by the Autonomous Community of La Rioja, and by Grant PID2024-155834NB-I00 funded by MICIU/AEI/10.13039/501100011033. J. H. was also supported by the Government of La Rioja through Proyecto Inicia 2023/01. Author contributions Conceptualization: D.S., C.G-L.-H., A.M.-B. and J-C.O.-M. Software: C.G.-L.-H., V.U., D.S. and C.T.R. Interface design: D.S., C.G-L-H., C.T.R., C.F-B. and J-Y.T. Validation and testing: C.G.-L.-H., C.F-B., V.U., D.S, D.F-B., J.H. and A.I. Writing: C.G-L-H., C.F-B., V.U., A.M-B., J.-C.O.-M. and D.S. Review and editing: all authors. Supervision: A.M-B., J-C.O-M., K.W.E. and D.S. Peer review Peer review information Nature Communications thanks Julius Bamwenda and Marcus Chin for their contribution to the peer review of this work. A peer review file is available. Data availability The datasets used in this study are publicly available and are referenced throughout the manuscript. No new datasets were generated for this work. Source data are provided with this paper. Code availability The complete source code for the SAMJ platform, encompassing the library’s core Java implementation, its ImageJ/Fiji integration, and associated components, is accessible at https://github.com/segment-anything-models-java. For comprehensive documentation, usage instructions, and examples, please refer to the SAMJ-IJ README available at https://github.com/segment-anything-models-java/SAMJ-IJ/blob/main/README.md. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Daniel Sage, Email: daniel.sage@epfl.ch. Arrate Muñoz-Barrutia, Email: mamunozb@ing.uc3m.es. Supplementary information The online version contains supplementary material available at 10.1038/s41467-026-71752-x. References Associated Data Supplementary Materials Supplementary Information Reporting Summary Transparent Peer Review file Source Data Data Availability Statement The datasets used in this study are publicly available and are referenced throughout the manuscript. No new datasets were generated for this work. Source data are provided with this paper. The complete source code for the SAMJ platform, encompassing the library’s core Java implementation, its ImageJ/Fiji integration, and associated components, is accessible at https://github.com/segment-anything-models-java. For comprehensive documentation, usage instructions, and examples, please refer to the SAMJ-IJ README available at https://github.com/segment-anything-models-java/SAMJ-IJ/blob/main/README.md.