An intelligent point-of-care photoacoustic and ultrasound dual-modality imaging system for Rheumatoid arthritis in fingers: Technical development and initial clinical test☆☆☆ Authors: Xiaorui Peng, Kyoungmo Koo, Jianwei Qiu, Grigorios Karageorgos, Aaron Dentinger, Zhanpeng Xu, Siddarth Ragupathi, Zhaoyuan Yang, Soumya Ghose, Janggun Jo, Nada Abdulaziz, Guan Xu, Wei Zhao, David Chamberland, Mark Draelos, Girish Gandikota, David Mills, Xueding Wang Published: 2026 ABSTRACT AbstractAiming at an intelligent point-of-care imaging technology for rheumatology clinics, a fully automatic 3D photoacoustic (PA) and ultrasound (US) dual-modality system driven by a robot and powered by deep learning (DL)-based image processing was developed. Automated scanning of volumetric images from patient joints, plus DL-based tissue segmentation and quantification of imaging biomarkers, ensures that the measurements from this system are objective and reproducible. Clinical validation was conducted via a longitudinal study on 43 finger joints from patients affected by inflammatory arthritis. Using manual segmentations as the gold standard, our DL algorithm utilizing the 3D Deep Attentive Feature (DAF3D) model showed satisfactory performance in automatic segmentation of joint space and synovial region and achieved a Dice score of 0.77±0.03 and an IoU of 0.64±0.03. Based on the tissue segmentations facilitated by the DAF3D model, six volumetric imaging biomarkers reflecting the activity of arthritis and its change in response to the treatment were quantified, including hyperemia, blood oxygenation, US power Doppler, joint space echogenicity, joint space volume, and synovial volume. The imaging biomarkers quantified from DL-based segmentation and manual segmentation showed moderate to strong correlations (R: 0.41−0.86). Hyperemia quantified from PA imaging has the strongest association with the disease activity indicated by Clinical Assessment Questionnaire (CAQ) scores, with R2=0.41. Linear models combining the two biomarkers from PA imaging, the four biomarkers from US imaging, and all six imaging biomarkers rendered moderate to very strong associations with the disease activity scores, with R2 of 0.41, 0.26, and 0.52, respectively.Keywords: Ultrasound, Photoacoustics, Medical robotics, Deep learning, Clinical study Introduction Rheumatoid arthritis (RA), as a chronic autoimmune disorder affecting approximately 0.5%–1% of the global population [1], is the leading cause of disability in adults, and is associated with substantial activity limitation, work disability, reduced quality of life, and high health-care costs [2]. Early, rapid, and cost-effective diagnosis is essential to initiate timely intervention and achieve remission before irreversible joint damage occurs. However, current imaging modalities used to detect pathological changes are inherently limited in capturing early disease processes. X-ray-based scoring systems primarily identify structural alterations, such as bone erosion and joint space narrowing, which represent late-stage manifestations of disease. Despite providing high contrast for osseous structures, X-rays lack sensitivity to functional and soft-tissue changes [3], [4], particularly within synovial tissues affected by early inflammation. Magnetic resonance imaging (MRI) offers superior visualization of soft-tissue inflammation; however, its clinical utility is constrained by high cost, prolonged acquisition times, the requirement for intravenous contrast agents, and limited portability [5]. Rheumatology and radiology clinics would enormously benefit from better imaging technologies that are comparable to or better than MRI in sensitivity to soft-tissue inflammation but significantly reduced in costs, improved in availability and portability, and more patient and operator-friendly. With many advantages as an inexpensive, accessible, non-ionizing, non-invasive imaging modality and the ability to produce real-time images and document both bony erosions and enhanced blood flow associated with soft tissue inflammation, ultrasound (US) Doppler imaging has been employed as the primary modality for clinical diagnosis of inflammatory arthritis [6]. Although Doppler US can detect fast blood flow in larger vessels, it remains insensitive to the slow microvascular flow that predominates in early synovitis [7]. Recent clinical studies have highlighted the potential of photoacoustic (PA) imaging for assessing soft-tissue inflammation [8], [9], [10], [11], [12], [13]. By sensitively capturing hemodynamic and metabolic changes, PA imaging enables the mapping of key inflammatory biomarkers, including hyperemia and hypoxia. These features arise from increased metabolic demand coupled with insufficient oxygen delivery in inflamed tissues. PA imaging can provide additional valuable diagnostic information contributing to a more robust set of parameters for identification and characteristic quantification of inflammatory arthritis. The encouraging findings from the earlier pioneering clinical studies suggest that integration of PA imaging with established US modalities may enable low-cost, accessible point-of-care diagnostic devices, so that critical disease-modifying treatments can be deployed, positively altering the curve of joint deterioration and thus reducing joint-related pain and disability. With this goal in mind, we developed an intelligent three-dimensional (3D) PA-US dual-modality system powered by a robot for automatic image scanning. In addition, a deep learning (DL) algorithm was developed for automatic image processing and analysis. The highly automatic features of this system, including automated 3D image scanning and image processing, tissue segmentation, and quantification of biomarkers, removes potential operator dependence and renders imaging findings reliable and reproducible. Unlike current clinical US which needs to be operated by experienced radiologists or sonographers, such an intelligent imaging system can be used by anyone after simple training. In addition, unlike 2D US or PA imaging that may lead to inconsistent and inaccurate diagnoses given its susceptibility to scanning plane deviation, angular misalignment, and skills of the operator [14], 3D volumetric measures facilitated by this automated system could be superior in both accuracy and reproducibility. The feasibility and performance of this robot and DL powered intelligent PA-US dual-modality imaging system was evaluated via a prospective longitudinal clinical study on a total of 43 finger joints from a cohort of 14 arthritis patients recruited from the University of Michigan Rheumatology clinic. Facilitated by the high-quality 3D PA and US images, volumetric assessment of multiple functional and morphological imaging biomarkers in localized articular tissues reflecting the activity of arthritis were quantified, including hyperemia, blood oxygenation, US Doppler (i.e., blood flow), joint space echogenicity, synovial volume, and joint space volume. The imaging biomarkers quantified from DL-based tissue segmentation and processing were compared to those quantified from manual tissue segmentation conducted by board certified radiologists as the gold standard. With the longitudinal imaging measurements both before and after the initiation of the medical treatment, the association between each of these imaging biomarkers and the disease activity as indicated by the joint specific patient Clinical Assessment Questionnaire (CAQ) scores was studied via linear mixed model. This revealed which imaging biomarkers are primarily sensitive to the activity of arthritis and its change in response to the treatment. In addition, to evaluate whether multiple imaging biomarkers, facilitated by this multi-modal imaging system, can better reflect disease condition compared to those from a single imaging modality, we fit three linear mixed models combining different groups of imaging biomarkers, including the biomarkers from PA imaging, from US imaging, and from PA-US dual-modality imaging, and quantified and compared their predictive performance on disease activity as indicated by the joint specific CAQ scores. Building on prior developments [15], [16], [17], the present work makes the following two contributions: •An integrated, fully automated end-to-end pipeline. We integrate robotic scanning, automatic deep learning-based segmentation, quantitative biomarker extraction, and the relation of these biomarkers to clinical severity into a single unified system. This integration removes the operator dependence and manual analysis inherent to the separate components and represents a necessary step towards the clinical translation of PA imaging in rheumatology. An integrated, fully automated end-to-end pipeline. We integrate robotic scanning, automatic deep learning-based segmentation, quantitative biomarker extraction, and the relation of these biomarkers to clinical severity into a single unified system. This integration removes the operator dependence and manual analysis inherent to the separate components and represents a necessary step towards the clinical translation of PA imaging in rheumatology. •Prospective clinical evaluation in RA patients. We evaluate the integrated system in a prospective study of RA patients, providing initial clinical evidence that an automated robotic PA-US platform can be used end-to-end—from scanning to biomarker-based assessment—for the evaluation of arthritis in finger joints. Prospective clinical evaluation in RA patients. We evaluate the integrated system in a prospective study of RA patients, providing initial clinical evidence that an automated robotic PA-US platform can be used end-to-end—from scanning to biomarker-based assessment—for the evaluation of arthritis in finger joints. Methods Patient recruitment and study All imaging procedures were conducted in accordance with a protocol approved by the institutional review board of the University of Michigan Medical School (PI: Xueding Wang; protocol #: HUM00003693). Participants were recruited between June 2022 and December 2023 by certified rheumatologists at the University of Michigan Rheumatology Clinic. Women are more likely to develop arthritis than men [18]. All subjects involved in this study were women over 18 years old, with noticeable swelling, pain, and active synovitis on physical exam in at least one of their finger joints at the initial clinic visit. Joint specific disease activity was assessed by joint specific CAQ scores. The joint-specific CAQ score was modeled after the Patient Global Assessment (PGA), which is part of the Clinical Disease Activity Index (CDAI). PGA is one of the most widely used patient-reported outcomes (PROs) in RA practice and research, and is often assessed by questions with 0–10 or 0–100 responses to reflect patient’s perspective of disease activity [19], [20], [21]. In total, 19 subjects were recruited, and 14 subjects among them (median age, 58 ± 14 years) were able to finish the longitudinal study, including imaging of affected joints both before the initiation of the medical treatment and during each biweekly follow-up visit after the treatment, for assessment of their response to the treatment. The basic clinical information for these 14 patients is shown in Table 1. A flow diagram of the patient study is illustrated in Fig. 1. Table 1Demographic characteristics of patients involved.Table 1DemographicsAll participantsEarly-Stage RALate-Stage RAP-Valueacharacteristicswith RA (n = 14)(<5y) (n = 4)(≥5y) (n = 10)Age (y), median (range)58 (28–69)62 (47–69)56 (28–68).26Sex, N F14410 M000Height (cm), median (range)166.4 (147.3–177.8)172.7 (172.7–177.8)161.3 (147.3–172.7).01Weight (kg), median (range)82.8 (55.0–146.7)77.6 (56.7–112.0)83.3 (55.0–146.7).41ap-value from t-tests that compares Early-Stage RA participants and Late-Stage RA participants. Demographic characteristics of patients involved. p-value from t-tests that compares Early-Stage RA participants and Late-Stage RA participants. Patient study flow diagram. To recruit each patient, a full explanation of the study was provided, and written informed consent from the patient was received. The patient’s identification information was removed to achieve compliance with HIPAA. After recruitment of each patient, the patient’s condition of RA was evaluated by the board-certified rheumatologists following the American College of Rheumatology (ACR) criteria. After confirming the patient’s condition, including identifying the symptomatic finger joints with obvious inflammation as the targeted joints for imaging assessment, the patient entered the longitudinal study. Prior to the initiation of medication-based treatment, baseline 3D US imaging and PA imaging of specifically targeted inflamed joints, as diagnosed by the rheumatologists, were conducted. After the medication-based treatment started, each patient had biweekly follow-up examinations. We repeated 3D US and PA imaging during each visit for the same identified joints. During each visit, a CAQ was conducted by using the questionnaire. Patients’ joint-specific CAQ scores were used for association with joint-specific biomarkers. We studied a total of 43 joints from the 14 patients who finished the longitudinal study. Each joint underwent 3 to 4 scans at different time points of biweekly visits starting from pre-initiation of medication. During each visit, 3D US imaging and PA imaging of targeted joints were acquired. During the studies of the first 8 patients, the dual-wavelength PA imaging at 750 nm and 850 nm was not ready for use, so PA imaging was conducted at a single wavelength of 750 nm only. Hence, for each of the 18 joints from the first 8 patients, we longitudinally assessed 5 imaging biomarkers, including blood flow, hyperemia, joint space echogenicity, synovial volume, and joint space volume. For each of the 25 joints from the remaining 6 patients, dual-wavelength PA imaging at 750 nm and 850 nm was conducted during each patient visit, facilitating an additional imaging biomarker of blood oxygenation, increasing the total number of quantified imaging biomarkers to 6. Three-dimensional (3D) automated PA-US dual-modality imaging system The schematic of the automated PA-US dual-modality imaging system for human inflammatory arthritis is shown in Fig. 2(a) with the customized probe holder shown in Fig. 2(b). Scanning was performed using a 6-DOF UR3 robotic arm (Universal Robots), with a repeatability of ±0.1 mm and a reach of 500 mm. The US/PA probe was mounted on the end-effector via a custom holder. A Nd:YAG OPO tunable laser (Phocus MOBILE, OPOTEK Inc.) was used as the PA excitation source, coupled with a custom-designed fiber bundle. The laser operated at a repetition rate of 10 Hz with a pulse width of 5 ns. At 750 nm, the laser light fluence on the skin was approximately 20 mJ cm-2, which was below the ANSI maximum permissible exposure limit of 25.2 mJ cm-2 for skin at this wavelength (ANSI Z136.1). US and PA image streaming were performed using a GEHC Vivid™ E95 cardiovascular system equipped with an L8–18i-D high-frequency linear array probe. For PA acquisition, the laser trigger output was used to synchronize image capture. The system interface allowed seamless switching between B-mode US, Doppler US, and PA imaging modes. More technical details of this imaging system are described in our previous publications [12], [15], [22]. A custom-designed hand rest was used to minimize patient hand motion during scanning. Finger joint locations were identified using an RGB-D camera mounted on the system, with detection performed via OpenPose (Carnegie Mellon University) [23], [24]. The robot executed a linear trajectory around each joint, acquiring multiple 2D scans for subsequent 3D reconstruction. The scanning parameters are adjustable, including the span, the probe speed, and the number of 2D images scanned. 3D automated PA-US dual-modality imaging system. (a) Schematic of the imaging system including a GE HealthCare Vivid™ E95 US system, an Nd:YAG pumped OPO laser, a control desktop computer, a Universal Robot UR3 robotic arm holding an imaging probe, a volunteer’s hand placed in water tank, and an overhead camera for hand joint detection. The scanning path for 3D imaging of a target finger joint is marked. (b) Customized probe holder with an attached L8-18i-D linear array US probe and an optic fiber bundle. (c) Photograph of the entire imaging system and a volunteer. The platform operates in two complementary acquisition modes: (i) B-mode US paired with dual-wavelength PA imaging, and (ii) B-mode US paired with Doppler US. PA excitation was performed at two wavelengths, 750 nm and 850 nm, selected to enable spectral discrimination between oxygenated and deoxygenated hemoglobin. The three modalities were continuously streamed at their native frame rates (B-mode US: 23 fps; PA: 65 fps; Doppler US: 12 fps). Tap water was used as the acoustic coupling medium; hot and cold water was mixed to reach near body temperature (34–39 °C), and the water was then degassed to remove air bubbles. For B-mode US + PA acquisition, the robot moved continuously along a linear trajectory while image pairs were captured throughout the movement. As a representative setting, 135 image pairs were acquired over a 2 cm span in 7.4 s. For B-mode US + Doppler US acquisition, a move–stop mode was used instead: the robot moved to each predefined position along the linear trajectory, paused for a second (which is adjustable), and acquired an image pair before proceeding to the next location. This approach is essential because Doppler US is highly sensitive to motion, and continuous robotic movement introduces significant flow artifacts. Upon the robot’s arrival at each acquisition position, the most recently streamed image pair (B-mode US + Doppler US) was captured. As a representative setting, 100 image pairs were acquired over a 2 cm span in 180 s. B-mode US and Doppler US imaging were performed at transducer frequencies of 7.0 MHz and 7.1 MHz, respectively. A Linux-based control computer coordinated robotic motion, image acquisition, user interface, path planning, and joint localization throughout the scan. The acquired images were reconstructed into a 3D volume upon completion of the scan. A photograph of the whole imaging system and a volunteer whose hand is being imaged is shown in Fig. 2(c). Analysis Image reconstruction and quantification of imaging biomarkers Real-time PA imaging was enabled by configuring the VE95’s parallel delay-and-sum beamforming algorithm for one-way US propagation of the PA signal [12]. The PA signals, after being processed with a bandpass filter centered at 12.5 MHz and a bandwidth of 3.8 MHz, were used to create a PA image using the power of the beamformed signals following a single laser pulse illumination. The live streaming of US and PA B-mode combined image, similar to US Doppler imaging, was displayed on the VE 95 US machine and was used for biomarker quantification. With the 3D US B-mode image of each joint, the joint space and the synovial region were segmented both manually and by using DL-based automated segmentation. For manually segmented synovial and joint space masks, our radiologists only sparsely annotated 5 to 7 frames (i.e., 2D slides from the 3D scan) per joint volume due to large dataset size. To extend these sparse annotations to the entire joint volume, we applied the 1st mask (from the 1st annotated frame) to the 1st frame through the frame just before the 2nd annotated frame. This process was repeated for subsequent annotations, applying the nth mask to the remaining frames after the nth annotated frame, which allowed us to have a mask for every frame for the entire joint. For DL-based segmented masks, we applied the dense masks onto the US B-mode, US Doppler, and PA images in one-to-one correspondence. With the joint space and synovial volume segmented, we then quantified six morphological and functional imaging biomarkers from the acquired 3D US B-mode, US power Doppler, and PA images, including US power Doppler, joint space echogenicity, synovial volume, joint space volume, hyperemia, and blood oxygenation. In this study, the US power Doppler as a functional biomarker of arthritis was quantified by averaging the US power Doppler signal intensities within the joint space-masked 3D US Doppler image. Clinically, the enhanced blood flow within the synovium indicates increased vascularity associated with inflammation which can be visualized by US color or power Doppler US. The joint space echogenicity was quantified by the average gray pixel intensity level in the joint space normalized by the average gray pixel intensity level in the subcutaneous fat in the 3D US B-mode image. A more “hypoechoic” (darker) appearance on US image often indicates increased synovial thickening with or without fluid, which can be a sign of active inflammation in a joint. The synovial volume was quantified by counting the total number of voxels in the segmented synovial mask using the 3D US B-mode image. The enlarged size of a joint’s synovium reflects the synovial fluid buildup and is an indicator of synovitis which can be assessed by US. The joint space volume was quantified by counting the total number of voxels in the segmented joint space mask in the 3D US B-mode image. Joint space thickening in the context of arthritis usually refers to a visible sign on X-ray or US where the tissues around the joint space are swelling and thickening due to inflammation. The above four US-based imaging biomarkers are frequently used in rheumatology and radiology clinics to assess the activity of arthritis [25], [26]. Facilitated by PA imaging, two additional biomarkers associated with soft tissue inflammation were assessed, including hyperemia and blood oxygenation. Hyperemia reflecting the increased blood volume in the joint space due to inflammation was quantified by averaging the PA signal intensity in the voxels within the joint space-masked 3D PA image acquired at 750 nm. Blood oxygenation in the joint space was quantified from the PA images at 750 nm and 850 nm by considering the distinctive optical absorption spectra of oxygenated hemoglobin and deoxygenated hemoglobin [27]. As demonstrated in our previous studies and those by other groups [8], [9], [11], [16], [28], the blood oxygenation levels in inflammatory arthritic tissues are relatively low, which is due to the increased metabolic demand and the relatively inadequate oxygen delivery to inflammatory synovial tissue [29]. Three-dimensional automated imaging results Three-dimensional (3D) images of each patient’s finger joints that were affected by arthritis were acquired by the robot-powered automated PA-US dual-modality imaging system. Fig. 3 shows an example imaging result from the right 3rd metacarpophalangeal (MCP) joint of a patient affected by inflammatory arthritis. The high-quality volumetric B-mode US image in Fig. 3(a) enabled clear identifications of the major tissue morphology. The bone structure of the MCP joint, which includes proximal phalange and metacarpal bone, and other tissues including skin, tendon, and synovial fluid, are shown in the center sagittal-sectional US image in Fig. 3(b). In the B-mode US image, the dorsal skin surface of the hand joint and soft tissues, such as subcutaneous fat, tendon, and synovial area, are presented in Fig. 3(c). The green area in Fig. 3(c) marks the area of the subcutaneous fat. The blue area in Fig. 3(c) and the brown area in Fig. 3(d) show the manual segmentation masks of the synovial area and the joint space delineated by a board-certified radiologist. Fig. 3(e)–(g) show pseudo-color functional images in the segmented joint space superimposed on the gray-scale US B-mode image. These functional images include the US power Doppler image presenting the blood flow, the PA image at 750 nm presenting the hyperemia, and the blood oxygenation image which was calculated from the two PA images at 750 nm and 850 nm. US and PA functional images along with additional sagittal sections and cross sections in this imaged finger joint can be found in video 1. Example imaging result of a right 3rd metacarpophalangeal (MCP) joint of a patient affected by inflammatory arthritis. (a) A series of 2D sagittal-sectional slices in the 3D volumetric B-mode US image. (b) 2D US B-mode image along the center sagittal section of the finger with major tissue morphology annotated. (c) The 2D US B-mode image in (b) with the segmented subcutaneous fat area marked in green and the synovial area marked in blue. (d) with the segmented joint space marked in brown. (e) with superimposed US power Doppler signals in pseudo color presenting blood flow. (f) with superimposed PA image acquired at 750 nm in pseudo color presenting hyperemia. (g) with superimposed blood oxygenation in pseudo color calculated from the PA images acquired at 750 nm and 850 nm. Fig. 4 shows the longitudinal 3D volumetric imaging results from a patient’s right 3rd MCP joint acquired during the first, second, and third scans. The first scan was before the initiation of the medical treatment, while the second scan was two weeks after the first scan and the initiation of the medical treatment, and the third scan was two weeks after the second scan. In the 3D US images in Fig. 4(a), morphological characteristics such as dorsal skin surface, proximal phalange, and metacarpal bone can be seen clearly. Fig. 4(b)–(f) show the 3D US images with overlaid pseudo-color synovial area masks, joint space masks, US power Doppler images, hyperemia images, and blood oxygenation images. In the right column of Fig. 4, the six subfigures display the six quantified imaging biomarkers as a function of time, showing possible change of each imaging biomarker in response to the medical treatment. The joint specific CAQ scores of this patient at the three scanning times are also shown. Video 2 shows the 3D volumetric images with superimposed pseudo-color US power Doppler images, hyperemia images, and blood oxygenation images from this longitudinal study along different view angles. 3D volumetric US and PA images of a patient’s right 3rd MCP joint acquired during the first, second, and third scans. Each scan was two weeks apart. The first scan was before the initiation of the medical treatment, while the second and the third scan were after the initiation of the medical treatment. (a) Grayscale 3D US B-mode images. (b) The 3D US B-mode images in (a) with the annotated synovial areas in blue. (c) with the annotated joint space in red. (d) with superimposed US power Doppler image presenting the blood flow in the joint space. (e) with superimposed PA image at 750 nm presenting hyperemia in the joint space. (f) with superimposed PA blood oxygenation image in the joint space. The six subfigures in the right column show the six imaging biomarkers quantified from the imaging results acquired at the three different scanning times, each normalized by the result at the first scan. In each subfigure, the blue curve shows the quantified imaging biomarkers, and the red curve shows the patients’ joint-specific CAQ scores at the three scanning times. Modeling and results Development and validation of the deep learning model for automated segmentation In order to achieve automated segmentation of joint space and synovial region for quantitative assessment of the US and PA imaging biomarkers, we have developed a deep-learning (DL)-based segmentation pipeline that utilizes the 3D Deep Attentive Feature (DAF3D)-based model to address the challenges in finger joint tissue segmentation in 3D US images [31]. A total of N = 43 3D US images of patients’ finger joints were acquired, including metacarpophalangeal (MCP) joints, proximal interphalangeal (PIP) joints, and distal interphalangeal (DIP) joints. To train the DL-based segmentation, board-certified radiologists sparsely annotated the joint space region in the 3D US images by manually delineating the dark gray/black (hypoechoic) area above the finger bones across multiple 2D slices using ITK-SNAP software [32]. Similarly, the synovial area was also sparsely annotated. Corresponding 3D dense annotations for both the joint space and synovial region were generated using our previous method, a DL-based automated sparse-to-dense annotation model [17]. To artificially enhance the size and diversity of training dataset, various offline data augmentation techniques from TorchIO [33], a Python library designed for medical imaging augmentation, were applied on the US volumes and the corresponding dense 3-D annotations. These techniques included horizontal/vertical flip, random rotation, random brightness/contrast adjustment, random Gaussian blur, and random noise. The architecture of the DAF3D network is illustrated in Fig. 5. The DAF3D network architecture employs a 3D ResNeXt backbone [34] enhanced with dilated convolutions to capture multi-scale contextual information, which is crucial for accurately delineating the complex anatomical structures of human finger joints. The key innovation of DAF3D lies in its attention mechanism, which refines multi-layer features by computing attention weights at each layer, thereby enhancing feature maps at different resolutions. For segmenting the joint space and synovial region, the DAF3D model processes the 3D US data to extract detailed features from the input images. The network’s ability to integrate multi-level features through its attention modules allows for the precise identification of boundaries and textures specific to these regions. This capability is particularly beneficial in 3D US, where image quality can be variable, and the structures of interest are often small and complex. The manually segmented masks generated by the radiologists were used directly as the ground-truth labels for both training and validation. The DAF3D model was trained using a hybrid loss function that combines Dice loss and cross-entropy loss. Dice loss is particularly effective for addressing class imbalance by maximizing the overlap between the predicted segmentation and the ground truth. Cross-entropy loss complements this by penalizing misclassifications, thereby improving the overall accuracy of the segmentation. The training process was optimized using the Adam optimizer, with the learning rate set to 10−4. This setup ensures a stable and gradual convergence of the model, leveraging the adaptive learning rate capabilities of the Adam optimizer. Network diagram of DAF3D segmentation model. FPN: feature pyramid network; SLF: single-layer features; MLF: multi-layer features; ASPP: atrous spatial pyramid pooling [30]. DL-based joint tissue segmentation results To evaluate the performance of the DAF3D model in terms of its accuracy in segmenting human finger joint tissues including the joint space and the synovial region using the manual segmentation results as the ground truth, we employed a 6-fold cross-validation approach to compare the segmentation results from the DL-based method and the manual method. This approach ensures robust assessment and validation by dividing the dataset into six subsets. In each fold, five subsets are used for training, while the remaining subset is used for validation. This process is repeated six times, with each subset serving as the validation set once. Fig. 6 illustrates an example of automated DAF3D segmentation for both the joint space and the synovial region. The 2D slice images, along with their corresponding ground truth (GT) labels and segmentation masks, are extracted from the 3D volume. The performance metrics used to evaluate the DAF3D segmentation model include the Dice score and Intersection over Union (IoU), computed across 6-fold cross-validation. The summarized performance metrics are presented in Table 2. Overall, the DAF3D model achieved a Dice score of 0.769 ± 0.026 and an IoU of 0.639 ± 0.031. Both evaluation metrics confirmed the good performance of the automated DAF3D segmentation [35], [36]. Additionally, this level of agreement is comparable to the inter-observer Dice coefficients reported among expert readers performing manual segmentation of soft-tissue lesions on similar imaging modalities. Webb et al. reported pairwise inter-expert Dice values of 0.79–0.81 for breast lesion segmentation in ultrasound among three trained radiologists and sonographers [37], and Covert et al. reported a mean inter-observer Dice of 0.79 among three board-certified radiologists contouring hepatocellular carcinoma lesions on contrast-enhanced MRI [38]. The performance of the proposed model therefore falls within the range of agreement observed among expert readers themselves, supporting its suitability for clinical application. Example result of DL-based automated segmentation of the joint space and synovial region in the 5th MCP joint of a patient overlaid on the input B-mode US image. Column 1 shows 5 sparse 2D slices in the input 3D volumetric B-mode US image. Columns 2 and 3 display the manual segmentation and the DL-based automated segmentation of the joint space, respectively. Columns 4 and 5 display the manual segmentation and the DL-based segmentation of the synovial region, respectively. Table 2Performance statistics (Dice Score and IoU) of 6-Fold cross-validation for DAF3D segmentation.Table 2MetricF1F2F3F4F5F6MeanStdDice Score0.8010.7290.7920.7730.7650.7520.7690.026IoU0.6880.5900.6630.6460.6290.6210.6390.031 Performance statistics (Dice Score and IoU) of 6-Fold cross-validation for DAF3D segmentation. Comparing imaging biomarkers based on DL automated segmentation vs. Manual segmentation We studied a total of 43 joints from the 14 patients, each scanned at 3–4 different time points, which led to a total of 155 joint imaging results. For each of the 155 joint imaging results, the joint space and the synovial region were segmented both manually by radiologists and automatically by DL-based segmentation. With the joint space and the synovial region segmented from the 3D US image of the joint, either manually or automatically, we quantified the six imaging biomarkers including hyperemia, US power Doppler, joint space echogenicity, joint space volume, synovial volume, and blood oxygenation. Then we compared the quantified imaging biomarkers based on manual segmentation to those based on DL segmentation. The total data points for hyperemia, US power Doppler, joint space echogenicity, joint space volume, and synovial volume are 155 (i.e., N = 155). The total data points for blood oxygenation are 85 (i.e., N = 85). This is because dual-wavelength PA imaging of blood oxygenation was only conducted on the 25 joints of the 6 patients. The six panels in Fig. 7 show the six quantified imaging biomarkers, where in each panel the results based on manual segmentation are compared to those based on DL automated segmentation. To assess the agreement between the imaging biomarkers based on manual segmentation performed by radiologists vs. those based on DL segmentation, we calculated the Pearson’s correlation coefficient R for each of the six sets of the quantified imaging biomarkers. The Pearson’s correlation coefficient R ranging from 0.41 to 0.86 exhibits moderate to very strong agreement between the two groups of results [39], indicating that, using the results from the manual segmentations as the gold standard, our DL algorithm utilizing the DAF3D model showed satisfactory performance in assessing the PA and US imaging biomarkers reflecting the activity of arthritis. As illustrated in Fig. 7, functional biomarkers (PA hyperemia, blood oxygenation, and US Doppler flow) show lower correlation compared to structural biomarkers (joint space volume, synovium volume, echogenicity). Expected reason of such results is as follows: The manual ground truth was annotated on every fifth slice, whereas the DL model segmented all slices; this difference in annotation density contributes to the deviation observed for the functional biomarkers. In addition, structural biomarkers represent aggregate measures over the entire segmented region, so segmentation errors have limited impact on the final value. In contrast, functional biomarkers are often concentrated near the tissue boundary and margin regions, precisely where segmentation errors are most likely to occur. This spatial alignment between functional signal concentration and segmentation error regions results in greater sensitivity of functional biomarkers to boundary misalignment. While our segmentation performance (Dice = 0.769) is comparable to expert inter-observer agreement, this level of boundary variation has a more pronounced effect on functional signal quantification. Comparison of the imaging biomarkers, quantified based on manual segmentation vs. DL-based segmentation of the 3D US images of the patients’ finger joints. For each of the six sets of the imaging biomarkers, Pearson’s correlation coefficient R was calculated to assess the agreement between the results from manual segmentation vs. DL segmentation. Red dashed lines show the regression lines. Studying the associations between imaging biomarkers and patients’ disease activity scores To evaluate how each imaging biomarker can reflect the arthritis disease activity using the patients’ joint specific CAQ scores as the gold standard, we applied linear mixed models (LMMs) [40] to fit each of the six imaging biomarkers to the patients’ joint specific CAQ scores. The choice of LMM over standard linear regression model was justified because each joint was scanned multiple times at different time points, resulting in non-independent data points. LMMs are well-suited for handling such repeated measures, as they account for the association within the data by including random effects that capture the variability between joints or patients. The patients’ joint specific CAQ scores were obtained for each imaged joint at every visit, and all the values were normalized by the baseline joint specific CAQ scores recorded during the first visit. Each of the imaging biomarkers obtained at every visit was also normalized by the baseline measurement from the first scan during the first visit. For each of the LMM models of the six imaging biomarkers, we reported the regression coefficient (i.e., beta), proportion of variance explained (R2), and p-value, as shown in Table 3. All statistical analyses were performed using MATLAB and R Studio. Table 3Association between each individual imaging biomarker and patients’ joint specific CAQ scores.Table 3Hyper-BloodUSJSJSSyn.emiaOxy.Dop.Echo.Vol.Vol.(N= 155)(N= 85)(N= 155)(N= 155)(N= 155)(N= 155)Beta0.42−0.240.15−0.07−0.05−0.07R20.410.300.050.010.0050.01P-val**********0.370.17Regression results of the linear mixed model (LMM) using each individual imaging biomarker quantified based on DL segmentation to predict patients’ joint specific CAQ scores with individual-level random effect to account for correlation among repeated observations. N represents the total number of observations. Blood Oxy.: Blood Oxygenation; US Dop.: US Power Doppler; JS Echo.: Joint Space Echogenicity; JS Vol.: Joint Space Volume; Syn. Vol.: Synovial Volume ; ***: p<0.001, **: p<0.01, *: p<0.05. Association between each individual imaging biomarker and patients’ joint specific CAQ scores. Regression results of the linear mixed model (LMM) using each individual imaging biomarker quantified based on DL segmentation to predict patients’ joint specific CAQ scores with individual-level random effect to account for correlation among repeated observations. N represents the total number of observations. Blood Oxy.: Blood Oxygenation; US Dop.: US Power Doppler; JS Echo.: Joint Space Echogenicity; JS Vol.: Joint Space Volume; Syn. Vol.: Synovial Volume ; ***: p<0.001, **: p<0.01, *: p<0.05. Among the six imaging biomarkers, hyperemia and blood oxygenation quantified from PA imaging exhibit the strongest association with disease activity, with R2=0.41 and R2=0.30, respectively, and both p<0.001 [41]. Both the US Doppler and the joint space echogenicity, as two biomarkers quantified from US imaging, also show moderate association with the disease activity assessed by the patients’ joint specific CAQ scores, with R2=0.05, p<0.01 and R2=0.01, p<0.01, respectively. The other two biomarkers quantified from US imaging, including the joint space volume and the synovial volume, do not show statistically significant associations with the disease activity. To examine the hypotheses that a combination of multiple imaging biomarkers can assess arthritis patient’s condition more accurately than a single imaging biomarker alone, we applied three additional linear mixed models (LMMs), which combine the two biomarkers from PA imaging (i.e., hyperemia and blood oxygenation), the four biomarkers from US imaging (i.e., US Doppler, joint space echogenicity, joint space volume, and synovial volume), and all the six biomarkers from the PA-US dual-modality imaging, respectively, to fit the patients’ joint specific CAQ scores. Table 4 Part A shows the results from these three LMMs which utilized the 85 data points (i.e., N = 85) containing blood oxygenation measurements. These three LMMs combining the multiple imaging biomarkers rendered moderate to very strong associations with the patients’ joint specific CAQ scores, with R2 of 0.41, 0.26, and 0.52, respectively. The LMM utilizing the two biomarkers from PA imaging achieved a higher association with the patients’ disease activity scores than the LMM utilizing the four biomarkers from US imaging, while the LMM utilizing all the six biomarkers facilitated by the PA-US dual-modality imaging achieved the highest association of R2=0.52 with the patients’ joint specific CAQ scores [42]. Table 4Association between groups of imaging biomarkers and patients’ joint specific CAQ scores across different cohorts.Table 4CohortMetricPA imagingUS imagingPA-US imagingHyp.Oxy.Dop.Echo.Vol.Syn.Hyp.Oxy.Dop.Echo.Vol.Syn.Part A (N=85)Beta0.46−0.080.17−0.16−0.02−0.010.43−0.060.08−0.17−0.090.01R20.410.260.52P-value**********0.720.19*********0.060.62Part B (N=155)Beta0.42–0.14−0.13−0.06−0.040.39–0.07−0.12−0.06−0.03R20.410.110.46P-value***–*****0.620.79***–0.06**0.460.22Note: Part A includes blood oxygenation (Oxy.) as a PA biomarker, whereas Part B utilizes the full cohort without blood oxygenation data.Hyp: Hyperemia, Oxy: Blood Oxygenation, Dop: US Doppler, Echo: JS Echogenicity, Vol: JS Volume, Syn: Synovial Volume.P-value: ***: p<0.001, **: p<0.01, *: p<0.05. Association between groups of imaging biomarkers and patients’ joint specific CAQ scores across different cohorts. Note: Part A includes blood oxygenation (Oxy.) as a PA biomarker, whereas Part B utilizes the full cohort without blood oxygenation data. Hyp: Hyperemia, Oxy: Blood Oxygenation, Dop: US Doppler, Echo: JS Echogenicity, Vol: JS Volume, Syn: Synovial Volume. P-value: ***: p<0.001, **: p<0.01, *: p<0.05. The LMMs combining multiple imaging biomarkers were applied again by utilizing all the 155 data points. Since part of the 155 data points have no blood oxygenation measurements, the first LMM utilized only the hyperemia from PA imaging, the second LMM utilized the four biomarkers (i.e., US Doppler, joint space echogenicity, joint space volume, and synovial volume) from US imaging, and the third LMM utilized five biomarkers, including hyperemia, US Doppler, joint space echogenicity, joint space volume, and synovial volume, from the PA-US dual-modality imaging. Per the results shown in Table 4 Part B, these LMMs combining multiple imaging biomarkers based on a larger data set also rendered moderate to very strong associations with the patients’ joint specific CAQ scores, with R2 of 0.41, 0.11, and 0.46, respectively. The LMM combining the five biomarkers facilitated by the PA-US dual-modality imaging achieved the highest association of R2=0.46 with the patients’ joint specific CAQ scores. In addition, ANOVA tests were conducted to compare the LMMs based on different combinations of imaging biomarkers. For the 85 data points, the ANOVA tests indicated that the LMM combining all the six biomarkers facilitated by PA-US dual-modality imaging achieved a significantly better fit to the patients’ joint specific CAQ scores than either the LMM combining the two biomarkers from PA imaging (i.e., hyperemia and blood oxygenation) (chi-squared = 20.77, p<0.001) or the LMM combining the four biomarkers from US imaging (i.e., US Doppler, joint space echogenicity, joint space volume, and synovial volume) (chi-squared = 41.16, p<0.0001). For the 155 data points, the ANOVA tests indicated that the LMM combining all the five biomarkers facilitated by PA-US dual-modality imaging (i.e., hyperemia, US Doppler, joint space echogenicity, joint space volume, and synovial volume) achieved a significantly better fit to the patients’ joint specific CAQ scores than either the LMM based on hyperemia from PA imaging (chi-squared = 16.28, p<0.001) or the LMM combining the four biomarkers from US imaging (chi-squared = 36.67, p<0.0001). The outcomes indicated again that the LMM combining the biomarkers facilitated by the PA-US dual-modality imaging achieved better predictive accuracy of the patients’ disease conditions than the LMM based on the biomarkers from a single imaging modality. Discussion and conclusion Several previous clinical studies have demonstrated that, with high sensitivity in identifying the functional changes in soft tissues affected by inflammation, the emerging photoacoustic (PA) imaging combined with established ultrasound (US) imaging may allow early detection of inflammatory arthritis so critical disease modifying anti-rheumatic drugs can be started thus preventing further joint damage and minimizing negative downstream disease consequences. To promote the translation of PA imaging to rheumatology clinics, in this work we developed an intelligent 3D PA-US dual-modality system powered by a robot for automatic scanning of both PA and US images and a DL algorithm for automatic image segmentation and biomarker analysis. This imaging system enables quantitative and volumetric assessment of multiple morphological and functional biomarkers in human peripheral joints reflecting the activity of arthritis. To assess the clinical viability of this intelligent PA-US imaging system as well as its performance in assessing the arthritis condition and responses to medical treatment, we conducted a prospective study involving 43 individual joints from 14 patients. A limitation of the current dataset is that the patient cohort consisted exclusively of female participants (Table 1). This reflects the known epidemiology of rheumatoid arthritis, which predominantly affects women, as well as the composition of the available patient cohort during the study period. Nevertheless, we expect the proposed biomarkers to remain applicable to male patients: the structures imaged by our platform—the synovium, joint space, bone surface, and periarticular vasculature—are anatomically common to both sexes and exhibit comparable appearances in ultrasound, and the PA- and Doppler-derived biomarkers reflect the underlying inflammatory pathophysiology of the joint rather than sex-specific characteristics. Future studies incorporating male participants and a broader demographic range will be conducted to fully establish the generalizability of the proposed platform. This patient study demonstrated that this imaging system powered by a robotic arm could automatically acquire 3D volumetric B-mode US with either US Doppler image or PA image with high image quality, good safety, and satisfactory repeatability. The DL algorithm utilizing the DAF3D model was able to segment joint space and synovial region in the human finger joint automatically and achieved a Dice score of 0.77±0.03 and an IoU of 0.64±0.03 when comparing to the manual segmentations made by board-certified radiologists as the gold standard. Due to its automatic features and 3D volumetric measurements, this system could robustly repeat imaging scans of the same joints during follow-up patient visits, facilitating longitudinal patient assessment with satisfactory reproducibility. Although only patients with noticeable symptoms were recruited for the current study, our results suggest the potential for earlier detection. Our previous longitudinal study [15] demonstrated that PA- and Doppler-derived biomarkers tracked the progression of joint inflammation from its early course. Because these biomarkers are sensitive to the vascular activity associated with active inflammation, PA and Doppler US allow earlier detection of arthritis than conventional modalities such as X-ray, which is sensitive only to late-stage structural bone damage such as bone erosion and joint space narrowing. From a practical standpoint, the imaging workflow is designed to be efficient and to require minimal operator involvement. While the scans in this study used a dense sampling of images for data collection purposes, a routine clinical scan requires far fewer images. Given an azimuthal resolution of approximately 4.1 mm for the US probe, the Nyquist sampling theorem indicates that an inter-image spacing of 2 mm (approximately half the azimuthal resolution) is sufficient to capture the available spatial information with minimal loss; for a 2 cm scan range, this corresponds to only 11 images per joint. Under this setting, the B-mode US + PA mode completes acquisition in under 10 s per joint, and the B-mode US + Doppler US mode, which requires a move–stop acquisition, completes in under 20 s per joint. Operator interaction is limited to initiating the system: finger localization is performed automatically by an RGB-D camera, scanning is carried out by the robotic arm, and biomarker measurement and assessment are subsequently handled by the deep learning model. This high degree of automation reduces dependence on operator skill and supports consistent, reproducible acquisition. The main barrier to routine clinical deployment is regulatory: while the current system is validated as a research platform, formal regulatory approval would be required before adoption in routine clinical practice. Within the DL-based automatically segmented regions of interest in joint, including joint space and synovial region, we were able to quantify six morphological and physiological imaging biomarkers. The LMMs applied to fit individual imaging biomarkers to the disease activity scores indicated that hyperemia, blood oxygenation, US Doppler, and joint space echogenicity showed statistically significant association with the patients’ joint specific CAQ scores. Among the six imaging biomarkers, hyperemia measured by PA imaging achieved the strongest association (R2=0.41) with the disease activity, followed by blood oxygenation (R2=0.30) and US Doppler (R2=0.05). The substantially stronger association of PA-derived hyperemia compared with US Doppler is consistent with the differing sensitivities of the two modalities: Doppler US primarily detects fast flow in larger vessels, whereas PA imaging is sensitive to the slow-flow microvasculature characteristic of active inflammation. As a result, PA imaging more directly captures the angiogenic microvascular changes that underlie disease activity, while the two modalities provide complementary information. The outcomes from the LMMs utilizing the biomarkers from PA imaging (R2=0.41), the biomarkers from US imaging (R2=0.26), and all the biomarkers (R2=0.52), respectively, as well as additional ANOVA tests, indicated that the LMM combining all the biomarkers facilitated by the PA-US dual-modality imaging could achieve better predictive accuracy of the patients’ disease condition. In particular, our analysis suggests that hyperemia, US Doppler, joint space echogenicity, and blood oxygenation are the most informative biomarkers and provide complementary information on disease activity. In our previous studies [8], [9], we have demonstrated that hypoxia in synovial tissue is a hallmark of arthritis. The findings from this study confirmed this. As shown in Table 3, blood oxygenation, when analyzed alone, was significantly associated with the patient’s condition. However, when blood oxygenation and hyperemia were fitted simultaneously in the model (Table 4), the contribution from hyperemia to the LMM is significant (p<0.001) while the contribution of blood oxygenation was comparatively weaker (p<0.05). This may be due to the fact that, despite the association between blood oxygenation and the disease activity, the contribution from the blood oxygenation in the LMMs could overlap with other biomarkers in the models, e.g., hyperemia. This lack of improvement may be due to the moderate correlation between blood oxygenation and hyperemia. The calculated correlation coefficient of R=0.62 (N = 85) between them suggested that these two biomarkers share overlapping information. In conclusion, this initial study on arthritis patients demonstrates the clinical feasibility of the newly developed intelligent PA-US dual-modality automated imaging system, for quantitative assessment of arthritis disease progression and response to medical treatment. This system is powered by a robot and a DL algorithm. It can perform automated scanning of 3D structural and functional PA and US images, and automated tissue segmentation to facilitate quantitative analyses of multiple imaging biomarkers reflecting the activity of arthritis. The outcome from the initial patient study demonstrated strong associations between the imaging biomarkers and the patients’ joint specific CAQ scores, which demonstrated potential value of this system to radiology and rheumatology clinics for timely diagnosis and treatment monitoring of arthritis with improved accuracy and reduced cost. CRediT authorship contribution statement Xiaorui Peng: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Kyoungmo Koo: Writing – review & editing, Visualization, Supervision, Investigation. Jianwei Qiu: Software, Methodology, Investigation, Formal analysis. Grigorios Karageorgos: Software, Methodology, Investigation, Formal analysis. Aaron Dentinger: Software, Methodology, Investigation, Formal analysis. Zhanpeng Xu: Software, Investigation, Formal analysis, Data curation. Siddarth Ragupathi: Software, Methodology, Investigation, Formal analysis. Zhaoyuan Yang: Software, Methodology, Investigation, Formal analysis, Data curation. Soumya Ghose: Software, Methodology, Investigation, Formal analysis. Janggun Jo: Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Nada Abdulaziz: Methodology, Investigation, Formal analysis, Conceptualization. Guan Xu: Investigation, Formal analysis, Data curation, Conceptualization. Wei Zhao: Investigation, Formal analysis, Data curation, Conceptualization. David Chamberland: Investigation, Formal analysis, Data curation, Conceptualization. Mark Draelos: Methodology, Investigation, Formal analysis, Conceptualization. Girish Gandikota: Validation, Supervision, Project administration, Investigation. David Mills: Validation, Supervision, Resources, Project administration, Investigation. Xueding Wang: Writing – review & editing, Writing – original draft, Validation, Supervision, Resources, Project administration, Methodology, Investigation, Funding acquisition. Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Xueding Wang reports financial support was provided by National Institutes of Health. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This research is supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institute of Health under grant number R01AR060350. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Biographies Xiaorui Peng received her B.S. degree in Electrical Engineering from Beijing Institute of Technology in 2019 and her master’s degree in Biomedical Engineering from Duke University in 2021. Now, Xiaorui joined the Optical Imaging Laboratory and is pursuing her Ph.D. degree in Biomedical Engineering at the University of Michigan. Her current research interests focus on potential clinical applications of photoacoutics. Kyoungmo Koo received his BS degree in electrical and computer engineering from Seoul National University, Seoul, Korea, in 2023, and his MS degree in electrical and computer engineering from the University of Michigan, Ann Arbor, Michigan, USA, in 2025. He joined the Image-Guided Medical Robotics Laboratory at the University of Michigan in 2023, where he is pursuing his Ph.D. in biomedical engineering. His research interests include medical imaging and robotics. Jianwei Qiu received the B.E. degree in electrical engineering (2015) and the M.S. degree in computer engineering (2016) from Stony Brook University, Stony Brook, NY, USA. He is currently a Senior Scientist – AI and Computer Vision at GE HealthCare Technology and Innovation Center, Niskayuna, NY, USA. His research interests include computer vision and deep learning for medical imaging, with emphasis on image reconstruction, quantitative analysis, and multimodal imaging, including CT, ultrasound, fMRI, and digital histopathology. Grigorios Marios Karageorgos received the Diploma degree in electrical and computer engineering from the National Technical University of Athens (NTUA), Athens, Greece, in 2016, major in electronic engineering, and the Ph.D. degree in biomedical engineering from Columbia University, New York, NY, USA. He conducted his Diploma thesis with the BIOmedical Simulations and IMaging Laboratory (BIOSIM), NTUA, where he worked on the development of a self-powered electromagnetic sensor for cardiovascular system monitoring. He joined the Ultrasound Imaging Elasticity Laboratory (UEIL), Columbia University. His research interests include the development of ultrasound imaging techniques to estimate the mechanical properties of large arteries, and methods for vascular disease diagnosis and monitoring. Aaron Dentinger received his B.S. degree in engineering physics in 1992 and his M. S. and Ph.D. degrees in electrical engineering in 1994 and 2006, respectively, from Rensselaer Polytechnic Institute, Troy, NY., Since 1995, he has worked as an Electrical Engineer at GE Global Research in Niskayuna, NY and is currently a member of the Ultrasound and Biomedical Laboratory. Prior to joining GE, he was employed at Reveo, Inc. His current research interests are in ultrasound signal and image processing for vascular, cardiac, and physiological measurements. Zhanpeng Xu is a Ph.D. candidate from the Department of Biomedical Engineering, University of Michigan, Ann Arbor. He received his B.S. degree from Sichuan University and M.S. degree from Zhejiang University in China. His research interests focus on photoacoustic measurement and imaging, photoacoustic microscopy, hyperspectral imaging et al. He has published more than 10 peer-reviewed journal papers, and served as the independent reviewer for Optics Express, Optics Letters, Biomedical Optics Express, Progress in Electromagnetics Research, Applied Optics, and Plos One. Siddarth Ragupathi, MD, is a Radiologist at Montefiore Medical Center in New York. He completed his Radiology residency in India and a Cardiothoracic Imaging fellowship at Montefiore. Currently, he is pursuing a Nuclear Medicine residency, with a focus on thoracic and cardiovascular imaging. Zhaoyuan Yang received the B.S. and M.S. degrees in electrical and computer engineering from The Ohio State University, Columbus, OH, USA, in 2017 and 2018, respectively.,Since 2018, he has been working as a Research Engineer at GE Research, Niskayuna, NY, USA. He is currently a member of the Computer Vision Laboratory. His current research interests are in robustness and uncertainty quantification for computer vision models. Soumya Ghose holds a Master’s in Computer Application degree from India and a Master’s in Computer Vision and Robotics from Heriot Watt University, UK. He received his Ph.D. degree in Computer Engineering – Medical Image Analysis from University of Dijon, France in 2012. He worked as a post-doctoral fellow in Australia’s Commonwealth Scientific Research Organization on MR only radiation therapy planning for prostate cancer between 2012–2015. Soumya joined Case Western Reserve University in 2015 as a senior Research Associate working with Prof. Anant Madabhushi on prostate cancer risk stratification from multimodal medical images. He joined GE Research as a Lead Scientist in AI and Computer Vision group in 2018. He is currently a Principal Scientist in GE HealthCare working on business and NIH funded projects. Soumya’s research is focused towards developing novel machine learning, deep learning, segmentation, and registration methods applied to detection, prognosis prediction and treatment response of diseases from multimodal medical images, clinicopathological data and omics. Janggun Jo received his B.S. and M.S. degree in Electronic Engineering from KoreaTech in Korea. He received his Ph.D. degree in Bioengineering at the University of Kansas. And Dr. Jo finished his postdoc fellowship at the University of Michigan. Currently he is a Research Investigator in the Department of Biomedical Engineering at the University of Michigan. His research interests include Photoacoustic imaging modality including Optical and Ultrasound imaging as well as high intensity focused ultrasound treatment. Nada Abdulaziz received her medical degree from the University of Khartoum Faculty of Medicine. Her followship in Rheumatology was at the University of Michigan, where she pursued her career as Clinical assistant professor in Rheumatology Division. As a faculty member, she has been awarded W.J. (Joe) McCune Award for Teaching Excellence. She is currently a member in the fellowship Competency Committees that oversee the academic progression of rheumatology fellows across all phases of their training. She has an interest in inflammatory myositis diseases and is part of a multidisciplinary team in myositis program. Guan Xu received his Ph.D. and postdoctoral training in optical and ultrasound imaging in biomedicine. He received a predoctoral award from Congressionally Directed Medical Research Programs, a postdoctoral fellowship from American Heart Association, a Career Development Award from American Gastroenterology Association, a Senior Research Award from Crohn’s and Colitis Foundation and an R37 MERIT award from National Cancer Institute. Wei Zhao is a Research Assistant Professor at Survey Research Center and an Assistant Professor at the Department of Epidemiology at the University of Michigan. As an interdisciplinary researcher, she has expertise in statistics, genetics, neuroscience, psychology, and survey methodology. She utilizes her analytical skills to integrate complex multi-omics data and environmental factors to understand etiology of a variety of complex diseases, including obesity, cardiovascular disease, diabetes, kidney disease, osteoarthritis, menopause symptoms, depression, and Alzheimer’s disease and related dementia. She is committed to translating and promoting innovative statistical methodologies in biomedical and social research. David Chamberland received his B.S. in Electrical Engineering from the University of Portland and his M.D. from the Oregon Health Sciences University. He joined the University of Michigan in 2004 and has been working on medical applications of photoacoustic technology for over 15 years. His training in electrical engineering and also a long history of working as a clinical rheumatologist with board certifications in internal medicine and rheumatology enable him to help with translation of applications into the clinical space. Mark Draelos, MD, Ph.D., is a surgically-trained physician and engineer who develops novel applications of medical robotics and imaging to improve patient care. Currently, he is an assistant professor of Robotics and Ophthalmology at the University of Michigan, where he directs the Image-Guided Medical Robotics Laboratory. He is a recipient of the NIH Director’s New Innovator Award and the NIH Pathway to Independence Award. Girish Gandikota obtained board certifications in surgery (F.R.C.S.) and Radiology (F.R.C.R.) in the U.K. He did his fellowship in Musculoskeletal Radiology at McMaster University, Hamilton, Ontario, Canada. After the fellowship, he joined the University of Michigan as a lecturer. He has been promoted successfully to the professor and has received American board certification (A.B.R.). He has also obtained ultrasound R.M.S.K. certification from APCA/ARDMS. Winner of the excellence in teaching awards, Dr. Gandikota has been an invited speaker, delivering more than 300 invited lectures and national presentations. He has over 75 scientific papers/exhibits in national conferences and over 68 peer-reviewed publications to date. His main research interests include M.S.K. Ultrasound, Sports Medicine, and Arthritis. Presently has over 15 years of M.S.K. ultrasound experience. Conducts C.M.E. courses on M.S.K. radiology and ultrasound in U.S.A. and India. He is a co-investigator in a NIH RO1 Grant ($3,545,507) on Arthritis. He has mentored many residents, fellows, technologists and medical students. Dr. Gandikota recently joined us from the University of Michigan where he was Professor of Radiology. In addition to teaching and research, Dr. Gandikota had multiple administrative roles at the University of Michigan and at the national level. He was the Chair of Clinical Competency Committee, Radiology Residency Program, and in charge of M.S.K. resident training, curriculum development, and evaluation. He was also the Director, Domino Farms Radiology. He is a member of national professional societies like the Radiological Society of North America (R.S.N.A.), American Roentgen Ray Society (A.R.R.S.), Society of Skeletal Radiology (S.S.R.), and American Institute of Ultrasound in Medicine (A.I.U.M.), where he regularly lectures. He was the Chair of the Assessment Oversight Team (A.O.T.) RMSK-APCA appointed for the musculoskeletal physician examination. He is a reviewer for multiple esteemed radiology and rheumatology journals, including Arthritis and rheumatology. David Mills is a Principal Scientist in the Ultrasound and Bioinstrumentation lab at GE Research in Niskayuna, NY. He received the B.S. degree in engineering (summa cum laude) from LeTourneau University, Longview, TX, USA, in 1994, and the Ph.D. degree in biomedical engineering from Duke University, Durham, NC, USA, in 2000. He joined GE Research in 2000 where he is involved in the research and development of medical ultrasound transducers and advanced ultrasound applications. He has led multiyear research programs on ultrasound applications and drug delivery using microbubbles, including scan guidance and image analysis technology. He has authored many publications and conference presentations in these areas and holds more than 25 patents related to medical ultrasound. His current research interests include the design of medical ultrasound transducers of both piezoelectric and cMUT-based transducers and new clinical applications for ultrasound that are enabled by real-time 3-D ultrasound imaging, visualization algorithms, and customized user interfaces to meet specific user needs. Xueding Wang is a Jonathan Rubin Collegiate Professor of the Department of Biomedical Engineering and the Professor of the Department of Radiology at the University of Michigan School of Medicine. Before working as an independent principal investigator, Dr. Wang received his Ph.D. at Texas A&M University and Postdoctoral training at the University of Michigan. Dr. Wang has extensive experience in development of medical imaging and treatment technologies, especially those involving light and ultrasound. Sponsored by NIH, NSF, DoD and other funding agencies, his research has led to over 160+peer-reviewed journal papers. Dr. Wang was the recipient of the Sontag Foundation Fellow of the Arthritis National Research Foundation in 2005, the Distinguished Investigator Award of the Academy of Radiology Research in 2013, and was elected to the fellow of AIMBE in 2020 and the fellow of SPIE in 2021. He is also sitting on the editorial boards of scientific journals including Photoacoustics, Ultrasonic Imaging, Journal of Biomedical Optics, Medical Physics, and Scientific Reports. Footnotes This article is part of a Special issue entitled: ‘Surgical, Interventional Guidance and Robotics’ published in Photoacoustics. Given his/her/their role as Associate Editor, Xueding Wang had no involvement in the peer-review of this article and has no access to information regarding its peer-review. Full responsibility for the editorial process for this article was delegated to another journal editor. Supplementary material related to this article can be found online at https://doi.org/10.1016/j.pacs.2026.100849. Contributor Information Xiaorui Peng, Email: xiaoruip@umich.edu. Kyoungmo Koo, Email: kmkoo@umich.edu. Jianwei Qiu, Email: jianwei.qiu@gehealthcare.com. Grigorios Karageorgos, Email: gregorykarageorge@gmail.com. Aaron Dentinger, Email: dentinge@gehealthcare.com. Zhanpeng Xu, Email: zhanpeng@umich.edu. Siddarth Ragupathi, Email: drsiddarthragupathi@gmail.com. Zhaoyuan Yang, Email: zhyyang1015@gmail.com. Soumya Ghose, Email: soumya.ghose@gehealthcare.com. Janggun Jo, Email: janggunj@umich.edu. Nada Abdulaziz, Email: nabdulaz@umich.edu. Guan Xu, Email: guanx@umich.edu. Wei Zhao, Email: zhaowei@umich.edu. David Chamberland, Email: davicham@umich.edu. Mark Draelos, Email: mdraelos@umich.edu. Girish Gandikota, Email: ggirish@umich.edu. David Mills, Email: millsda@gehealthcare.com. Xueding Wang, Email: xdwang@umich.edu. Supplementary data The following is the Supplementary material related to this article. Video S1 . Video S2 . Data availability Data will be made available on request. References Associated Data Supplementary Materials Video S1 . Video S2 . Data Availability Statement Data will be made available on request.