Tractography‐Based Automated Identification of Retinogeniculate Visual Pathway With Novel Microstructure‐Informed Supervised Contrastive Learning Authors: Sipei Li, Wei Zhang, Shun Yao, Jianzhong He, Jingjing Gao, Tengfei Xue, Guoqiang Xie, Yuqian Chen, Erickson F Torio, Yuanjing Feng, Dhiego C A Bastos, Yogesh Rathi, Nikos Makris, Ron Kikinis, Wenya Linda Bi, Alexandra J Golby, Lauren J O'Donnell, Fan Zhang Published: 2024 ABSTRACT ABSTRACTThe retinogeniculate visual pathway (RGVP) is responsible for carrying visual information from the retina to the lateral geniculate nucleus. Identification and visualization of the RGVP are important in studying the anatomy of the visual system and can inform the treatment of related brain diseases. Diffusion MRI (dMRI) tractography is an advanced imaging method that uniquely enables in vivo mapping of the 3D trajectory of the RGVP. Currently, identification of the RGVP from tractography data relies on expert (manual) selection of tractography streamlines, which is time‐consuming, has high clinical and expert labor costs, and is affected by inter‐observer variability. In this paper, we present a novel deep learning framework, DeepRGVP, to enable fast and accurate identification of the RGVP from dMRI tractography data. We design a novel microstructure‐informed supervised contrastive learning method that leverages both streamline label and tissue microstructure information to determine positive and negative pairs. We propose a new streamline‐level data augmentation method to address highly imbalanced training data, where the number of RGVP streamlines is much lower than that of non‐RGVP streamlines. In the experiments, we perform comparisons with several state‐of‐the‐art deep learning methods that were designed for tractography parcellation. Furthermore, to assess the generalizability of the proposed RGVP method, we apply our method to dMRI tractography data from neurosurgical patients with pituitary tumors. In comparison with the state‐of‐the‐art methods, we show superior RGVP identification results using DeepRGVP with significantly higher accuracy and F1 scores. In the patient data experiment, we show DeepRGVP can successfully identify RGVPs despite the effect of lesions affecting the RGVPs. Overall, our study shows the high potential of using deep learning to automatically identify the RGVP.Keywords: cranial nerves, diffusion MRI, retinogeniculate visual pathway, tractography Introduction The retinogeniculate visual pathway (RGVP) is responsible for carrying visual information from the retina to the lateral geniculate nucleus (LGN) (Chacko 1948). It consists of three anatomical segments, including the optic nerve (ON), the optic chiasm (OC), and the optic tract (OT) (He et al. 2021). The RGVP is affected in many diseases, including pituitary tumors (Laws, Trautmann, and Hollenhorst 1977), optic neuritis (Beck et al. 2003), and many others (Attyé et al. 2018; Pisa et al. 2022). In particular, pituitary tumors notably influence the RGVP, because of their close proximity to the ONs and OC, and can lead to neurological complications such as vision impairment, headaches, and nausea (Gittleman et al. 2014; Ho, Huang, and Ho 2015). Identification and visualization of the RGVP are important in studying the anatomy of the visual system (Wichmann and Müller‐Forell 2004) and may inform treatment of brain diseases such as lesions intrinsic or extrinsic to the pathway (Ma et al. 2016). Diffusion MRI (dMRI) tractography is an advanced imaging method (Basser et al. 2000) that uniquely enables in vivo reconstruction of the 3D streamline trajectory of the RGVP in a noninvasive way. Unlike the widely used T1‐weighted (T1w) and T2‐weighted (T2w) MRI or other imaging techniques, e.g. steady‐state free precession (SSFP), that can only indicate whether the RGVP is present at a particular location, dMRI tractography can assess the 3D fiber pathway of the RGVP and the continuity of its subdivisions. In the context of surgical planning, dMRI tractography allows neurosurgeons to visualize the relationship of the RGVP and its subdivisions to the surrounding structures. In this way, it allows neurosurgeons to evaluate the proximity of the planned surgical path. As shown in Figure 1, dMRI tractography can enable the visualization of the 3D pathway of the RGVP and its relation to the tumor. Due to this unique property, dMRI tractography has been used successfully in presurgical cranial nerve (CN) mapping (Roundy, Delashaw, and Cetas 2012) for making optimal surgical plans, improving the possibility of preserving CNs and contributing to improved quality of life (Churi, Gupta, and Misra 2019; Song et al. 2016). Mapping of the retinogeniculate visual pathway (RGVP) in a retrospective pituitary adenoma patient using dMRI tractography. The 3D fiber pathway of the RGVP is reconstructed consisting of three anatomical segments, i.e., the optic nerves, the optic chiasm, and the optic tracts. The tumor (in green) causes displacement of RGVP fibers near the optic chiasm region. Many tractography‐based studies have shown successful mapping of the RGVP for clinical and research purposes (Altıntaş et al. 2017; Ather et al. 2019; Hofer, Karaus, and Frahm 2010; Panesar et al. 2019; Yoshino et al. 2016). Currently, identification of the RGVP from tractography mainly relies on expert selection, where streamlines are selected based on whether they end in and/or pass through regions of interest (ROIs) drawn by experts (Oishi et al. 2008). ROI‐based RGVP selection, however, is time‐consuming, is inefficient with high clinical and expert labor costs, and is also affected by inter‐observer variability depending on the experience of experts. Recently, there have been studies for automated identification of the RGVP as well as other CNs (Zeng et al. 2023; Zhang, Xie, et al. 2020) using traditional machine‐learning techniques. In recent years, deep‐learning‐based methods have been demonstrated to be highly successful in tractography parcellation in white matter tracts (Chen et al. 2023; Wasserthal, Neher, and Maier‐Hein 2018; Xu et al. 2019; Zhang, Karayumak‐Cetin, et al. 2020). To date, there have been no deep‐learning‐based methods for RGVP identification using tractography in surgical patient data with lesions affecting the RGVP. Recent deep learning advances enable fast, accurate RGVP identification, but two key challenges remain. First, current deep‐learning‐based tractography parcellation methods mostly use streamline geometric features extracted based on the streamline point spatial coordinates (Xu et al. 2019; Gupta et al. 2017; Lam et al. 2018; Xue et al. 2023). Spatial information helps distinguish the large‐size white matter tracts, but small geometric differences between RGVP and nearby non‐RGVP streamlines, or false positives from partial voluming and low SNR, remain challenging (He et al. 2021; Hofer, Karaus, and Frahm 2010; Carrozzi et al. 2023). In tractography data, various features can be computed, including not only streamline geometric features but also diffusion microstructure measures like fractional anisotropy (FA) (Zhang, Daducci, et al. 2022). Adding microstructure features will improve RGVP identification, as similar anatomical structures should share both geometric trajectories and FA values. A second challenge is that there can be a highly imbalanced streamline sample distribution between RGVP and non‐RGVP streamlines (our data shows a ratio of 1:8 between RGVP and non‐RGVP streamlines; see Section II‐B). Imbalanced training data is a well‐known challenge in deep learning and limits network generalization for small‐size sample categories (Johnson and Khoshgoftaar 2019). Data augmentation (DA) is one effective solution to resolve this challenge and has been used in several tractography studies (Gupta et al. 2017; Benou and Raviv 2019). Our recent study proposed a subject‐level tractography DA strategy that generates multiple new datasets by downsampling each subject's tractography data (Zhang, Xue, et al. 2022). Inspired by this work, we propose a new streamline‐level DA method that uses random subsampling to increase the sample size of RGVP streamlines. In light of the above, this study presents a novel deep learning framework, namely DeepRGVP, for automated identification of the RGVP using dMRI tractography. In this study, our contributions are as follows. First, we present what we believe is the first deep‐learning approach that enables fast and accurate RGVP identification. Second, we design a microstructure‐informed supervised contrastive learning (SCL) method that leverages both streamline label (RGVP and non‐RGVP) and tissue microstructure (FA) information. Third, we design a simple and successful streamline‐level DA method to address the imbalanced training data. Forth, we demonstrate successful generalization of DeepRGVP to clinically typical dMRI data acquired from pituitary tumor patients (PTPs), even though DeepRGVP is trained using high‐quality dMRI tractography data acquired in healthy adults. When applying our trained model to the new data, neither FA nor streamline label is required. In this case, our method is robust to focal disruption or alteration of the FA due to the pathology. Fourth, considering that dMRI acquisition quality and the existence of brain lesions may affect the tracking of the RGVP, we design an ensemble RGVP tractography method that enables highly sensitive tracking of RGVP for improved identification of the RGVP. This investigation extends our previous conference publication (Li et al. 2023) to include a retrospective analysis of an additional dataset from PTPs with the newly designed ensemble RGVP tractography method, as well as an improved method description with in‐depth technical details and an extensive evaluation on data from additional sources. The rest of this paper is organized as follows. Section 2 describes the datasets used, the proposed framework, and the model training and testing. Section 3 presents the experimental setup and results on high‐quality dMRI data from healthy adults and typical clinical dMRI data from neurosurgical patients, followed by the discussion in Section 4. Finally, the conclusions are given in Section 5. Methodology Our overall goal is to identify streamlines that belong to the RGVP from input tractography data generated in the skull base region. Figure 2 gives a visualization of example input tractography data, including RGVP streamlines selected using expert‐drawn ROIs, and all other unselected non‐RGVP streamlines resulting from false positive tracking or belonging to other structures. The RGVP and non‐RGVP streamlines are visually highly similar in terms of their geometric trajectory; however, the non‐RGVP streamlines fail to satisfy the strict anatomical ROI selection criteria. We can observe different FA values in local streamline regions of RGVP and non‐RGVP streamlines (indicated using red arrows). This motivates the design of our microstructure‐informed SCL (MicroSCL) method (Section 2.2.2). In addition, we can observe a highly imbalanced streamline distribution where the RGVP streamlines are fewer than the non‐RGVP streamlines due to the use of a tractography seeding mask that is larger than the region that the RGVP passes through. This motivates the design of our streamline‐level data augmentation (StreamDA) method to resolve any potential training biases due to the imbalanced input data (Section 2.2.4). Example RGVP tractography data: (a) tractography seeding mask and ROIs used for generating ground truth expert‐selected data, (b) RGVP including four subdivisions and non‐RGVP streamlines of an example HCP subject, and (c) RGVP and non‐RGVP streamlines of an example pituitary tumor patient. The red arrows in (b, c) show local streamline regions of RGVP and non‐RGVP streamlines with different FA values. Datasets The study uses two different dMRI datasets. The first dataset is from the Human Connectome Project (HCP) database (Van Essen et al. 2013) acquired from young healthy adults. The HCP dataset is used for model training and testing. The second dataset is acquired from patients with a diagnosis of pituitary tumor. The patient data is used for testing to demonstrate how DeepRGVP generalizes to a different source and a different population. HCP Dataset We use a total of 100 dMRI scans (the 100 unrelated subjects) from the HCP database (Van Essen et al. 2013). We analyze this cohort because the dMRI data for RGVP reconstruction has been well curated in our previous study (He et al. 2021), including data quality check and expert ROI drawing. The HCP data was acquired with a high‐quality image acquisition protocol and preprocessed for data artifact correction (Glasser et al. 2013). The acquisition parameters were TE/TR = 89.5/5520 ms, and voxel size = 1.25 mm × 1.25 mm × 1.25 mm. In this study, we use the single‐shell b = 1000 s/mm2 data because it has been shown to enable highly effective RGVP tracking (He et al. 2021) and it is commonly seen in clinically typical data. We perform a visual check of all dMRI data and exclude 38 subjects that had dMRI data with incomplete RGVP coverage due to face removal for data anonymization, as described in our previous work (He et al. 2021). Thus, tractography data from 62 subjects are used in this study. Tractography is performed within a seeding mask (Figure 2a) using two‐tensor unscented Kalman filter (UKF) tractography (Reddy and Rathi 2016) via SlicerDMRI (Zhang, Noh, et al. 2020). UKF fits tensor models to the diffusion data while tracking fibers, in a recursive estimation fashion (the current tracking estimate is guided by the previous one) using a Kalman filter. We choose UKF because it can accurately track the RGVP (He et al. 2021) and other CNs (Zhang, Xie, et al. 2020; Xie et al. 2020) despite the differences in dMRI acquisition parameters across datasets. In addtion, the UKF method allows the estimation of streamline‐specific microstructure measures, including the measure of interest, FA. In our study, we used the FA derived from the tensor that represented the principle fiber orientation at each streamline point location. Details of the RGVP tractography procedure is described in (He et al. 2021). In brief, a streamline length threshold of 80 mm is applied to eliminate any effect from streamlines too short to form the RGVP (Schiefer and Hart 2007). In UKF tractography, the major parameters include seedingFA, stoppingFA, Qm, and Ql. Tractography is seeded in all voxels within the RGVP seeding mask, where FA is greater than the seedingFA threshold value. Tracking stops in voxels where the FA value falls below the stoppingFA threshold value. During the tracking, the UKF tractography methods use the Qm parameter to control process noise for angles/direction, and use the Ql parameter to control process noise for eigenvalues. Following our previous work that investigates how these major parameters affect the RGVP tracking (He et al. 2021), we set seedingFA = 0.02, stoppingFA = 0.01, Qm = 0.001, Ql = 50, which was shown to generate the best RGVP reconstruction result. We leverage ground truth RGVP streamlines selected using ROIs drawn by an expert (practicing neurosurgeon G.X.) and multi‐rater validated in our previous study (He et al. 2021). These ROIs include the ON, OC, and OT (Figure 2a). Four fiber selections were performed, within each three ROIs were used, including one of the ON ROIs (left or right), one of the OT ROIs (left or right), and the OC ROI. The results from the four fiber selections were appended to obtain the entire RGVP. For model training, we also leverage the non‐RGVP streamlines (Figure 2b) not selected by these ROIs. On average, there are 150 RGVP and 1200 non‐RGVP streamlines per subject (thus, a ratio of 1:8 of RVGP to non‐RGVP samples for training). PTP Dataset We use dMRI from 10 PTPs (see Table 1) from the First Affiliated Hospital of Sun Yat‐sen University, China. All research procedures were approved by the Ethics Committee of Clinical Research and Animal Trials of the First Affiliated Hospital of Sun Yat‐sen University (Study Number [2022]057). The acquisition parameters are: TE = 97 ms, TR = 2700 ms, and voxel size = 1.5 mm × 1.5 mm × 1.8 mm, with a field of view (FOV) of the skull base covering the RGVP. A total of 69 volumes were acquired for each subject, including 5 baseline images and 64 diffusion‐weighted images at b = 1000 s/mm3. The data were pre‐processed using our well‐established dMRI data processing pipeline for motion correction (publicly available at: https://github.com/pnlbwh/pnlutil). TABLE 1Demographics, diagnosis, and lesion size of the PTP dataset.IDAge (Y)GenderPrimaryDiagnosisLesion diameter0141MaleYesGH‐PA8 mm0256FemaleYesNFPA17 mm0344MaleYesGH‐PA24 mm0440FemaleYesNFPA12 mm0540MaleYesGH‐PA9 mm0634FemaleYesProlactinomas7 mm0750MaleRecurrentGH‐PA4 mm0837MaleYesGH‐PA17 mm0936FemaleYesNFPA17 mm1054FemaleRecurrentGH‐PA15 mmAbbreviations: GH‐PA: Growth hormone‐secreting pituitary adenoma; NFPA: nonfunctioning pituitary adenoma; Prolactinomas: Prolactin‐secreting adenoma. Demographics, diagnosis, and lesion size of the PTP dataset. Abbreviations: GH‐PA: Growth hormone‐secreting pituitary adenoma; NFPA: nonfunctioning pituitary adenoma; Prolactinomas: Prolactin‐secreting adenoma. Tractography of the RGVP in the PTP data is performed using a newly designed ensemble UKF fiber tracking strategy. Unlike the high‐quality HCP data, the PTP data was acquired with a protocol geared toward clinical applications with lower spatial and angular resolutions. RGVP fiber tracking is challenging in relatively low‐quality data and is also impacted by the lesion affecting the RGVP. To increase the RGVP fiber tracking performance, we perform fiber tracking at various sets of tractography parameters and ensemble the tracked RGVP streamlines. This is inspired by the fact that tract‐specific tracking parameters are needed for improved reconstruction of local fiber structures (Takemura et al. 2016; Tax et al. 2014; Wasserthal et al. 2019). In related work, ensemble fiber tracking has been used for effective RGVP reconstruction and evaluation of the decussation of optic nerves within the optic chiasm (Puzniak et al. 2019). In our study, we perform RGVP fiber tracking with varying combinations of Qm and Ql, as the tracking of specific RGVP segments is notably influenced by the settings of these parameters. Specifically, we compute 28 sets of RGVP tractography, with fiber tracking varying Qm across the range of 0.001 to 0.004 with a step size of 0.001 and Ql across the range of 50–350 with a step size of 50 (thus 4 Qm values and 7 Ql values). In our previous study that investigated the effects of tractography parameters on RGVP reconstruction, we showed that high seeding and stopping thresholds resulted in decreased RGVP detection rate (He et al. 2021). Therefore, in the present study, we set seedingFA (= 0.02) and stoppingFA (= 0.01) to be relatively low to enable highly sensitive tracking of the RGVP, as shown in (He et al. 2021). Then, all 28 sets of RGVP tractography results are assembled together as input to the proposed network. To assess the RGVP identification results obtained with DeepRGVP, we perform RGVP streamline selection using expert‐drawn ROIs, which are used as ground truth RGVP results. Unlike the HCP data where five ROIs (bilateral ONs, bilateral OTs, and the OC) are used, the OC ROI cannot be visually identified in the PTP dMRI data (see Figure S1). Therefore, we draw only the ON and OT ROIs for the initial ROI selection, followed by the interactive selection of non‐RGVP streamlines, where the expert (S.Y., a practicing neurosurgeon) interactively draws ROIs to exclude streamlines not belonging to the RGVP. MicroSCL for RGVP Identification The overall framework of DeepRGVP includes the following major components. The first is a PointNet subnetwork that takes the feature representation of input tractography streamlines as point clouds and learns a global feature representation for each streamline (Section 2.2.1). The second component is a contrastive learning subnetwork that leverages both label information and tissue microstructure information to improve global features to be more discriminative between different classes (Section 2.2.2). The third component is a streamline classification subnetwork that classifies the input tractography streamlines into RGVP and non‐RGVP based on the global features (Section 2.2.3). The fourth component is a streamline‐level data augmentation process that increases the size of the training data to address the imbalanced training sample problem (Section 2.2.4). Streamline Global Feature Learning via PointNet The global feature representation is computed based on PointNet (Charles et al. 2017), which is widely used for point cloud classification and segmentation (Guo et al. 2020). In computer vision, a point cloud‐based network processes 3D data by analyzing collections of points in space, representing an object's shape and structure. It helps with tasks like object recognition, 3D modeling, and navigation in fields like robotics and virtual reality. In our work, coordinates of streamline points can be the input for PointNet networks, as used in (Chen et al. 2023; Xue et al. 2023; Astolfi et al. 2020) for tractography parcellation and filtering. An advantage of point cloud networks is that the global features generated are invariant to the order of points along a streamline, allowing the same global feature representation for forward and reverse point orders. The classic PointNet architecture includes a shared multi‐layer perceptron (MLP), a symmetric aggregation function, and fully connected (FC) layers, as well as data‐dependent transformation nets. In DeepRGVP, we make two modifications to the PointNet architecture by (1) removing the transformation nets to preserve spatial position of streamlines in the RGVP tractography space (as suggested in our previous tractography parcellation work (Chen et al. 2023; Xue et al. 2023)) and (2) adding additional MLP layers with sizes 256 and 512 between the layers of size 128 and 1024 (for a total of five layers of sizes 64, 128, 256, 512, and 1024) for better model training with the augmented data (see Section 2.2.4). The detailed network architecture for learning the global feature is shown in Figure 3a. The input of the encoder is RAS (Right, Anterior, Superior) point coordinates of a streamline; therefore, the dimension of the input is n × 3. We set n = 60, which is higher than n = 15 used in our previous work (Xue et al. 2023) for the representation of the superficial white matter streamlines that generally have shorter lengths than the RGVP streamlines. Then, each point input is encoded by the five shared MLP layers, where each layer has batch normalization and rectified linear units (ReLU) activation, generating an output with n × 1024 dimensions. Following that, the global feature g of dimension 1 × 1024 is formed through max‐pooling. Overview of the DeepRGVP network, including the MicroSCL network and the downstream classifier for streamline classification. Microstructure‐Informed Supervised Contrastive Learning SCL (Khosla et al. 2020) extends self‐SCL (Chen et al. 2020) to a fully supervised mode by proposing a supervised contrastive loss, which aims to pull global features with the same class label (positive pairs) closer in the latent space and push apart global features with different class labels (negative pairs). SCL is a method used to train machine learning models by teaching them to group similar items together and separate different ones. In this approach, the model learns from labeled data, meaning it knows which items are similar or belong to the same category. By comparing items in this way, the model becomes better at recognizing patterns and distinguishing between classes, leading to more accurate predictions in tasks like image recognition or classification. SCL has been shown to successfully improve the performance of supervised learning in computer vision (Kopuklu et al. 2021; Schiffer et al. 2021; Zhong et al. 2021) and natural language processing (Gunel et al. 2020; Huang et al. 2021) tasks, as well as in tractography parcellation tasks (Xue et al. 2023). In DeepRGVP, we extend the SupWMA method (Xue et al. 2023) by designing a microSCL method that leverages both streamline label (RGVP and non‐RGVP) and tissue microstructure (FA) information to determine positive and negative pairs, rather than only relying on the label information as traditionally used in SCL. In this way, the learned global feature can better differentiate streamlines with similar trajectories but from different classes. In detail, in addition to the traditional usage of label information for positive and negative sample pair determination, we include additional information about tissue microstructure to improve pair determination. To achieve this, we compute the absolute difference of mean streamline FA (the average of the FA values at each streamline point) between each streamline pair. We constrain positive pairs to be streamlines from the same class that satisfy the following: where TFA is a threshold on the allowable FA difference between streamlines in a positive pair. Overall, the supervised contrastive loss LMicroSCL used in our study is: where i is a streamline in a training batch I; Mi is the set of streamlines with the same class label as streamline i; Ni is the set of streamlines that satisfies the ΔFA condition in Equation (1); Pi is the intersection of Mi and Ni; Ai is the set that includes all streamlines except for streamline i in batch I; zi, zp, and za are contrastive features of streamlines i, p∈Pi and a∈Ai, respectively; τ (temperature) is a hyperparameter for optimization set to be 0.1 as suggested in (Chen et al. 2020). RGVP Streamline Classification After obtaining the global feature for each streamline from MicroSCL, the streamline classification subnetwork predicts streamline class according to the global feature generated (Figure 3a). A subnetwork including 3 FC layers with sizes of 512, 256, and 2 (number of classes) and a cross‐entropy loss is used. Streamline‐Level DA DA is a technique commonly used in deep learning to increase the diversity and quantity of training data by applying various transformations to the existing dataset (Shorten and Khoshgoftaar 2019). This helps improve the generalization and robustness of machine learning models, particularly in scenarios where the original dataset is limited. Developing DA methods to increase sample size is a known challenge in dMRI‐related research (Barile et al. 2021). In our work, to curate a training dataset with a balanced sample distribution and an increased training sample size for improved model training, we propose the following StreamDA method. Each streamline consists of a sequence of points estimated by a tractography algorithm. For input to the network, we represent each streamline using P points sampled along the streamline. For DA, we generate additional samples from each streamline by repeating the streamline point subsampling process multiple times, such that each time a different point subset with the same starting and ending points is generated (as demonstrated in Figure 4). We note that StreamDA is different from commonly used DA strategies, such as data repetition and adding noise, where additional samples are synthetically generated. Since each point subset contains unique information, StreamDA can help the algorithm to learn the features of the RGVP and better differentiate RGVP from non‐RGVP streamlines. In our study, given the original 1:8 ratio of RGVP to non‐RGVP streamlines, we perform the StreamDA process 8 times for each RGVP streamline. Graphic illustration of the proposed StreamDA method. RGVP Identification in New dMRI Data To identify the RGVP, the trained model is deployed to automate the identification process on new dMRI data with unlabeled tractography streamlines generated using the UKF method (see Section 2.1). The inference process works as follows (as illustrated in Figure 3b). First, the streamline point RAS coordinates of new dMRI tractography data are fed into the encoder subnetwork as the input, followed by a computation of the global feature for each streamline. Then, the generated global feature is fed into the classifier to predict the streamline labels (non‐RGVP and RGVP streamlines) using the trained model. It is worth noting that during the training of the SCL model, both streamline labels and microstructure information are utilized. However, when applying the trained model to new data, neither of these is required. Implementation Details Our method is implemented using Pytorch (v1.7) (Paszke et al. 2019), and model training for HCP data is performed on an NVIDIA GeForce GTX 1080 Ti machine. All hyperparameters are set to the default settings suggested in SupWMA (Xue et al. 2023), except for a modified learning rate (0.01) and batch size (512) that are tuned to accompany the addition of FA for pair determination. The threshold TFA in Equation (2) is set to be 0.1 (parameter search from 0.01 to 0.5). Both training phases utilize Adam (Kingma and Ba 2014) as the optimizer with no weight decay. On average, each training epoch takes 4 s with 3GB GPU memory usage when using StreamDA. For inference, the computation can be performed on standard computers equipped with or without GPUs. The whole pipeline takes about 3 min for tractography computation and less than 1 min for RGVP streamline identification using either CPU or GPU. Experimental Evaluation We perform the following experiments to assess the performance of DeepRGVP, including: (1) a comparison to three state‐of‐the‐art (SOTA) methods for tractography parcellation, (2) an ablation study to evaluate each DeepRGVP's sub‐component, and (3) an assessment of DeepRGVP on pituitary patient data. Comparison to SOTA Methods We conduct a comparative analysis of DeepRGVP against three SOTA deep learning approaches that classify streamlines into various categories for tractography parcellation within the white matter of the brain. These benchmark methods include DeepWMA (Zhang, Karayumak‐Cetin, et al. 2020), DCNN (Xu et al. 2019), and SupWMA (Xue et al. 2023). In brief, DeepWMA (Zhang, Karayumak‐Cetin, et al. 2020) is designed for tractography streamline classification using a convolutional neural network (CNN) and streamline spatial coordinate features. DeepWMA has eight CNN layers and three FC layers. It uses a “FiberMap” input feature that converts spatial coordinates of streamlines into 3‐channel images that enable streamlines with forward and backward point orders to have nearly equivalent representations. DeepWMA achieves consistent deep white matter parcellation results across populations (Zhang, Karayumak‐Cetin, et al. 2020). DCNN (Xu et al. 2019) is also based on CNN for tractography streamline classification. DCNN was inspired by (He et al. 2015) and adapted in (Xu et al. 2018) to employ a soft spatial attention (ATT) module (Xu et al. 2015). During training, the DCNN method employs two losses: focal loss (Lin et al. 2017) to help in unbalanced datasets and center loss (CL) (Wen et al. 2016) to assist the network in obtaining better streamline representations. This framework obtains satisfactory accuracy for predicting functionally important white matter pathways that should be protected in the surgery of epilepsy patients (Xu et al. 2019). SupWMA (Xue et al. 2023) is a point‐cloud‐based network (Qi et al. 2017) with SCL (Khosla et al. 2020), which is designed for the classification of tract streamlines for superficial white matter parcellation. It has been shown to obtain highly consistent and accurate parcellation of superficial white matter across the lifespan in health and disease (Xue et al. 2023). For each compared method, we apply the suggested parameter settings described in the papers (Xu et al. 2019; Zhang, Karayumak‐Cetin, et al. 2020; Xue et al. 2023) and the authors' implementation. The comparison to the SOTA methods is performed using the HCP data, with a random splitting where 40 subjects for training, 10 subjects for validation, and 12 subjects for testing. For each method, we train models using training data with and without the proposed StreamDA DA. For each compared method and each HCP testing subject, the accuracy, F1, recall, and precision scores are computed between the prediction and the gold standard RGVP streamlines, and then the average score across all subjects is obtained. Ablation Study We perform an ablation study to evaluate the effects of the proposed MicroSCL and StreamDA. To do so, we compared the following methods. First, we compare with the baseline method that does not perform SCL (i.e., no supervised contrastive loss in our network) nor any DA process. Second, we compare the SCLlabel method that performs label‐based SCL, rather than the proposed microSCL. This method is performed on original training data without any DA process. Third, we compare the SCLlabel+micro method that performs the proposed MicroSCL on original training data without any DA process. Fourth, we compare with SCLlabel+micro+Augrepetition method that performs the proposed MicroSCL with DA by simple duplication of RGVP streamlines, rather than the proposed StreamDA. Fifth, we include the SCLlabel+micro+AugStreamDA method that performs the proposed MicroSCL with StreamDA. The comparison in the ablation study is performed using the HCP data, with a random splitting where 40 subjects for training, 10 subjects for validation, and 12 subjects for testing. For a fair comparison, all methods use the same training/validation/testing split. For each compared method and each HCP testing subject, the accuracy, F1, recall, and precision scores are computed between the prediction and the gold standard RGVP streamlines, and then the average score across all subjects is obtained. RGVP Identification in PTP Data We assess DeepRGVP's generalization to dMRI data from neurosurgical patients with pituitary tumors. For each PTP dataset, we perform RGVP fiber tracking to obtain the input tractography (as described in Section 2.1.2), followed by applying the trained model. The model takes the input new tractography data and classifies them into non‐RGVP streamlines and RGVP streamlines. For quantitative evaluation, in each patient dataset we compute the spatial overlap between the automated identification result and the expert‐selected RGVP (Section 2.1.2). We employ the weighted Dice (wDice) score, an overlap metric used in many tractography parcellation studies (Zhang, Karayumak‐Cetin, et al. 2020; Zhang et al. 2019; Cousineau et al. 2017). Experimental Results Comparison to SOTA Methods As shown in Table 2, DeepRGVP generates the highest accuracy and the highest F1 using both original and augmented data, demonstrating the advantage of the proposed MicroSCL process. All methods except for DCNN achieve better performance using the augmented data compared to the original data, suggesting the benefit of our proposed DA strategy. In addition, DeepRGVP generates relatively high scores for both precision and recall, demonstrating balanced classification performance between RGVP and non‐RGVP streamlines. TABLE 2Quantitative comparisons with the SOTA methods.MethodData Acc F1 Prec Rec DeepWMAOriginal92.18882.98079.01989.270DCNNOriginal95.68779.91381.09978.762SupWMAOriginal96.11782.14982.28182.017ProposedOriginal96.26883.07082.11784.045DeepWMAStreamDA93.44984.61481.95788.034DCNNStreamDA93.14173.78963.20488.634SupWMAStreamDA96.18182.82481.18984.525ProposedStreamDA96.64684.31685.92882.764 Quantitative comparisons with the SOTA methods. Figure 5 visually compares the identified RGVP across the compared SOTA methods. DCNN and DeepWMA are visually overinclusive with more streamlines compared to the expert‐selected ground truth RGVP (e.g., see the inset images of Subjects 1 and 2). DeepRGVP and SupWMA are visually similar to expert selected RGVP. But DeepRGVP has improved sensitivity in identifying local structures, e.g., the ipsilateral pathway passing through the optic chiasm in Subject 2 (indicated by the red arrows). Visualization of expert selected and automatically identified RGVPs (colored by FA values) in two example HCP subjects. The inset zoomed images are provided for better visualization of local RGVP regions. Ablation Study Table 3 provides the results of the ablation study. The higher accuracy and F1 scores obtained by SCLlabel compared with the baseline method (without SCL) show the benefit of using SCL for RGVP identification. The further improvement from SCLlabel+micro over SCLlabel shows the benefit of including microstructure information for better positive and negative sample pair selection in SCL. Furthermore, SCLlabel+micro+Augrepetition further improves the model performance compared to SCLlabel+micro, showing the advantage of using a balanced training dataset. However, the accuracy only slightly increases from 96.449% when using SCLlabel+micro+Augrepetition to 96.646% when using SCLlabel+micro+AugStreamDA, while using the proposed AugStreamDA can further improve the prediction accuracy and F1 scores. Overall, the proposed SCLlabel+micro+AugStreamDA method achieves the best performance, showing the benefits of our network structure, in combination with the newly proposed microstructure‐informed SCL and streamline‐specific DA, for improved RGVP identification. TABLE 3Quantitative comparisons of the ablation methods.Method Acc F1 Prec Rec Baseline96.07682.89078.84987.247 SCLlabel 96.11782.14982.28182.017 SCLlabel+micro 96.26883.07082.11784.045 SCLlabel+micro+Augrepetition 96.44983.41084.90981.964 SCLlabel+micro+AugStreamDA 96.64684.31685.928982.764 Quantitative comparisons of the ablation methods. RGVP Identification in PTP Data Figure 6 shows a visualization of the identified RGVPs using DeepRGVP from example PTP datasets. We can see that DeepRGVP enables successful RGVP identification in neurosurgical patients despite the effects of tumors affecting the RGVP fibers. In addition, the identified RGVP is highly visually comparable with the expert selected RGVPs. Our quantitative evaluation also shows a high spatial overlap between RGVPs obtained from the proposed method and the expert selection, with a high wDice = 0.82 ± 0.08 (a wDice score over 0.72 is suggested to be a good tract overlap (Cousineau et al. 2017; Zhang et al. 2019)). Visual comparison of the identified RGVPs in four example neurosurgical patients with ground truth expert‐selected RGVP. The inset images are provided for better visualization of local RGVP regions. Discussion In this work, we present a novel deep‐learning framework for automated identification of the RGVP. We design a novel microSCL method that leverages both streamline label and tissue microstructure information to determine positive and negative pairs. We propose a simple and successful streamline‐level DA method to address highly imbalanced training data. We perform comparisons with several SOTA deep learning methods designed for tractography parcellation, and we show superior RGVP identification results using DeepRGVP. We also demonstrate a successful application of DeepRGVP in neurosurgical patient data with pituitary lesions affecting the RGVP. We demonstrate that DeepRGVP outperforms several SOTA methods. The DeepWMA and DCNN methods yield less favorable RGVP identification results than DeepRGVP and SupWMA, in terms of quantitative metrics and visualization. This is likely due to the fact that DeepWMA and DCNN are based on traditional CNNs with input in the form of feature vectors or 2D images, while DeepRGVP and SupWMA use an advanced SCL framework with point clouds for streamline representation. Compared to SupWMA, the proposed SupWMA achieves improved identification accuracy and F1 scores. Statistical comparison results using a paired t test show that significantly increased accuracy and F1 scores are obtained, indicating statistically meaningful results are obtained. This is likely due to the novel design of the microSCL strategy that can learn a more discriminative global feature for the classification of RGVP and non‐RGVP streamlines. The proposed streamline‐level DA, StreamDA, can help improve the identification of the RGVP. Across the tractography parcellation methods under study, the DeepWMA, SupWMA, and DeepRGVP methods all benefited from the DA, leading to improved RGVP identification accuracy compared with using the original unbalanced training data. Statistical results show that significantly increased performance is achieved. However, we noticed a decreased performance of DCNN when using the augmented (balanced) data. This may relate to the design of the DCNN method to perform well on unbalanced datasets (Xu et al. 2019). This suggests that DA should be used with careful consideration of the design of the classifier. We demonstrate the generalization of DeepRGVP to dMRI data acquired using different imaging protocols. The ability of a dMRI parcellation method to generalize to data from different acquisitions is important, because there is no standard, and dMRI acquisitions can have widely varying properties such as spatial and angular resolutions, posing a challenge for machine learning approaches. In our study, we train a deep learning model from the high‐quality HCP data, from which the reconstruction of RGVP fibers is relatively easy due to the high spatial and angular resolutions. For the data acquired with a clinical acquisition protocol, we design an ensemble RGVP fiber tracking strategy to improve RGVP fiber tracking so that the RGVP fibers can be effectively reconstructed. In this case, the DeepRGVP method can successfully identify the RGVP by classifying the RGVP and non‐RGVP streamlines. In our previous work, we investigated how tractography parameters affected the RGVP tracking (see (He et al. 2021) for details) and found the detection rate of the different RGVP subdivisions varied. Because our proposed method relies on the reconstruction of the RGVP streamlines, using a single set of parameters will degrade the identification rate of certain RGVP subdivisions. In addition, it is worth noting that distortion artifacts are a known issue in dMRI data. In our study, we used the multi‐tensor UKF method, which was shown to enable highly sensitive tracking through regions affected by distortion artifacts (including the skull base) in both high‐quality and clinically typical dMRI data (Zhang, Xie, et al. 2020; Xie et al. 2020; Albi et al. 2018; Epprecht et al. 2024). Our results on the patient data acquired with standard clinical protocol also show robust identification performance despite the potential distortion artifacts. We show that our method enables automated identification of the RGVPs in pituitary tumor patients. Despite any potential effects from lesions affecting the RGVP, we show good RGVP identification generalization performance in the patient data, even though the RGVP identification model is trained using data from healthy subjects. In related work, previous studies have shown successful parcellation of the major long‐range white matter tracts such as the arcuate fasciculus and corticospinal tract in patient data (Zhang, Karayumak‐Cetin, et al. 2020; Zhang et al. 2018; Wasserthal et al. 2019). To the best of our knowledge, our method is the first to show successful automated identification of the RGVP in neurosurgical patient data. This can improve the current manual selection strategy to resolve inter‐observer variability and may improve clinical efficiency, possibly serving as an important tool for surgical planning in diseases affecting the RGVP. In our current study, we focus on PTPs for illustration, which is one of the representative diseases affecting the RGVP. Our Figure S2 results also show successful identification of the RGVP on dMRI data from other cohorts with different acquisition protocols. A future investigation could include an evaluation of RGVP identification in the setting of other diseases affecting the RGVP (such as optic neuritis, optic gliomas, optic nerve sheath meningiomas, and others), where different regions of RGVP might be affected due to inflammation and tumor infiltration, rather than displacement and tract compression. Potential limitations of the present study, including suggested future work to address limitations, are as follows. First, while our method is designed for the identification of the RGVP, it can generally be used to identify any other CNs and/or anatomical fiber tracts from tractography data. It would be an interesting future investigation to test the performance of DeepRGVP on these brain structures. Second, when generalizing to new patient data, we showed the proposed method obtained highly comparable results to the expert manual selection results, but we noticed that both methods may fail to identify certain subdivisions of the RGVP, e.g., the non‐decussating RGVP portion. This was related to the fact that the relatively low imaging quality and effects of lesions influenced the RGVP tracking performance. Future work could include designing an RGVP‐specific tractography method to enable improved RGVP reconstruction. Fourth, one of the benefits of our method is that the identification of RGVP in new tractography data only uses the streamline trajectory information but not the microstructure FA information. It would be interesting to investigate whether including such information can help with global feature computation in new data. However, this would require harmonization of the testing dMRI data with the training data to remove scanner‐specific differences, considering the substantial variability in microstructure parameters computed from dMRI data from various sources (Cetin‐Karayumak et al. 2019). Also, this would require considering the situation where there is a focal disruption or alteration of tissue microstructure property due to the pathology. In contrast, the existing approach remains generally applicable to diverse tractography data from multiple acquisitions. Finally, future work can include an investigation into more advanced machine learning and artificial intelligence methods to further improve the quantitative measures. Conclusion In this paper, we present a novel deep learning method, DeepRGVP, to enable automated identification of the RGVP using diffusion MRI tractography. Experimental results show that DeepRGVP achieves improved identification accuracy compared to several SOTA methods. In addition, we show a successful application of DeepRGVP to neurosurgical patient dMRI data acquired with different MRI acquisition protocols. We anticipate that DeepRGVP will be a useful tool in dMRI‐based analyses of the RGVP, e.g., in neurosurgical planning research. Overall, our study shows the high potential of using deep learning to automatically identify the RGVP. Conflicts of Interest The authors declare no conflicts of interest. Supporting information Figure S1. The top subfigure shows the placement of the ON, OT, and OC ROIs in one example HCP subject. The bottom subfigure shows the ROI placement of the OTs and ONs, while the OC ROI cannot be visually identified possibly due to effects of the pituitary tumor. Figure S2. RGVP identification performance on dMRI data from Parkinson’s Progression Markers Initiative (PPMI) (https://www.ppmi‐info.org/) database and Autism Brain Imaging Data Exchange (ABIDE) database (https://fcon_1000.projects.nitrc.org/indi/abide). (a–c) are from three example teenagers diagnosed with autism in ABIDE, and (d–f) are from three example elderly adults diagnosed with Parkinson’s Progression in PPMI. The data from these databases are clinically typical dMRI data, which is essentially different from the high‐quality HCP dMRI data. Overall, the obtained RGVPs are visually plausible, showing a good generalization ability of our proposed method. Acknowledgments This work is in part supported by the National Key R&D Program of China (No. 2023YFE0118600), the National Natural Science Foundation of China (Nos. 62371107, 62303413, U23A20334), the Guangdong Basic and Applied Basic Research Foundation (2023A1515011644), and the National Institutes of Health (R01MH108574, P41EB015902, R01MH074794, R01MH125860, R01MH119222, R01MH132610, R01NS125781, R21NS136960, R01AG042512). F.Z. acknowledges a BWH Radiology Research Pilot Grant Award. Funding: This work was supported by National Key Research and Development Program of China (2023YFE0118600), National Natural Science Foundation of China (62303413, 62371107, U23A20334), National Institutes of Health (P41EB015902, R01AG042512, R01MH074794, R01MH108574, R01MH119222, R01MH125860, R01MH132610, R01NS125781, R21NS136960), Basic and Applied Basic Research Foundation of Guangdong Province (2023A1515011644), F.Z. acknowledges a BWH Radiology Research Pilot Grant Award. Sipei Li, Wei Zhang, and Shun Yao contributed equally to this study. Contributor Information Lauren J. O'Donnell, Email: odonnell@bwh.harvard.edu. Fan Zhang, Email: fan.zhang@uestc.edu.cn. Data Availability Statement The data used in this project include the public Human Connectome Project (HCP) dataset (www.humanconnectome.org). The PTP data is our in‐house dataset, which is not publicly available because public availability would compromise participant confidentiality and participant privacy. For The UKF tracography algorithm for RGVP reconstruction is available in the open‐source software SlicerDMRI (https://dmri.slicer.org), and. The code and trained model for automated RGVP identification are available at: https://github.com/SlicerDMRI/DeepRGVP. References Associated Data Supplementary Materials Figure S1. The top subfigure shows the placement of the ON, OT, and OC ROIs in one example HCP subject. The bottom subfigure shows the ROI placement of the OTs and ONs, while the OC ROI cannot be visually identified possibly due to effects of the pituitary tumor. Figure S2. RGVP identification performance on dMRI data from Parkinson’s Progression Markers Initiative (PPMI) (https://www.ppmi‐info.org/) database and Autism Brain Imaging Data Exchange (ABIDE) database (https://fcon_1000.projects.nitrc.org/indi/abide). (a–c) are from three example teenagers diagnosed with autism in ABIDE, and (d–f) are from three example elderly adults diagnosed with Parkinson’s Progression in PPMI. The data from these databases are clinically typical dMRI data, which is essentially different from the high‐quality HCP dMRI data. Overall, the obtained RGVPs are visually plausible, showing a good generalization ability of our proposed method. Data Availability Statement The data used in this project include the public Human Connectome Project (HCP) dataset (www.humanconnectome.org). The PTP data is our in‐house dataset, which is not publicly available because public availability would compromise participant confidentiality and participant privacy. For The UKF tracography algorithm for RGVP reconstruction is available in the open‐source software SlicerDMRI (https://dmri.slicer.org), and. The code and trained model for automated RGVP identification are available at: https://github.com/SlicerDMRI/DeepRGVP.