Estimate renal cell carcinoma recurrence rates using electronic health records Authors: J. Hou, J. Wen, R. Bhattacharya, Z. Wang, S. Morini Sweet, W. Xu, A. Elfiky, L. Wang, R. Srivastava, J. Lu, V. Turzhitsky, R.R. McKay, G. Jayram, M. Sundaram, T. Cai, T. Cai Published: 2026 ABSTRACT BackgroundLack of readily available recurrence data has limited the use of electronic health records (EHR) for risk assessment of cancer recurrence and optimal patient management. This study aims to derive high-quality EHR recurrence data and estimate recurrence rates in overall population and specific subgroups.Materials and methodsUsing EHR data between 1 January 2000 and 1 September 2022, we developed a computational tool for automatically annotating the renal cell carcinoma (RCC) recurrence outcome and a natural language processing (NLP) tool for extracting key RCC characteristics. Using data constructed from stage I-III RCC patients who underwent nephrectomy at Mass General Brigham (2000-2022), we analyzed recurrence rates by TNM (tumor–node–metastasis) stage, grade, and histological subtype. Analyses were conducted from 1 September 2022 to 16 August 2024.ResultsA total of 5603 patients whose EHR met the eligibility criteria were included in the study [3590 (64%) men, 2013 (36%) women; median age at baseline 62 years (range 36-87 years); 4225 (75%) non-Hispanic white, 1378 (25%) other race-ethnicity. Tumor stage was as follows: 3324 (59%) stage I, 778 (14%) stage II, and 128 (2%) stage III, 1373 (25%) missing stage information]. Among patients with TNM stage T1-3 N0M0 clear-cell RCC any grade, EHR-derived recurrences were indicative for true recurrence with area under the receiver operating characteristic curve (AUC) of 0.914 for 5-year recurrence status cross-validated against expert annotated gold standard recurrence times. The estimated overall 5-year recurrence rate was 11.1%. We observe a substantially higher recurrence risk for T3 group (48.8%) versus T1 (2.8%) or T2 (14.2%) and G4 group (45.3%) versus G1 (3.7%), G2 (6.8%), or G3 (18.9%).ConclusionsOur computational approach demonstrates that high-quality recurrence data can be reliably extracted from EHR systems, providing a scalable solution for real-world RCC risk determination. These tools enable health care systems to better identify high-risk patients and potentially guide personalized follow-up strategies and adjuvant treatment options. Introduction Approximately one-third of patients with renal cell carcinoma (RCC)1 are diagnosed with regional or distant RCC. Advanced cancer stage is associated with higher mortality. The 5-year survival rate drops from 82.4% for localized disease to 67.6% for regional and 13.9% for distant disease.2 Although surgery (i.e. partial, or radical nephrectomy) may represent a potentially curative option for patients newly diagnosed with organ confined RCC, up to 40% and 15% of patients may experience recurrence and metastasis, respectively, within 5 years following nephrectomy. The risk for developing any recurrence or metastasis increases with disease stage at the initial surgery.3,4 The treatment landscape for advanced RCC following nephrectomy has rapidly evolved over the past decade with the advent of targeted therapies and immunotherapies.5, 6, 7, 8, 9 The success of these therapies in advanced RCC led to the investigation of their potential in the adjuvant setting.10, 11, 12, 13, 14 Designing adjuvant trials is challenging because of uncertainties of prevalence and outcome of high-risk RCC, despite the use of validated risk scores.14 Real-world studies have demonstrated the potential of differentiating recurrence risks across subgroups. For example, analysis of patients with non-metastatic RCC (N = 643) in the US SEER-Medicare database showed that 45% of all intermediate- to high-risk patients (pT2N0 high grade, pT3N0 any grade) and 85% of high-risk (pT4N0, pTanyN1) patients experienced disease recurrence 5 years after nephrectomy.15,16 The reported 5-year recurrence rate by tumor stage (T) and grade (G) for intermediate- to high-risk patients was: 37% for patients with T3G1-2 disease, 50% for patients with T3G3 disease, and 72% for patient with T3G4 disease.17 Another electronic health records (EHR) study from US-based community oncology clinics included patients’ estimated 5-year recurrence-free survival, 40% for patients with intermediate-to high-risk RCC (N = 239) (60.7% for patients with T3G2 disease, 36.1% for patients with T3G3 disease and 30.4% for patient with T3G4 disease) and 17% for patients with high-risk (N = 35) RCC.18 Although these prior studies did show significant recurrence risk in real-world early-stage RCC patients, the studies are limited by sample size, focused on the elderly Medicare population in the United States and excluded some risk categories such as T1 and most T2 patients. On top of stage and grade, there are other recurrence prediction tools such as the UCLA Integrated Staging System (UISS) and Leibovich criteria.19, 20, 21 These criteria, however, largely use clinical and pathologic features not readily available from EHRs, and there remains significant heterogeneity in validation of these models.22 Given this background, we aimed to conduct a real-world study evaluating the risks of recurrence for early-stage RCC. Key cancer clinical data such as TNM (tumor–node–metastasis) stage, grade, histology, and radiological recurrence, however, cannot be well captured by structured codes in EHRs.23 Abstraction of these cancer clinical data is viable from manual chart review,24,25 but the process is typically time consuming and not scalable for large study cohorts. Named entity recognition natural language processing (NLP) tools can identify the mention of concepts in the narrative notes.26,27 Rule-based and large language model (LLM)-based tools had been developed to extract numeric variables by linking target concepts and corresponding numerical values.28,29 These NLP methods have been applied and validated for vital signs and laboratory tests,28 lung cancers and breast cancer characteristics,29,30 and some characteristics for early-stage RCC including stage, histological type, primary site, and tumor size31; no method has been developed for other cancer characteristics and recurrence to the best of our knowledge. Leveraging recent advancements in natural language processing28,30 and deep learning incidence phenotyping,32 we proposed to develop computational tools to extract RCC characteristics from narrative notes (e.g. TNM stage, grade, histology, nephrectomy) and deep learning algorithms to derive recurrence outcomes from longitudinal EHR data. After validation of extracted RCC characteristics and recurrence against manual chart review, we estimated 5-year recurrence rates across stage-grade subgroups within clinical patient data and across (i) all histological subtypes or (ii) clear cell histological subtype only. Materials and methods The Mass General Brigham (MGB) Institutional Review Board approved the use of the EHR data for this study with a waiver of informed consent owing to use of deidentified data. This study followed the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) reporting guidelines. Figure 1 illustrates the study schematics.Figure 1Study schematics. The LATTE22 algorithm was employed to identify recurring time patterns within the longitudinal EHR data, while the NICE21 tool was used to extract information regarding the stage, grade, and nephrectomy status. EHR, electronic health record; RFS, recurrence-free survival. Study schematics. The LATTE22 algorithm was employed to identify recurring time patterns within the longitudinal EHR data, while the NICE21 tool was used to extract information regarding the stage, grade, and nephrectomy status. EHR, electronic health record; RFS, recurrence-free survival. EHR cohort We created an RCC EHR data mart from the MGB Research Patient Data Repository (RPDR) consisting of 28 715 patients with at least one International Classification of Diseases, Ninth Revision, or International Classification of Diseases and Related Health Problems, Tenth Revision, code for RCC between 1 January 2000 and 1 September 2022. We further classified the RCC status for patients in the data mart based on the unsupervised phenotyping algorithm PheNorm and validated its performance against 200 gold standard labels on the true RCC status created from manual chart review for the random selected subset.33 To identify patients with early-stage RCC, we adapted a cancer NLP tool, named NICE, to extract longitudinal RCC characteristics including nephrectomy, TNM stage, grade, and histology (clear cell, chromophobe, or papillary).30 NICE is a rule-based tool that extracts numeric variables from narrative notes (Supplementary Method 1, available at https://doi.org/10.1016/j.esmorw.2026.100695). We set the baseline as the date of first nephrectomy carried out after the RCC diagnosis. To validate the performance of NICE, we randomly selected 100 patients from the study cohort. A clinical expert reviewed the clinical notes to annotate the RCC stage, grade, and histology at the nephrectomy following the initial diagnosis of RCC to validate if the NICE produced the same data. Mortality data at the last EHR encounter was extracted from both the EHRs and the linked state death registry. Death was considered as a competing risk for recurrence rate analysis. We excluded patients with metastatic cancers (TNM stage M1+) at the initial RCC diagnosis. Patients without documentation of nephrectomy or those that had past nephrectomy before RCC diagnosis were excluded from the study. Patients with TNM stage T4 or N1+, chromophobe, or papillary histological types, missing RCC characteristics were included for derivation of EHR recurrence data but excluded in estimation of recurrence rates. Recurrence data from EHRs We adapted the label efficient computational algorithm for incident phenotyping, named LATTE, to efficiently ascertain the RCC recurrence outcome from longitudinal trajectories of EHR features.32 A clinically trained member of the study team annotated the recurrence records for 199 patients in manual chart review of their EHRs. When a documented recurrence was observed, the patient was labeled as ‘recurrence positive’ at the first date of clinical diagnosis for recurrence. Patients with no evidence documented for recurrence through the last EHR encounter were labeled as ‘recurrence free’. Due to the data leakage from discontinuous care at MGB, we observed some patients with brief documentation on recurrence event yet without detailed information to ascertain the date of recurrence, which we labeled as ‘recurrence likely’. We treated ‘recurrence likely’ in two different ways: (i) as ‘recurrence positive’ with missing data on date of recurrence for the primary analysis and (ii) as ‘recurrence free’. To train the recurrence LATTE algorithm, we selected 104 potentially indicative EHR features (Supplementary Figure S1, available at https://doi.org/10.1016/j.esmorw.2026.100695) and their corresponding semantic embedding vectors from the ARCH medical knowledge graph34,35 and aggregate 3-month counts for each patient.32 The ARCH medical knowledge graph provides a similarity score for each pair of medical concepts, and the 104 selected EHR features were those with similarity greater than 0.1 with RCC. Semantic embedding of concepts reflect the relevance among the concepts with smaller angles between two semantic embeddings indicating closer relationships between corresponding concepts.36 The ARCH medical knowledge network measured the relevance by the occurrence of the EHR features within 30 days for the same patient. The resulting semantic embeddings were used to calculate the similarity score used in our feature selection.34,37 The 3-month window was chosen to balance the resolution of the algorithm (better with smaller windows) and precision of accuracy (better with larger windows) according to previous experiments using LATTE for other clinical variables when the window length generally matches the typical intervals between follow-up visits.32,38 We followed the same hyperparameter settings and training process of LATTE (Supplementary Method 2, available at https://doi.org/10.1016/j.esmorw.2026.100695).32 The trained recurrence LATTE algorithm assigns a cumulative probability score of recurrence over time, and we assign the recurrence time as the first time the score exceeds a threshold (to be reported in Results), and the recurrence status as ‘no’ if the score is below the threshold across all times. The threshold was selected to match the 10-year cumulative incidence rate of recurrence in the labeled subset with ‘likely’ labels interpreted as recurrence with undetermined date. We conducted a sensitivity analysis that interpreted ‘likely’ labels as no recurrence in Supplementary Material (Supplementary Figure S2, and Supplementary Tables S1 and S2, available at https://doi.org/10.1016/j.esmorw.2026.100695). The accuracy of LATTE-derived recurrence was validated against the gold standard labels, for 5-year recurrence as well as recurrence occurring within a 3-month window. We carried out 10-fold cross-validation to estimate classification accuracies. For each fold of the labeled data, we used the other nine folds of labels and the unlabeled data to train a LATTE model and calculated the LATTE scores for imputed cumulative recurrence probability for the in-fold data. Then, we pooled the folds of labels on recurrence and their out-of-fold LATTE scores to evaluate their accuracy. To interpret the role of each EHR feature, we extracted the importance scores of EHR features and visualized them in a word cloud, in which the size of the description reflects the magnitude of the importance score (Supplementary Method 3, Supplementary Figure S1, available at https://doi.org/10.1016/j.esmorw.2026.100695), the optimized weights reflecting the contribution of their embeddings through a ‘concept weighting’ module.32 Statistical analysis We estimated the cumulative incidence rate of recurrence and recurrence-free survival rate up to 5 years for: (i) full study cohort; (ii) T-stage subsets (T1N0, T2N0, and T3N0); (iii) grade subsets (G1, G2, G3, and G4); and (iv) grade subsets within TNM stage T3 (T3N0 & G1/G2, T3N0 & G3, T3N0 & G4). For each set, we estimated the recurrence-free survival rate by the Kaplan–Meier estimator39 and cumulative incidence rate of recurrence by the Fine–Gray estimator.40 We obtained variance estimators via a resampling method.41 We reported other characteristics including the year of RCC diagnosis, waiting time for nephrectomy, overall survival and follow-up duration (Supplementary Figures S3-S6). Results Study cohorts The PheNorm algorithm yielded an area under the receiver operating characteristic curve (AUC) of 0.975 for classifying the RCC status. By setting a specificity of 90%, 14 143 patients were classified as having RCC with a positive predictive value (PPV) of 0.953. Applying the eligibility criteria to these patients, we retained 5603 patients for further analyses [3590 (64%) men, 2013 (36%) women; median age at baseline 62 years (range, 36-87 years); 4225 (75%) non-Hispanic white, 1378 (25%) other race/ethnicity; 3324 (59%) with stage I, 778 (14%) with stage II, 128 (2%) with stage III, 1373 (25%) without documented stage]. The baseline characteristics of the eligible EHR cohort are presented in Table 1. The baseline cancer characteristics from our NLP tool demonstrated high accuracy when compared against manually annotated RCC characteristics from clinical notes over a random subset of 50 patients (accuracy 98% for TNM stage, 100% for grade, 96% for histological subtype). Using the data of 9284 patients who were phenotype RCC and had at least one extraction of RCC characteristics from NICE, we completed the training of the LATTE model for their 3-month RCC recurrence within 1 hour on a single graphics processing unit (GPU) (NVIDIA RTX 6000) with 100 gigabyte memory.Table 1Patient characteristics of the study cohortVariableFull data (N = 5603)TNM T1-3 (N = 3279)TNM T1-3 clear cell (N = 2312)Age (years)62 (53-70)61 (52-68)60 (52-68)Sex, n (%) Female2013 (36)1175 (36)895 (39) Male3590 (64)2104 (64)1417 (61)Race, n (%) Non-Hispanic white4225 (75)2481 (76)1789 (77) Other1378 (25)798 (24)523 (23)Stage I, n (%)3324 (59)2698 (82)1878 (81)Stage II, n (%)778 (14)581 (18)434 (19)Stage III, n (%)128 (2)——Stage missing, n (%)1373 (25)——TMN stage T1, n (%)2817 (50)2337 (71)1644 (71)TMN stage T2, n (%)557 (10)361 (11)234 (10)TMN stage T3, n (%)791 (14)581 (18)434 (19)TMN stage T4, n (%)65 (1)——TMN stage T missing, n (%)1373 (25)——Grade 1, n (%)543 (10)372 (11)295 (13)Grade 2, n (%)2335 (42)1843 (56)1289 (56)Grade 3, n (%)1042 (19)890 (27)589 (25)Grade 4, n (%)238 (4)174 (5)139 (6)Grade missing, n (%)1445 (26)——Histology, n (%) Clear-cell3485 (62)2312 (71)2312 (100) Chromophobe431 (8)177 (5)— Papillary1348 (24)790 (24)— Not specified339 (6)——Sarcomatoid, n (%) Positive158 (3)73 (2)35 (2) Negative1167 (21)582 (18)384 (17) NOS4278 (76)2624 (80)1893 (82)Median (1st-3rd quartiles) for age and number (proportion) for other discrete variables. TNM, tumor–node–metastasis. Table 1Patient characteristics of the study cohortVariableFull data (N = 5603)TNM T1-3 (N = 3279)TNM T1-3 clear cell (N = 2312)Age (years)62 (53-70)61 (52-68)60 (52-68)Sex, n (%) Female2013 (36)1175 (36)895 (39) Male3590 (64)2104 (64)1417 (61)Race, n (%) Non-Hispanic white4225 (75)2481 (76)1789 (77) Other1378 (25)798 (24)523 (23)Stage I, n (%)3324 (59)2698 (82)1878 (81)Stage II, n (%)778 (14)581 (18)434 (19)Stage III, n (%)128 (2)——Stage missing, n (%)1373 (25)——TMN stage T1, n (%)2817 (50)2337 (71)1644 (71)TMN stage T2, n (%)557 (10)361 (11)234 (10)TMN stage T3, n (%)791 (14)581 (18)434 (19)TMN stage T4, n (%)65 (1)——TMN stage T missing, n (%)1373 (25)——Grade 1, n (%)543 (10)372 (11)295 (13)Grade 2, n (%)2335 (42)1843 (56)1289 (56)Grade 3, n (%)1042 (19)890 (27)589 (25)Grade 4, n (%)238 (4)174 (5)139 (6)Grade missing, n (%)1445 (26)——Histology, n (%) Clear-cell3485 (62)2312 (71)2312 (100) Chromophobe431 (8)177 (5)— Papillary1348 (24)790 (24)— Not specified339 (6)——Sarcomatoid, n (%) Positive158 (3)73 (2)35 (2) Negative1167 (21)582 (18)384 (17) NOS4278 (76)2624 (80)1893 (82)Median (1st-3rd quartiles) for age and number (proportion) for other discrete variables. TNM, tumor–node–metastasis. Patient characteristics of the study cohort Median (1st-3rd quartiles) for age and number (proportion) for other discrete variables. TNM, tumor–node–metastasis. EHR recurrence Patients were followed up through EHRs for a median of 7.29 years (95% range 0.60-23.45 years), during which we observed 1108 (20%) deaths and 938 (17%) recurrences (Table 2). For classifying the recurrence status in the 3-month window, LATTE attained an AUC of 0.922 when averaging over a follow-up window up to 5 years and 0.923 when averaging up to 10 years of follow up. Setting a cut-off to match the observed recurrence rate in the corresponding subset with manually annotated recurrence dates (stage I defined as T1-2 & N0M0: threshold 0.80, recurrence rate 7.0%; stage II defined as T3-4 & N0M0: threshold 0.10, recurrence rate 63.6%), the derived longitudinal 3-month recurrence status achieved sensitivity 0.749, specificity 0.994, and PPV 0.895 across all quarterly periods up to 10-year follow-up. Among patients with TNM stage T1-3 N0M0 clear cell RCC any grade, the LATTE recurrence score achieved an AUC of 0.914 for classifying 5-year recurrence status, sensitivity 0.745, specificity 0.990, and PPV 0.851.Table 2Patient outcomes based on EHR recurrence (unit: years)VariableFull data (N = 5603)TNM T1-3 (N = 3279)TNM T1-3 clear-cell (N = 2312)Follow-up (year, 95% range)7.29 (0.60-23.45)6.78 (0.61-19.60)6.52 (0.62-19.43)Mortality, n (%)1108 (20%)446 (14%)313 (14%)Recurrence, n (%)938 (17%)424 (13%)338 (15%)EHR, electronic health record; TNM, tumor–node–metastasis.Median (95% range or confidence intervals) for time-to-event. Count (percentage) for binary variables. Table 2Patient outcomes based on EHR recurrence (unit: years)VariableFull data (N = 5603)TNM T1-3 (N = 3279)TNM T1-3 clear-cell (N = 2312)Follow-up (year, 95% range)7.29 (0.60-23.45)6.78 (0.61-19.60)6.52 (0.62-19.43)Mortality, n (%)1108 (20%)446 (14%)313 (14%)Recurrence, n (%)938 (17%)424 (13%)338 (15%)EHR, electronic health record; TNM, tumor–node–metastasis.Median (95% range or confidence intervals) for time-to-event. Count (percentage) for binary variables. Patient outcomes based on EHR recurrence (unit: years) EHR, electronic health record; TNM, tumor–node–metastasis. Median (95% range or confidence intervals) for time-to-event. Count (percentage) for binary variables. Estimate RCC recurrence We identified 3279 patients who had TNM stage T1N0-T3N0 at RCC diagnosis and complete RCC characteristics (Figure 2). We estimated the cumulative incidence rates and recurrence-free survival for the overall eligible cohort and subsets defined by T stage and grades (Figure 3, Table 3). The estimated 5-year cumulative incidence rate was 11.8% [95% confidence interval (CI) 10.5% to 13.0%] for the overall cohort. We observed increasing cumulative incidence rates along T stage, 2.8% (95% CI 2.1% to 3.5%) for T1 subset, 14.2% (95% CI 10.2% to 17.9%) for T2 subset, and 48.8% (95% CI 43.8% to 53.3%) for T3 subset, and along grade, 3.7% (95% CI 1.6% to 6.0%) for grade 1, 6.8% (95% CI 5.6% to 8.0%) for grade 2, 18.9% (95% CI 16.3% to 21.4%) for grade 3, and 45.3% (95% CI 37.0% to 52.2%) for grade 4. Within the T3 subset, we observed stratified cumulative incidence rates by grade, 39.0% (95% CI 32.5% to 45.8%) for grade 1 to 2, 51.4% (95% CI 43.6% to 58.9%) for grade 3, and 68.9% (95% CI 57.2% to 80.0%) for grade 4. The pattern of recurrence-free survival decreasing with higher T stage and grade was consistent with the cumulative incidence rate of recurrence. Over the subset of clear cell histology type, the estimated 5-year cumulative incidence rate was 13.5% (95% CI 12.2% to 15.2%). We observed a similar trend in recurrences rate across the T stage and grade subgroups within the clear cell histology subset, 3.2% (95% CI 2.4% to 4.3%) for the T1 subset, 19.7% (95% CI 14.2% to 24.9%) for the T2 subset, and 52.3% (95% CI 46.9% to 57.3%) for the T3 subset, and along grade, 4.1% (95% CI 1.8% to 6.8%) for grade 1, 8.1% (95% CI 6.4% to 9.8%) for grade 2, 22.0% (95% CI 18.6% to 25.9%) for grade 3, and 51.7% (95% CI 42.6% to 60.7%) for grade 4. Within the T3 clear cell subset, we also observed stratified cumulative incidence rates by grade, 42.4% (95% CI 34.6% to 50.2%) for grade 1 to 2, 53.7% (95% CI 44.8% to 62.1%) for grade 3, and 73.3% (95% CI 61.7% to 85.8%) for grade 4.Figure 2Construction of the study cohort. RCC, renal cell carcinoma, TNM, tumor–node–metastasis.Figure 3Cumulativeincidence of recurrence andrecurrence-freesurvival (RFS). Plots of Kaplan-Meier estimation up to 10 years after nephrectomy in T stage-grade subgroups among patients with TNM stage T1-3 & N0M0, any grade and clear-cell histology. Shaded areas represent pointwise 95% confidence bands. Results at 5- and 10-year marks are reported in Table 3. TNM, tumor–node–metastasis.Table 3Estimated 5-year cumulative incidence of recurrence (5 year/10 year-recurrence) and recurrence-free survival (5 year/10 year-RFS) in subgroupsGroupStage T1-T3, any histologyStage T1-T3, clear-cellSize5-Year10-YearSize5-Year10-YearrecurrenceRFSrecurrenceRFSrecurrenceRFSrecurrenceRFSFull cohort327911.8 (10.5-13.0)86.9 (85.5-88.1)15.9 (14.5-17.3)77.0 (75.1-78.9)231213.7 (12.2-15.2)85.1 (83.4-86.7)17.9 (16.1-19.8)75.4 (73.2-77.9)TNM stage T123372.8 (2.1-3.5)96.0 (95.2-96.9)4.7 (3.7-5.9)89.1 (87.4-90.7)16443.2 (2.4-4.3)95.6 (94.6-96.7)5.5 (4.2-7.0)88.7 (86.5-90.8)TMN stage T236114.2 (10.2-17.9)83.7 (79.8-87.4)18.8 (14.0-23.4)70.6 (64.4-76.1)23419.7 (14.2-24.9)77.8 (71.9-82.6)25.9 (19.9-33.3)64.3 (57.2-71.6)TMN stage T358148.8 (43.8-53.3)49.3 (44.2-53.7)63.2 (57.5-68.3)27.3 (22.4-32.2)43452.3 (46.9-57.3)46.3 (40.9-51.7)64.1 (58.1-69.6)26.5 (20.4-32.6)Grade 13723.7 (1.6-6.0)95.3 (92.8-97.7)5.7 (2.7-8.9)89.2 (84.6-93.5)2954.1 (1.8-6.8)94.7 (91.9-97.4)6.6 (3.0-10.9)86.9 (81.4-92.5)Grade 218436.8 (5.6-8.0)92.0 (90.5-93.3)9.9 (8.4-11.8)82.4 (80.0-84.8)12898.1 (6.4- 9.8)90.9 (89.2-92.6)11.4 (9.5-13.8)81.6 (78.8-84.5)Grade 389018.9 (16.3-21.4)79.3 (76.4-82.1)25.9 (22.7-29.4)67.7 (63.5-71.2)58922.0 (18.6-25.9)76.3 (72.4-79.6)30.1 (25.6-34.5)64.9 (60.0-69.4)Grade 417445.3 (37.0-52.2)52.1 (44.1-60.1)52.3 (44.3-61.2)38.9 (29.6-48.0)13951.7 (42.6-60.7)45.7 (37.9-55.0)54.9 (45.4-65.0)34.7 (25.0-45.0)T3-G1/224939.0 (32.5-45.8)60.5 (53.7-67.5)53.7 (45.0-61.4)36.0 (26.1-45.3)18342.4 (34.6-50.2)57.6 (48.9-65.5)55.6 (46.4-65.7)35.3 (25.1-46.2)T3-G323651.4 (43.6-58.9)45.6 (38.1-53.3)68.5 (60.7-76.7)22.2 (15.4-29.6)17053.7 (44.8-62.1)43.3 (34.1-51.4)70.9 (61.9-80.1)21.6 (13.6-29.7)T3-G49668.9 (57.2-80.0)29.0 (17.4-40.6)75.6 (65.2-85.6)11.9 (1.1-25.4)8173.3 (61.7-85.8)25.5 (13.3-38.9)76.1 (63.5-88.6)8.5 (0.05-22.09)The same analyses generated the numbers presented in this table and Figure 3. RFS, recurrence-free survival; TNM, tumor–node–metastasis. Values are given as % (95% confidence interval). Construction of the study cohort. RCC, renal cell carcinoma, TNM, tumor–node–metastasis. Cumulativeincidence of recurrence andrecurrence-freesurvival (RFS). Plots of Kaplan-Meier estimation up to 10 years after nephrectomy in T stage-grade subgroups among patients with TNM stage T1-3 & N0M0, any grade and clear-cell histology. Shaded areas represent pointwise 95% confidence bands. Results at 5- and 10-year marks are reported in Table 3. TNM, tumor–node–metastasis. Table 3Estimated 5-year cumulative incidence of recurrence (5 year/10 year-recurrence) and recurrence-free survival (5 year/10 year-RFS) in subgroupsGroupStage T1-T3, any histologyStage T1-T3, clear-cellSize5-Year10-YearSize5-Year10-YearrecurrenceRFSrecurrenceRFSrecurrenceRFSrecurrenceRFSFull cohort327911.8 (10.5-13.0)86.9 (85.5-88.1)15.9 (14.5-17.3)77.0 (75.1-78.9)231213.7 (12.2-15.2)85.1 (83.4-86.7)17.9 (16.1-19.8)75.4 (73.2-77.9)TNM stage T123372.8 (2.1-3.5)96.0 (95.2-96.9)4.7 (3.7-5.9)89.1 (87.4-90.7)16443.2 (2.4-4.3)95.6 (94.6-96.7)5.5 (4.2-7.0)88.7 (86.5-90.8)TMN stage T236114.2 (10.2-17.9)83.7 (79.8-87.4)18.8 (14.0-23.4)70.6 (64.4-76.1)23419.7 (14.2-24.9)77.8 (71.9-82.6)25.9 (19.9-33.3)64.3 (57.2-71.6)TMN stage T358148.8 (43.8-53.3)49.3 (44.2-53.7)63.2 (57.5-68.3)27.3 (22.4-32.2)43452.3 (46.9-57.3)46.3 (40.9-51.7)64.1 (58.1-69.6)26.5 (20.4-32.6)Grade 13723.7 (1.6-6.0)95.3 (92.8-97.7)5.7 (2.7-8.9)89.2 (84.6-93.5)2954.1 (1.8-6.8)94.7 (91.9-97.4)6.6 (3.0-10.9)86.9 (81.4-92.5)Grade 218436.8 (5.6-8.0)92.0 (90.5-93.3)9.9 (8.4-11.8)82.4 (80.0-84.8)12898.1 (6.4- 9.8)90.9 (89.2-92.6)11.4 (9.5-13.8)81.6 (78.8-84.5)Grade 389018.9 (16.3-21.4)79.3 (76.4-82.1)25.9 (22.7-29.4)67.7 (63.5-71.2)58922.0 (18.6-25.9)76.3 (72.4-79.6)30.1 (25.6-34.5)64.9 (60.0-69.4)Grade 417445.3 (37.0-52.2)52.1 (44.1-60.1)52.3 (44.3-61.2)38.9 (29.6-48.0)13951.7 (42.6-60.7)45.7 (37.9-55.0)54.9 (45.4-65.0)34.7 (25.0-45.0)T3-G1/224939.0 (32.5-45.8)60.5 (53.7-67.5)53.7 (45.0-61.4)36.0 (26.1-45.3)18342.4 (34.6-50.2)57.6 (48.9-65.5)55.6 (46.4-65.7)35.3 (25.1-46.2)T3-G323651.4 (43.6-58.9)45.6 (38.1-53.3)68.5 (60.7-76.7)22.2 (15.4-29.6)17053.7 (44.8-62.1)43.3 (34.1-51.4)70.9 (61.9-80.1)21.6 (13.6-29.7)T3-G49668.9 (57.2-80.0)29.0 (17.4-40.6)75.6 (65.2-85.6)11.9 (1.1-25.4)8173.3 (61.7-85.8)25.5 (13.3-38.9)76.1 (63.5-88.6)8.5 (0.05-22.09)The same analyses generated the numbers presented in this table and Figure 3. RFS, recurrence-free survival; TNM, tumor–node–metastasis. Values are given as % (95% confidence interval). Estimated 5-year cumulative incidence of recurrence (5 year/10 year-recurrence) and recurrence-free survival (5 year/10 year-RFS) in subgroups The same analyses generated the numbers presented in this table and Figure 3. RFS, recurrence-free survival; TNM, tumor–node–metastasis. Values are given as % (95% confidence interval). Discussion We successfully derived high-quality EHR recurrence outcomes and used them to estimate the recurrence risks and recurrence-free survival for patients with T1-T3 RCC. Our scalable and accurate analytics pipeline for curating RCC characteristics and outcomes addresses the major challenge of a lack of key RCC variables to carry out cancer predictive modeling studies using EHRs.23 Unlike other available recurrence outcomes in real-world data,42,43 our study leveraged the rich narrative notes in EHRs to create gold standard labels on RCC recurrence. The gold standard labels not only enabled the training of the accurate model for the EHR RCC recurrence, but also provided validated assurance. The semi-supervised deep learning model LATTE employed in this study was highly label efficient, achieving precise identification of RCC recurrence based on merely 199 labels. This approach may offer a more scalable solution for cancer research using EHR rather than databases created purely from manual abstraction with potential cost efficiencies.24,25 Our NLP pipeline enabled stratified analysis of recurrence risk, identifying the elevated risks of recurrence along TNM stage and various grades. T3 clinical stage was confirmed to be associated with significant risks of recurrence, and we also observed an elevated risk of recurrence in T2 versus T1. We observed a low risk of recurrence in grade 1 and 2 subgroups and an elevated risk of recurrence in grade 3 and 4 subgroups. Consistent patterns were observed within the T3 and clear cell histological type subsets. Earlier publications3,4 based on registry and chart reviewed clinically annotated data have shown that at 5 years, >40% of T3 patients recur. The LATTE-estimated recurrence rates from EHR data from this study corroborated this, with a high percentage of T3 recurring at 5 years from nephrectomy, with a continuing trend until year 10. The recurrence outcome data, combined with longitudinal EHR information, might have applicability for a range of downstream analyses. These include developing comprehensive prediction models for RCC recurrence using EHR-derived risk features, imaging characteristics, and conducting comparative effectiveness studies on treatments. Furthermore, integrating EHR data with other modalities, such as imaging and genomics, holds potential for developing individualized monitoring and treatment strategies. Our artificial intelligence (AI)-driven approach represents a significant advancement in RCC risk stratification tools. While current prediction models like UISS, stage, size, grade and necrosis (SSIGN) score, and Leibovich score have provided valuable frameworks for risk assessment, they are limited by their reliance on clinical data not readily available in routinely collected EHRs. Our computational algorithm leverages rich longitudinal data within EHRs to derive cancer characteristics and recurrence outcomes with minimal burden for care providers, which opens up the route for more dynamic and comprehensive risk assessment incorporating other risk factors captured in EHRs. By automatically extracting and analyzing complex clinical data patterns, this tool could complement existing risk models and enable real-time risk stratification at scale. This capability is particularly valuable for treatment planning, as recent advances in adjuvant therapy require accurate identification of high-risk patients who would benefit most from treatment. Furthermore, the tool’s ability to process unstructured clinical data could guide the identification of subtle risk factors that might be missed in traditional scoring systems, potentially leading to more precise risk stratification for individual patients. Finally, the pipeline is computationally efficient and scalable for large-scale EHR data. Model training and inference are lightweight, and the computational cost grows linearly with patient history length, enabling efficient application to large cohorts. In addition, data preprocessing and feature extraction can be parallelized across patients, supporting routine deployment in clinical and population-level research settings. Limitations The study has limitations. Due to the potential heterogeneity in EHR documentation patterns, the RCC recurrence outcomes derived from our single-institution study may not achieve accuracy levels in another institution. It is important to create at least 50 gold standard annotations of recurrence data in the new EHR to evaluate the transportability of our algorithm for RCC recurrence. In the case of unsatisfactory accuracy, researchers should consider creating about 100 gold standard annotations and retrain the algorithm according to the pipeline described in the paper. In the study, we relied heavily on the narrative notes for extracting RCC TNM stage, grade, histology, and indicators for RCC recurrence, as well as validation in terms of agreement with narrative notes through manual chart review. Therefore, the quality of derived RCC data is limited by the quality of documentation in clinical notes. Standard documentation of key clinical information would greatly improve the data quality of our pipeline and its transferability to an environment sharing the same standard. A rigorous validation framework may be required when human expertise cannot consistently determine the underlying truth.44 The emergence of language artificial intelligence may assist the providers in producing complete documentation with minimal additional effort. The lack of a unified EHR system across different health care institutions may also cause a data leakage issue so that part of the EHR related to RCC care at another institution was not available in the MGB RPDR. Thus, the reported rates should be interpreted with caution because: (i) the baseline time could be misaligned by nephrectomy carried out at another hospital and (ii) recurrence time could be overestimated if recurrence was diagnosed during a period of care under providers outside MGB. As a rule-based NLP tool, NICE can be sensitive to the template of documentation for RCC characteristics. While we envision that the straightforward rules of NICE should be largely robust, it is still possible to fail in some institutes during certain periods of time. Therefore, we recommend researchers always sample a small validation set to validate the local performance of NICE.45 In case of poor performance, different rules should be considered based on the insights from the validation set. Recent advances in LLMs may provide an alternative to identify the mention of key baseline characteristics and outcomes in narrative notes along with the referred timing of the event, which can be more robust to the template of documentation by learning the semantic meanings. For example, a framework for validating LLM-extracted data has been proposed and demonstrated for breast cancer.44 Finally, the study only involved data from a single institute and lacked the validation of generalizability and strategy of integration for multi-institute studies. While the proposed methodology can be repeated in another health care institute, further methodological development is needed for leveraging external data through transfer learning and aggregating multiple health care institutes through federated learning without sharing individual-level data.