Artificial intelligence-guided analysis of the tumor microenvironment predicts response to pembrolizumab in rare tumors Authors: Mohamed H Derbala, Bettzy Stephen, Woochan Hwang, Chan-Young Ock, Soohyun Hwang, Joud Hajjar, Seungeun Lee, Jianling Zhou, Serdar A Gurses, Mohamed A Gouda, Kathryn E McGonagle, Anas Alshawa, Lilibeth Castillo, Abdulrazzak Zarifa, Siqing Fu, Sarina A Piha-Paul, Apostolia M Tsimberidou, Funda Meric-Bernstam, Luisa Maren Solis Soto, Maria Gabriela Raso, Siraj Ali, Aung Naing Published: 2026 ABSTRACT AbstractBackgroundPathologic tumor response and changes in the tumor microenvironment (TME) predict outcomes to immune checkpoint inhibitors, but are understudied in rare tumors. We investigated whether artificial intelligence (AI)-powered analyses of pretreatment and on-treatment biopsies may inform treatment outcomes to pembrolizumab.MethodsWe evaluated 256 baseline and 248 on-treatment biopsies from 84 patients with rare tumors (10 cohorts) in a phase II pembrolizumab trial. Intratumoral tumor-infiltrating lymphocyte (iTIL) density and tumor content (TC) were assessed on H&E-stained slides using a deep learning–based analyzer (Lunit SCOPE IO). Baseline iTIL and changes in iTIL and TC were correlated with progression-free survival (PFS) and overall survival (OS). Multiplex immunofluorescence was performed in 27 paired samples to assess TME changes.ResultsIn the high-iTIL tumor group, a baseline iTIL of ≥60 cells/mm2 was associated with favorable PFS (HR 0.49, 95% CI 0.25 to 0.99, p=0.046) and higher CD8+ and CD8+PD-1+ and lower FoxP3+CD8+PD-1+ T-cell density. However, this association with PFS was not seen in the overall cohort (HR 0.62, 95% CI 0.37 to 1.06, p=0.082). In paired biopsies, on-treatment increase in iTIL showed a trend toward improved PFS (HR 0.64, 95% CI 0.40 to 1.06, p=0.084) and was significantly associated with improved OS (HR 0.55, 95% CI 0.35 to 1.01, p=0.037). This increase was also associated with reduced spatial distance between CD8+ immune and tumor cells. Decreased TC during treatment was significantly associated with prolonged PFS and OS (PFS: HR 0.51, p=0.019; OS: HR 0.54, p=0.042). The combination of increased iTIL and decreased TC was significantly associated with better PFS (HR 0.36, p=0.009) and OS (HR 0.36, p=0.029).ConclusionAI-powered assessment of the TME before and during treatment may help inform treatment outcomes to pembrolizumab in patients with rare tumors.Trial registration numberNCT02721732.Keywords: Tumor Microenvironment, Immune Checkpoint Inhibitors, Biomarker, Immunotherapy WHAT IS ALREADY KNOWN ON THIS TOPIC Recent studies across multiple cancers have shown that dynamic changes in the tumor microenvironment (TME) after immune checkpoint inhibitors (ICIs) could serve as surrogate markers to predict clinical outcomes in these patients, either as neoadjuvant therapy or for metastatic disease. However, these associations have rarely been systematically studied in rare tumors. Recent studies across multiple cancers have shown that dynamic changes in the tumor microenvironment (TME) after immune checkpoint inhibitors (ICIs) could serve as surrogate markers to predict clinical outcomes in these patients, either as neoadjuvant therapy or for metastatic disease. However, these associations have rarely been systematically studied in rare tumors. WHAT THIS STUDY ADDS Using a novel artificial intelligence (AI)-guided model to analyze the TME in patients with rare tumors receiving ICI therapy, we found that dynamic changes in intratumoral tumor-infiltrating lymphocyte (iTIL) and tumor content (TC) were associated with progression-free survival and overall survival in these patients. While baseline iTIL levels were informative in tumors with high immune infiltration, iTIL dynamics and changes in TC offered additional pharmacodynamic insight even after adjusting for histologic subtype. Using a novel artificial intelligence (AI)-guided model to analyze the TME in patients with rare tumors receiving ICI therapy, we found that dynamic changes in intratumoral tumor-infiltrating lymphocyte (iTIL) and tumor content (TC) were associated with progression-free survival and overall survival in these patients. While baseline iTIL levels were informative in tumors with high immune infiltration, iTIL dynamics and changes in TC offered additional pharmacodynamic insight even after adjusting for histologic subtype. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY This research highlights the potential of AI-guided analysis of the TME to quantitatively assess iTIL and TC dynamics on routine biopsy specimens in patients with rare tumors receiving ICIs. The observed associations between dynamic changes in iTIL and TC and survival outcomes support further investigation of deep-learning enhanced histopathologic analysis as an approach for characterizing immunotherapy response in rare cancers. This research highlights the potential of AI-guided analysis of the TME to quantitatively assess iTIL and TC dynamics on routine biopsy specimens in patients with rare tumors receiving ICIs. The observed associations between dynamic changes in iTIL and TC and survival outcomes support further investigation of deep-learning enhanced histopathologic analysis as an approach for characterizing immunotherapy response in rare cancers. Background Immune checkpoint inhibitors (ICIs) have transformed the landscape of cancer treatment. Despite the clear survival benefit of ICIs in many cancer types, antitumor responses are often achieved in only a subset of patients.1 The broader use of these agents also comes with an increase in the incidence of immune-related adverse events.2 In tandem, the existing biomarkers for predicting response, such as programmed death-ligand 1 (PD-L1) expression, tumor mutational burden, and microsatellite instability, have their limitations.3 Therefore, there is a substantial unmet need to identify novel biomarkers that predict response to ICIs. The tumor microenvironment (TME) is pivotal in cancer development and response to treatment. Gaining a comprehensive understanding of the composition of the TME could greatly help in identifying novel biomarkers and developing effective treatment strategies. Recent studies have shown that dynamic changes in the TME after immunotherapy could be surrogate markers to predict clinical outcomes in patients treated with ICIs, either as neoadjuvant therapy or for metastatic disease, across multiple cancers.4,7 However, these associations have rarely been systematically studied in rare tumors, defined by the US National Cancer Institute as those with an incidence of less than 15 cases per 100,000 people per year.8 The current study investigated whether an artificial intelligence (AI)-powered TME analysis of pretreatment and early on-treatment biopsy specimens can predict survival outcomes in patients with rare tumors treated with pembrolizumab, an anti-programmed cell death protein 1 (anti-PD-1) inhibitor. Methods Study design The study included 84 patients who had both pretreatment and on-treatment biopsy specimens available. A Consolidated Standards of Reporting Trials (CONSORT) diagram showing patient selection is provided in online supplemental figure S1. All patients were enrolled in 1 of 10 cohorts (squamous cell carcinoma of the skin, small cell malignancies of non-pulmonary origin, adrenocortical carcinoma, medullary renal cell carcinoma, carcinoma of unknown primary, penile carcinoma, vascular sarcoma, germ cell/testicular tumor, paraganglioma/pheochromocytoma, or other rare histologic subtype) as part of a phase II study of pembrolizumab in rare cancers, conducted at The University of Texas MD Anderson Cancer Center (NCT02721732). All patients received treatment as follows. Pembrolizumab was administered as an intravenous dose of 200 mg on the first day of every 21-day dosing cycle until disease progression or unacceptable toxicity, death, withdrawal of consent, discontinuation from the study treatment for any other reason, or completion of 24 months of treatment with pembrolizumab. Tumor imaging was performed at baseline and at the end of three cycles (9 weeks). Response to treatment was evaluated using immune-related Response Evaluation Criteria in Solid Tumors.9 Biopsy collection and specimen evaluation We evaluated 256 baseline and 248 on-treatment (cycle 1, days 15–21) biopsy specimens from 84 patients. Biopsy specimens were taken from accessible, previously non-irradiated lesions (primary or metastatic) under imaging guidance (ultrasonography, CT, or MRI). All specimens were Tru-cut core biopsy specimens collected using a coaxial technique. Using a lesion selection tool developed by our group,10 we biopsied the same target lesion at both baseline and during cycle 1 between days 15 and 21. A “biopsy specimen” was defined as an individual core submitted as formalin-fixed paraffin-embedded (FFPE) tissue. All core-derived H&E slides underwent quality control. Specimens were evaluated for multiple morphometric and qualitative parameters, including overall length and width, from which the total tumor area was calculated. Quantitative assessment included the percentage of tumor present within the section, the proportion of malignant versus non-malignant cells in the tumor area, and the extent of necrosis. Additional characterization documented the presence and proportion of non-malignant cellular components, as well as the percentage of non-cellular areas both inside and outside the tumor region. Qualitative review also captured the degree of intratumoral inflammation. The sample was further assessed for adequacy, including confirmation of whether at least 100 FFPE or 300 fresh-frozen malignant cells were present, and a final determination of tissue validity was recorded. A specimen was considered “unanalyzable” if it failed adequacy criteria such as insufficient malignant cells, extensive necrosis precluding reliable tumor content (TC) estimation, or absence of viable tumor. One patient was excluded due to lack of analyzable TC at the relevant time point based on these predefined criteria. Up to five cores were targeted per patient per time point. When multiple cores were obtained, each core was processed and evaluated independently. Among patients with more than one evaluable core (slide) at a given time point, within-patient variability in iTIL density (online supplemental figure S2A) and TC (online supplemental figure S2B) was reviewed prior to averaging. Given this observed biological variability, values from all evaluable slides at the same time point were averaged to generate a single patient-level measurement, thereby reducing the impact of intralesional spatial heterogeneity and core-level sampling noise. This approach is consistent with prior analyses from the same phase II trial.11 The distribution of core counts per patient per time point is now summarized in online supplemental table 1. One patient had unanalyzable TC in pretreatment biopsy specimen and was excluded from the analysis. AI-powered assessment of the TME Lunit SCOPE IO (Lunit, Seoul, Republic of Korea), a deep learning-based whole-slide image (WSI) analyzer developed across more than 26 different tumor types and origins, was applied to H&E-stained WSIs to assess the TME. The cell detection model was developed based on 20,617 patches extracted from 5,609 H&E-stained WSIs, and the tissue segmentation model was developed using 76,110 patches extracted from 18,935 H&E-stained WSIs. Annotations by board-certified pathologists of 2,828,448 cells and 1.07×1010 µm2 of cancer areas and stroma were used for development. The inference process consists of tissue segmentation and cell detection, using convolutional neural networks for semantic segmentation of cancer areas and cancer stroma and for identifying tumor cells and lymphocytes.12,14 More information about Lunit SCOPE IO development and validation can be found in the online supplemental methods. Baseline and on-treatment changes were identified based on iTIL and TC. iTIL was defined as the density of detected lymphocytes in the segmented cancer area tissue (cells/mm2), and TC was defined as the ratio of cancer area to the total of cancer area, stroma, and background tissue (figure 1). Stromal TIL (sTIL) density was defined as the sTIL count divided by the stromal area (cells/mm2). The difference between on-treatment and baseline iTIL and TC values was normalized by their values at baseline, and further analysis was performed using the normalized unit. Study design using AI-powered analysis of the TME. AI, artificial intelligence; D, day; iTIL, intratumoral TIL; sTIL, stromal TIL; TIL, tumor-infiltrating lymphocyte; TME, tumor microenvironment; WSI, whole-slide image. Multiplex immunofluorescence and data analysis For exploratory analysis of whether AI-based predictive subgroups were associated with specific TME composition or spatial distribution between tumor and immune cells, 8-plex multiplex immunofluorescence (mIF) staining (Akoya Biosciences) was performed using antibodies against CD3, CD8, forkhead box P3 (FoxP3; a regulatory T-cell marker), Ki-67 (a cellular proliferation marker), CD68, PD-1, PD-L1, and pan-cytokeratin, per the manufacturer’s protocol.15 16 Up to seven representative regions of interest (ROIs) per sample were annotated using the Phenochart viewer (Akoya Biosciences). For dichotomized cell-type analysis, simulated brightfield images acquired via inForm software were analyzed using an AI-based immunohistochemistry analyzer (Lunit SCOPE uIHC).17 Each ROI was segmented into cancer and stromal areas based on pan-cytokeratin staining. After image registration across markers, intratumoral immune cell densities and co-expression profiles within each ROI were quantified. The initial dataset included 336 ROIs from 88 samples across 50 available patients. ROIs were excluded from further analysis if they had insufficient cancer area (<0.01 mm2) to provide a stable denominator for cell density calculations. Furthermore, to prevent skewed results from focal hot spots, we excluded ROIs with extreme CD3+ or CD3+CD8+ cell densities, defined by a threshold of 1.5 times the IQR. The ROIs were grouped into baseline and on-treatment samples for each patient, using mean values. Patient samples were excluded if either baseline or on-treatment samples were not available after ROI exclusion. The final analysis set included paired baseline and on-treatment mIF data from 174 ROIs across 27 patients. We analyzed the median distance between pairs of tumor cells (CK+, CK+PD-L1+) and immune cells with specific markers of interest (CD8+, CD8+PD-1+, CD8+Ki-67+) for baseline and on-treatment samples. To remove the effect of extreme outliers, distance was calculated only for cells within 10 µm, and ROIs without any detected pairs were removed. Subsequently, patients that did not have both baseline and on-treatment ROIs after removal were removed from spatial analysis for the given marker pair. Statistical analysis Associations of iTIL and TC with PFS and OS were analyzed across the different rare tumors included in the study. PFS was defined as the interval from initiation of treatment with pembrolizumab until disease progression, as determined by radiographic imaging, clinical progression, or death, whichever occurred first. Patients who were alive and progression-free were censored at last imaging visit. OS was defined as the interval from the initiation of treatment with pembrolizumab until death from any cause. Patients who were alive were censored at last follow-up. PFS and OS were estimated using the Kaplan-Meier method, and group differences were evaluated with the log-rank test. HRs and 95% CIs were calculated using the Cox proportional hazards model. To account for potential confounding by histologic subtype, penalized Cox regression (penalizer=0.1) was performed with histology included as a categorical covariate. Correlations were assessed using the Pearson correlation coefficient. The Mann-Whitney U test was used to compare continuous variables between groups. Paired differences between baseline and on-treatment samples were assessed using the Wilcoxon signed-rank test. All p values were two-sided, with statistical significance defined as p<0.05. Sensitivity analyses for different thresholds were performed; details are provided in the online supplemental material. Statistical analyses were carried out using Python V.3.13. Results Study population Patient demographics are summarized in table 1 and online supplemental table 2. The median age was 56 years, and 47 patients (56%) were female. Carcinoma of unknown primary was the most common tumor type (23%), followed by other rare histologic subtypes (17%) and adrenocortical carcinoma (14%). All patients received 200 mg of pembrolizumab on day 1 of a 21-day dosing cycle. Median progression-free survival (PFS) and overall survival (OS) for all patients on the study were 2.07 months and 8.02 months, respectively. Table 1Baseline patient characteristics, n=84CharacteristicNo. (%)Median age (range), years56 (22–84)SexMale37 (44)Female47 (56)RaceWhite or Caucasian64 (76)Black or African American5 (6)Asian4 (5)Mixed1 (1)Other9 (11)Unknown1 (1)EthnicityHispanic or Latino8 (10)Not Hispanic or Latino75 (89)Unknown1 (1)ECOG09 (11)175 (89)Prior therapies04 (5)128 (33)222 (26)310 (12)≥420 (24)PD-L1 CPS<1053 (63)≥1027 (32)Not available4 (5)Tumor typeSquamous cell carcinoma of the skin*8 (10)Small cell malignancies of non-pulmonary origin†7 (8)Adrenocortical carcinoma†12 (14)Medullary renal cell carcinoma*3 (4)Carcinoma of unknown primary*19 (23)Penile carcinoma†1 (1)Vascular sarcoma*6 (7)Germ cell/testicular tumor†7 (8)Paraganglioma/pheochromocytoma†7 (8)Other rare histology*14 (17)*Tumor histologic subtypes with high iTIL density at baseline (high-iTIL tumor group).†Tumor histologic subtypes with low iTIL at baseline (low-iTIL tumor group).CPS, combined positive score; ECOG, Eastern Cooperative Oncology Group; iTIL, intratumoral tumor-infiltrating lymphocyte; PD-L1, programmed death-ligand 1. Baseline patient characteristics, n=84 Tumor histologic subtypes with high iTIL density at baseline (high-iTIL tumor group). Tumor histologic subtypes with low iTIL at baseline (low-iTIL tumor group). CPS, combined positive score; ECOG, Eastern Cooperative Oncology Group; iTIL, intratumoral tumor-infiltrating lymphocyte; PD-L1, programmed death-ligand 1. Baseline intratumoral tumor-infiltrating lymphocyte and its association with survival outcomes We examined the association between baseline intratumoral tumor-infiltrating lymphocyte (iTIL) and PFS, following patient stratification as illustrated in online supplemental figure S3. At baseline, iTIL varied across tumor cohorts, with median values ranging from 5.8 to 57.4/mm2. To address the heterogeneity of the different subtypes in this basket trial analysis, the overall cohort was categorized into “high iTIL tumor group” or “low iTIL tumor group” (five subtypes each) based on the baseline median iTIL value for each tumor subtype (figure 2A and online supplemental table 3). The high-iTIL and low-iTIL tumor groups did not differ in PFS (HR 1.02, 95% CI 0.64 to 1.63, p=0.926). The median PFS was 2.0 months for the high-iTIL tumor group (n=50) and 2.2 months for the low-iTIL tumor group (n=33; figure 2B). Association between baseline iTIL and PFS. (A) Baseline iTIL distribution across rare tumor histologic subtypes. (B) Kaplan-Meier curves stratified by tumor groups: high-iTIL tumor group versus low-iTIL tumor group. Kaplan-Meier curves stratified by iTIL ≥60 cells/mm2 vs <60 cells/mm2 (C) in the overall cohort, (D) the high-iTIL tumor group, and (E) the low-iTIL tumor group. iTIL, intratumoral tumor-infiltrating lymphocyte; PFS, progression-free survival. We then stratified the overall cohort by iTIL density. An iTIL cut-off of 60 (cells/mm2) was selected for analysis of baseline samples so that approximately 25% of the samples had iTIL ≥60/mm2. In the overall cohort of analyzable specimens (n=83), there was only a trend toward significance for improved PFS in tumors with iTIL ≥60 cells/mm2 (n=23; HR 0.62, 95% CI 0.37 to 1.06, p=0.082; figure 2C). However, in the high-iTIL tumor group (n=50), iTIL ≥60 cells/mm2 was associated with favorable PFS (HR 0.49, 95% CI 0.25 to 0.99, p=0.046; figure 2D). Median PFS was 2.9 months in tumors with iTIL ≥60 cells/mm2 (n=16; 95% CI 2.04 to not reached) and 2.0 months in tumors with iTIL <60 cells/mm2 (95% CI 1.97 to 2.69). However, in the low-iTIL tumor group, the same cut-off of baseline iTIL was not predictive of longer PFS (HR 0.94, 95% CI 0.40 to 2.22, p=0.892, figure 2E). We also examined the association of baseline iTIL with OS. The high-iTIL tumor group (n=50) and low-iTIL tumor group (n=33) did not differ in OS (HR 1.10, 95% CI 0.67 to 1.82, p=0.707). The median OS was 11.4 months in the high-iTIL tumor group and 10.6 months in the low-iTIL tumor group (online supplemental figure S4A). The same cut-off of iTIL ≥60 cells/mm2 was not associated with improved OS either in the overall cohort (median OS 7.9 vs 11.4 months, HR 0.86, p=0.600; online supplemental figure S4B), the high-iTIL tumor group (median OS 6.1 vs 11.9 months, HR 0.90, p=0.762; online supplemental figure S4C), or the low-iTIL tumor group (median OS 7.9 vs 10.6 months, HR 0.74, p=0.559; online supplemental figure S4D). To assess whether these associations were specific to the intratumoral compartment, we also quantified sTIL density and examined its association with outcomes. We observed a positive correlation between iTIL and sTIL densities (r=0.512, p<0.001). However, sTIL density was not significantly associated with PFS or OS either at baseline or change post-treatment in this cohort. As an exploratory analysis, we examined mIF data from 27 patients (15 with high-iTIL tumors and 12 with low-iTIL tumors) to better understand immune microenvironment features at baseline associated with improved PFS in tumors with iTIL ≥60 cells/mm2 in high-iTIL tumors. There was significant correlation between the iTIL derived by H&E-stained WSIs and mIF assessment of CD3+ (r=0.30, p=0.013) and CD3+CD8+ (r=0.45, p<0.001) immune cells. Although there was no significant difference in CD3+CD8+/CD3+ ratio between the high-iTIL tumor group (mean 0.22, range 0.00–0.60) and low-iTIL tumor group (mean 0.34, range 0.15–0.75; online supplemental figure S5A), FoxP3+ density was significantly higher in the low-iTIL tumor group compared with the high-iTIL tumor group (p=0.042; online supplemental figure S5B). Within the high-iTIL tumor group, patients with iTIL ≥60 cells/mm2 had higher CD8+PD-1+ (p=0.040; online supplemental figure S6A) and CD3+CD8+ (p=0.029; online supplemental figure S6B) cell density compared with patients with iTIL<60 cells/mm2. iTIL changes and their association with survival outcomes We investigated the association between changes in iTIL and survival outcomes. In patients with matched baseline and early on-treatment biopsies (n=80), an increase in normalized iTIL levels by ≥1 unit (equivalent to the on-treatment iTIL being at least double the baseline value) was significantly associated with improved OS (n=27 vs n=53; median 15.8 vs 8.0 months; HR 0.55, 95% CI 0.31 to 0.96, p=0.037; figure 3A). There was a trend towards significance suggesting that the same threshold indicated improved PFS (median 2.7 vs 2.0 months; HR 0.64, 95% CI 0.39 to 1.06, p=0.084; figure 3B). Association between iTIL density changes and survival outcomes. (A) OS and (B) PFS according to iTIL changes. (C) Representative H&E and multiplex immunofluorescence images from a patient with increased on-treatment iTIL (by ≥1 unit). Images acquired at 40× magnification. (D) In the increased on-treatment iTIL (by ≥1 unit) group, spatial proximity between tumor cells (CK+) and immune cells (CD8+ or CD8+Ki-67+) was significantly reduced after treatment. DAPI, 4’,6-diamidino-2-phenylindole, a nuclear counterstain; FoxP3, forkhead box P3, a regulatory T-cell marker; Ki-67, a cellular proliferation marker; iTIL, intratumoral tumor-infiltrating lymphocyte; OS, overall survival; panCK, pancytokeratin; PD-1, programmed cell death protein 1; PD-L1, programmed death-ligand 1; PFS, progression-free survival We also evaluated absolute iTIL change thresholds without normalization (>0, >5, >10, and >20 cells/mm2; however, no significant associations with PFS or OS were identified at any threshold tested (online supplemental table 4A). To assess whether near-zero baseline values were influencing the normalized change estimates, we repeated the analysis after excluding patients with baseline iTIL <5 cells/mm2. The association with prolonged OS (HR 0.48, p=0.017) and PFS (HR 0.59, p=0.052) was strengthened in the 75 patients remaining after this exclusion (online supplemental table 4B). To explore whether increased on-treatment iTIL (≥1 unit) is associated with enhanced immune cell responses during treatment, we assessed the distance between tumor cells and their nearest neighboring immune cells in paired mIF images from baseline and on-treatment samples in 27 patients, as previously described.18 19 Among patients with increased on-treatment iTIL (≥1 unit), the distance between tumor cells (CK+) and immune cells (CD8+ or CD8+Ki-67+) was significantly reduced during treatment, consistent with enhanced immune activation in this subset of patients (figure 3C,D). In contrast, no significant change was observed in patients with on-treatment iTIL change <1 unit. Survival analysis based on distance change between tumor and CD8+ cells (n=25) showed a longer PFS (HR 0.56, 95% CI 0.24 to 1.29, p=0.173; online supplemental figure S7A) and OS (HR 0.56, 95% CI 0.21 to 1.49, p=0.247; online supplemental figure S7B) in patients with decreased distance in the on-treatment specimens, although it was not statistically significant. Similarly, reduced distance between tumor and CD8+Ki-67+ cells (n=24) in the on-treatment specimens showed longer PFS (HR 0.52, 95% CI 0.22 to 1.23, p=0.137; online supplemental figure S7C), although not statistically significant. However, reduced distance between tumor and CD8+Ki-67+ cells was associated with prolonged OS (HR 0.33, 95% CI 0.11 to 0.99, p=0.049; online supplemental figure S7D), suggesting that immune cell proximity and activation may reflect a more favorable immune microenvironment associated with survival outcomes. Changes in TC and their association with survival outcomes We next evaluated whether changes in TC in the on-treatment specimens were associated with survival outcomes. A decrease in normalized TC by ≥0.3 units, corresponding to a 30% decrease from baseline value, was significantly associated with prolonged PFS and OS. Patients with an on-treatment TC decrease of ≥0.3 units (n=22, median PFS 2.9 months, median OS 18.9 months) had prolonged PFS (HR 0.51, 95% CI 0.29 to 0.90, p=0.019; figure 4A) and OS (HR 0.54, 95% CI 0.29 to 0.98, p=0.042; figure 4B) compared with those who did not (n=61, median PFS 2.0 months, median OS 10.6 months). Association between TC changes and survival outcomes. (A) PFS and (B) OS based on on-treatment TC decrease of more than 0.3 units. OS, overall survival; PFS, progression-free survival; TC, tumor content. Absolute TC decrease thresholds were also evaluated without normalization across a range of values (0%, 5%, 10%, and 15%). While smaller absolute decrease thresholds were not significantly associated with PFS or OS, a 15% absolute decrease in TC (n=8) was associated with improved PFS (HR 0.32, p=0.011) and OS (HR 0.18, p=0.004; online supplemental table 4A). To assess whether normalized change associations were not driven by instability at low baseline values, we performed a sensitivity analysis excluding patients with baseline TC <2% cells/mm2. Similar trends were observed in the 75 patients after this exclusion (online supplemental table 4B). Combined iTIL and TC changes during treatment and their association with survival outcomes The combination of both an increase in iTIL by ≥1 unit and a decrease in TC by ≥0.3 units was significantly associated with improved PFS and OS. Patients who met both criteria (n=11, median PFS 8.8 months) had significantly prolonged PFS (HR 0.36, 95% CI 0.17 to 0.78, p=0.009; figure 5A) compared with those who did not (n=69, median PFS 2.0 months). Median OS was also prolonged (HR 0.36, 95% CI 0.14 to 0.90, p=0.029; figure 5B) in patients who met both criteria (median OS 41.9 months) compared with those who did not (median OS 10.6 months). Association between combination of iTIL and TC changes during treatment and survival outcomes. (A) PFS and (B) OS in patients with increased on-treatment iTIL (≥1 unit) and TC decrease ≥0.3 units versus those who did not meet both criteria. iTIL, intratumoral tumor-infiltrating lymphocyte; OS, overall survival; PFS, progression-free survival; TC, tumor content. We also assessed whether these associations were confounded by histologic heterogeneity by performing penalized multivariable Cox regression adjusting for histology across all predictors (online supplemental table 5). After adjustment, the HRs remained reasonably stable. Most notably, the “Combined Change” metric (iTIL increase and TC decrease) remained a robust and statistically significant predictor for both PFS (adjusted HR 0.40 (95% CI 0.20 to 0.83), p=0.013) and OS (adjusted HR 0.42 (95% CI 0.18 to 0.95), p=0.037). Discussion Our study used a novel AI-powered computational model to analyze the intricate landscape of the TME in patients with rare tumors, a group largely underexplored, particularly in the context of ICI therapy. We showed that dynamic changes in the TME, particularly iTIL and TC, were associated with clinical outcomes in patients with rare tumors treated with pembrolizumab. While baseline iTIL levels were informative in tumors with high immune infiltration, on-treatment changes in iTIL and TC offered additional pharmacodynamic insight even after adjusting for histologic subtype. Our study showed that a high baseline iTIL (≥60 cells/mm2) was significantly associated with longer PFS, particularly in the high-iTIL tumor group. This finding is consistent with recent studies evaluating TIL density in predicting outcomes of ICI therapy across various cancer types. For instance, AI-powered spatial analysis of TILs has shown promise in identifying immune phenotypes associated with longer PFS and treatment response in advanced non-small cell lung cancer20 and biliary tract cancer.12 Li et al showed in a meta-analysis that higher levels of baseline intratumoral, stromal, or invasive marginal CD8+ TILs predict better treatment outcomes across various cancers treated with ICIs.21 Moreover, Shen et al showed that patients with an immune-inflamed phenotype, characterized by high iTIL, had significantly higher objective response rates, PFS, and OS compared with patients with a non-immune-inflamed phenotype.7 These findings highlight the importance of assessing TIL composition and distribution in informing ICI therapy outcomes. Thus, integrating AI-driven TIL analysis into clinical practice could enhance patient stratification and improve personalized immunotherapy approaches. Analysis of routine H&E-stained pathology slides offers a practical, scalable, and high-throughput approach to informing patient outcomes in a consistent and reproducible manner. Our approach of intratumoral iTIL quantification from H&E-stained WSIs was well correlated with CD3+ and CD3+CD8+ cells identified by mIF, which is not readily accessible. Furthermore, our mIF analysis showed higher FoxP3+ density in the low-iTIL tumor group, which may contribute to relatively low immune cell infiltration. Given the exploratory nature of the mIF analysis and the limited sample size, these findings should be interpreted cautiously and considered hypothesis-generating. One of the strengths of the current study is the inclusion of on-treatment biopsy specimens obtained during cycle 1, days 15–21, from the same site as baseline samples, which allowed us to capture early changes in the TME and assess treatment response early in the course of therapy. In our study, we observed that a decrease in the normalized TC (≥0.3 units) was associated with longer OS and PFS. In a study by Tapia et al, which focused on a smaller group of patients with rare tumors, a reduction in TC during treatment to about 10% residual tumor was associated with better objective responses and longer time to progression.11 However, it is worth noting that there is no established cut-off to define major tumor regression in on-treatment biopsy specimens in rare tumors, and further clinical and translational investigation is required. Additionally, we found that increased on-treatment iTIL (≥1 unit) showed a trend toward favorable survival. Spatial profiling by mIF further supported these findings, showing that patients with greater iTIL increases had reduced tumor-CD8+ and CD8+Ki-67+ cell distances after treatment, indicating enhanced immune activation.22 In line with our findings, a previous study in patients with metastatic melanoma receiving anti-PD-1 therapy showed that higher densities of intratumoral CD8+ cells in close proximity to tumor cells were associated with longer PFS and better response.18 In another study in advanced non-oncogenic-addicted non-small cell lung cancer, the number of CD8+Ki-67+ T cells significantly increased after anti-PD-1 therapy in responders, despite no significant differences between responders and non-responders in baseline and on-treatment levels of these cells.23 This further supports the notion that dynamic treatment-induced changes could be more informative and serve as a pharmacodynamic marker of effective immune activation. Furthermore, we observed that the combination of a decrease in the normalized TC (≥0.3 units) and increase in the normalized iTIL (≥1 unit) was significantly associated with longer OS and PFS. An inherent challenge of basket trials spanning multiple histologic subtypes is distinguishing treatment-related effects from differences in natural history. In our study, adjustment for histology demonstrated that the key associations, particularly the combined change metric, were not driven by differences in underlying prognosis across histologic subtypes (online supplemental table 5). Nevertheless, we cannot definitively separate predictive from prognostic effects due to lack of a concurrent immunotherapy-naive comparator arm. Future studies incorporating such comparators will be needed to clarify this distinction. The potential of iTIL dynamics and changes in TC as pharmacodynamic markers for treatment outcomes is promising. Implementing early on-treatment biopsy as a standard practice may serve as a valuable parallel to surgical resection in patients receiving neoadjuvant ICI therapy, where surgical resection typically follows preoperative treatment cycles. Importantly, such analyses can be performed using routinely collected specimens—namely, baseline diagnostic biopsy and subsequent surgical specimen—thereby obviating the need for additional tissue sampling. Given the atypical response patterns observed with immunotherapy, where conventional imaging may not reliably reflect early treatment effect,24 25 tissue-based pharmacodynamic assessment could provide complementary biological evidence to support clinical decision-making. From a clinical perspective, early on-treatment assessment of iTIL and TC dynamics could inform critical ‘go-no-go’ decisions regarding treatment continuation or modification. For instance, patients demonstrating favorable TME changes, such as increased iTIL coupled with decreased TC, may benefit from continued immunotherapy, whereas the absence of such changes could prompt earlier consideration of alternative treatment strategies, thus preventing prolonged exposure to ineffective therapy and the potential risk of immune-related adverse events. However, further investigation is warranted to determine whether iTIL dynamics and changes in TC can serve as reliable early indicators of disease-free survival. The current study has some limitations. First, the sample size was small and highly heterogeneous, reflecting the diverse histologic subtypes included under the umbrella of rare tumors. As a result, subgroup analyses may have been underpowered, and findings should be interpreted with caution in tumor-specific contexts. Second, although the AI-powered platform enabled standardized and scalable quantification of iTIL and TC, the thresholds used for increase or decrease were defined empirically (online supplemental table 6A–C) due to the exploratory nature of this basket trial and the current lack of established biological benchmarks for immune infiltration and TC dynamics in rare tumors. As such, these findings require further validation in independent cohorts. Third, although spatial immune profiling by mIF provided supportive biological insights, these analyses were exploratory and hypothesis-generating and limited to a subset of the cohort. Finally, the primary focus of this study was the intratumoral compartment; however, stromal and tumor-stroma interface metrics may also be biologically and clinically relevant. Although higher iTIL levels were associated with improved outcomes overall, immune infiltration alone may not fully capture responsiveness to ICIs, as additional tumor- and microenvironment-related factors likely contribute to therapeutic response across different tumor histologic types. While we included exploratory analysis of sTIL, interface-based metrics were beyond the scope of the current study and represent an important direction for future work to further characterize the TME in rare tumors treated with pembrolizumab. Conclusions Our study highlights the potential of AI-powered analysis of the TME to inform treatment outcomes in patients with rare tumors receiving ICIs. By integrating quantitative assessments of iTIL and TC from pretreatment and on-treatment biopsy specimens, we identified dynamic tissue-based changes that were associated with survival outcomes. These findings suggest that routine histopathologic analysis, enhanced by deep learning, may provide insights into immunotherapy response in rare tumors and support further investigation of TME dynamics as potential pharmacodynamic markers in future studies. Supplementary material Acknowledgements We thank the patients, their families, and their caregivers for participating in the study. We thank Erica Goodoff, Senior Scientific Editor in the Research Medical Library at The University of Texas MD Anderson Cancer Center, for editing this article. Footnotes Funding: Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc. provided the study drug, partially funded the phase II study of pembrolizumab in rare cancers (NCT02721732), and approved this report. This work was supported in part by the National Cancer Institute at the National Institutes of Health (P30CA016672, and 1R01CA279749-01A1 to AN); National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health through UT Health-CCTS grant (1UM1 TR0045906); and The University of Texas MD Anderson Cancer Center through the Molecular Evaluation and/or Biopsy Related Support Program (to AN). The funders were not involved in the study design, collection, analysis and interpretation of the data and in the writing of the report. Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Ethics approval: The study obtained ethics approval from the IRB of the University of Texas MD Anderson Cancer Center (Federalwide Assurance Number (FWA): 00000363 and Office of Human Research Protection (OHRP) IRB Registration Number: IRB00000121). Participants gave informed consent to participate in the study before taking part. Data availability free text: Data availability: The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request and approval from study sponsor according to available guidelines at time of request. Code availability: No custom code was used in this study. Detailed software packages with references are provided in the Methods. Data availability statement Data are available upon reasonable request. References Associated Data Supplementary Materials Data Availability Statement Data are available upon reasonable request.