Aligning computational pathology with clinical practice for colorectal cancer Authors: Elias Baumann, José F. Carreño-Martínez, Ana Leni Frei, Javier García-Baroja, Mauro Gwerder, Amjad Khan, Rina Mehmeti, Jacob Hanimann, Philipp Zens, Heather E. Dawson, Alessandro Lugli, Inti Zlobec Published: 2025 ABSTRACT The pathology report in colorectal cancer (CRC) consists of more than 20 elements defined in guidelines such as the International Collaboration on Cancer Reporting (ICCR) guidelines. Recently, computational tools have been proposed to advance the CRC diagnostic routine, yet most lack clinically validated results and focus on only three report elements. This review gives an overview of the current gaps and will contribute to aligning computational pathology with clinical practice. Introduction Digital pathology is increasingly being integrated into clinical workflows of the fast-evolving field of pathology and digitalized pathology laboratories are an increasing trend worldwide1. Once the digital infrastructure is established in these laboratories (e.g., slide scanner and viewer, laboratory information systems or image management system etc.), they can benefit from AI-based tools such as AI algorithms that assist pathologists in analyzing tissue slides. Leveraging AI may be applicable in time-consuming cases involving extensive diagnostic criteria prone to interobserver variability. With the potential of accelerating such tasks, AI could also counteract the increasing workload of pathologists2. Moreover, several studies have shown increased diagnosis confidence among pathologists when assisted with AI3–5. Computational pathology is also impacting the field of biomarker discovery, as many novel histopathological biomarkers (e.g., to identify high-risk stage II CRC patients for adjuvant chemotherapy6–8) are based on AI-derived image features9–12. AI-based tools have been developed in numerous different pathologies, including colorectal cancer (CRC). CRC is the third most common cancer worldwide with a rising incidence among younger individuals due to sedentary lifestyle and consumption of processed food13. As a result of this, and widespread screening, CRC constitutes a considerable portion of cases in a pathologist’s diagnostic routine. On one hand, the World Health Organization (WHO) Classification covers all generally accepted criteria14 for standardized disease definition and classification. On the other hand, the College of American Pathologists (CAP) and ICCR have developed evidence-based guidelines for standardizing reporting in pathology that are implemented globally15,16. In this review, we focused on the ICCR CRC guideline15 which covers both the macroscopic as well as microscopic evaluation of CRC resection specimen. As automatic methods have been predominantly developed for microscopic assessment, we restrict the review to only include reporting elements from this category. These elements are classified into “core” elements that are key for disease management, staging and prognosis, such as histological tumor type and grade, lymphatic invasion, TNM staging; and “non-core” elements, recommended but not validated by all levels of evidence, including tumor budding, mismatch repair (MMR) status, coexistent pathology, etc. While various AI algorithms have been developed to assess individual core and non-core elements, it remains unclear whether these individual algorithms can be combined for an AI-based evaluation of all elements on the digitized whole slide image (WSI) and produce an automated CRC pathology report with enough quality to increase pathologists’ accuracy and decrease time of diagnosis. With this in mind, through this systematic review we aimed to reflect on:The current landscape of deep learning (DL) in computational pathology (CPath) for CRC.Published work on the 25 elements of the ICCR CRC diagnostic report.Gaps towards automated ICCR-based reporting.The potential avenues and future steps to close these gaps. The current landscape of deep learning (DL) in computational pathology (CPath) for CRC. The current landscape of deep learning (DL) in computational pathology (CPath) for CRC. Published work on the 25 elements of the ICCR CRC diagnostic report. Published work on the 25 elements of the ICCR CRC diagnostic report. Gaps towards automated ICCR-based reporting. Gaps towards automated ICCR-based reporting. The potential avenues and future steps to close these gaps. The potential avenues and future steps to close these gaps. Methods Structure of this review Given the breadth of this review, the “Results” section is divided into three major parts, starting with general meta-results on gathered papers, and followed by meta-analysis of each element of the ICCR CRC guideline grouped here as core or non-core. The elements are sorted by number of included publications. After this analysis, and a general discussion on the most important takeaways, we include recommendations for elements where currently no readily usable computational approach has been published (Fig. 1).Fig. 1Graphical overview of the review. Graphical overview of the review. Review protocol We perform a systematic literature review by following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines17. Search strategy For the review, we defined the following search criteria, with slight adaptations specific to the search engine. Essentially, search terms included a term related to cancer, to colon/colorectal, and a term related to histopathology to focus on the specific disease. Furthermore, deep learning, artificial intelligence, computer vision, and computer assistance-related search terms were added to cover all papers that develop computer-aided approaches. To include a wide range of publications across disciplines in our study, we included Pubmed, Web of Science, Embase, and IEEE Xplore as databases. Only papers published between January 1st 2015 and September 13th 2024 were eligible for inclusion in this study. Only abstracts written in English were included. All article abstracts were screened in covidence18. The full search terms for each database can be found in Supplementary Table 1. Article screening and eligibility Articles were screened by at least two raters, with a third rater reviewing the abstract in case of disagreement. Duplicates were removed automatically by covidence18, or manually, if the automatic method failed to recognize the duplicate. Initially, abstracts were screened to assess relevance for the review according to a list of exclusion criteria. Specifically, articles should include an evaluation on human CRC, and results on histopathology image data. Articles that did not include original research (such as reviews, letters and commentaries) as well as conference abstracts were also excluded. Moreover, multimodal studies were excluded if they did not include results using exclusively histopathology/clinical data. If the approach evaluated non-hematoxylin and eosin (H&E) images, did not report slide-level or patient-level results, or only worked in a semi-automatic fashion they were also excluded. Finally, the article had to be related to or directly address one of the elements, e.g., if a study concerned itself with tissue type segmentation but is not usable for or applicable to one of the elements, it was excluded. Moreover, the developed model had to produce an output closely related to how the respective element is specified in the ICCR CRC guideline. In a second round, full texts are assessed for eligibility. The same exclusion criteria were applied. The full list of exclusion criteria can be found in Supplementary Table 2. Adding references from other sources Additionally, some articles from other sources highly relevant to this review were included. These papers were identified via references or by using search engines such as google scholar or semantic scholar for the specific ICCR CRC guideline element. Moreover, some articles did not contain the necessary information of the search criteria in their abstract, but their content was relevant to this review, therefore they were manually included. Data extraction To thoroughly assess individual studies, information was extracted from each study. Specifically, data concerning the cohorts used (including staining, number of patients, cancer stage distribution, target distribution), the training setup (dataset splits, cross validation and involvement of pathologists), the DL pipeline (input size, pre-processing, model, post-processing, training task and target) and reported results (AUC, F1, … if available). Additionally, we assessed for each study whether there were slide-level or patient-level results, if the published model is clinically approved, whether data, code, or model weights were available and whether there is any information on inference time. Data were extracted by a single reviewer and verified by a second reviewer. Many studies report multiple results across several cohorts and in multiple configurations. To properly represent those studies in our analysis, the highest reported performance metrics were extracted from each paper unless specified otherwise in the specific chapter. For multimodal models which use data that would normally not be used in a pathology report, we report the results where only histopathology data is used. Papers which cover more than one ICCR element will be included multiple times in the analysis but evaluated specifically for the respective topic. Results The search across databases yielded 4863 papers in total, of which 1636 were identified as duplicates. Additionally, 20 papers from citation search or additional manual search were included, yielding 3247 studies to be screened. In accordance with the exclusion criteria, 3066 studies were subsequently excluded, leaving 181 studies for full text screening. After full text screening, 66 studies were included in the final review (Fig. 2).Fig. 2Flowchart following the PRISMA guidelines. Flowchart following the PRISMA guidelines. Meta results Although half of the ICCR CRC report elements are covered by the 66 selected studies, most publications concern only three topics. Specifically, 39/66 (59%) of studies showed prediction results on MMR status, 16/66 (24%) on BRAF V600E mutation, and 9/66 on pN (14%). Coexistent pathologies (5/66), Grade (4/66), and TNM Stage (3/66) are covered by five or less publications. For tumor budding and perineural invasion two studies were included each, whereas one study each for pT and response to neoadjuvant therapy was extracted. Publications can cover multiple topics at once, for example, 14 publications cover both MMR status and BRAF V600E mutation. For the remaining elements of the ICCR CRC guidelines the employed search and exclusion criteria did not yield any publications (Fig. 3A).Fig. 3Overview of all included articles.A Number of articles covering a specific element of the ICCR guidelines. B Number of published articles over time, aggregated by half-year. Studies for the second half of 2024 were only retrieved until 13.09.2024. C Number of papers with accessible data, code and model weights respectively. Overview of all included articles. A Number of articles covering a specific element of the ICCR guidelines. B Number of published articles over time, aggregated by half-year. Studies for the second half of 2024 were only retrieved until 13.09.2024. C Number of papers with accessible data, code and model weights respectively. Following the general trend of increased attention and impactful contributions in CPath19, we also observe an overall increase in publications over time, even when applying strict exclusion criteria (Fig. 3B). MSI and BRAF mutation prediction studies remain the most common over the last 4 years. Lymph node metastasis prediction studies were increasingly published starting in the second half of 2022. Overall, the topic variety among published studies also increased in the past 5 years, but without a substantial growth in publication volume on frequently studied ICCR elements since 2022. Considering data, code and model weight availability (Fig. 3C), more than half of the published papers (37/66, 56%) use some publicly available dataset or make their data available, and 31/66 (47%) use The Cancer Genome Atlas (TCGA) in some form. Code was available for less publications (25/66, 38%) and publications that made model weights available were scarce (8/66, 12%). It should be noted that resources which are available “upon reasonable request” were considered unavailable, although we did not verify whether access could have been granted in these cases. This initial analysis has shown substantial variation in coverage across all reporting elements. In the next step, we will analyze published work related to the core elements of the ICCR CRC guideline. ICCR CRC guidelines: core elements Elements classified as “Core” within the ICCR CRC guidelines are considered essential for clinical management and staging of CRC patients. Excluding clinical and macroscopically retrieved information, the microscopic core elements comprise histological tumor type, tumor grade, extent of invasion (and pT), lymphatic, venous and perineural invasion, lymph node status (and pN), tumor deposits, response to neoadjuvant therapy, margin status, margin status, histologically confirmed distant metastasis (and pM), the aggregation into pTNM, and confirmatory evaluation for neuroendocrine neoplasms. Of these twelve core elements, six have been addressed in 20 published articles, with pN (9/20) and tumor grade (4/20) receiving the most attention. 5/20 publications have used publicly available data or released their datasets, 6/20 made their code available, but none published model weights (Fig. 4). Next, we present the state of research of each individual core element in order of the number of included publications.Fig. 4Overview of included articles for core elements.A Number of articles covering a specific core element of the ICCR guidelines, split by the data, code, and weight availability for each topic. B Number of published articles over time, aggregated by half-year. Not shown are elements where no publications passed exclusion criteria. Overview of included articles for core elements. A Number of articles covering a specific core element of the ICCR guidelines, split by the data, code, and weight availability for each topic. B Number of published articles over time, aggregated by half-year. Not shown are elements where no publications passed exclusion criteria. Regional lymph nodes (pN) and lymph node status Regional lymph node (LN) status is crucial for determining adjuvant chemotherapy and cancer staging. The TNM classification of regional LNs (pN) includes: NX (nodes cannot be assessed), N0 (no metastasis), N1a (1 positive node), N1b (2–3 positive nodes), N1c (tumor deposits in soft tissue without nodal metastasis), N2a (4–6 positive nodes), and N2b (≥7 positive nodes)20. A LN is considered positive if it contains at least one metastatic lesion larger than 2 mm20. Pathologists aim to evaluate at least twelve nodes per case, though this number can be influenced by factors such as specimen length, patient age, or neoadjuvant therapy15,21. Smaller metastatic foci are classified as isolated tumor cells, and while their presence should be reported according to ICCR guidelines, they do not affect the lymph node status. Data and methods Several approaches have been used to automatically screen LN slides for metastases, listed in Table 1. Most methods are focused on metastasis detection/segmentation or based on metastasis detection further predicting the slide-level labels, whether positive or negative. Model evaluation was performed on both external (30% of the studies) and internal cohorts (70% of the studies). Clinical validation or pathologist involvement was reported in ~30% of studies.Table 1Extracted studies for lymph node status and pNStudyYearValidation Cohorts (WSI)Reported Resultsclinical validationExternal ValidationDataCodeWeightsChuang9420211 (1000)0.997 AUC-----Mainovskaya9520221 (-)0.863 DICE-----Tsuneki9620221 (928a)0.911 Sens. 0.931 Spec. 0.928 ACC-✓---Bándi9720231 (44)0.979 AUC--✓--Khan2420234 (172–2803)1.000 Sens. 1.000 Spec.✓✓---Tan9820231 (169)0.977 AUC 0.953 Sens. 0.935 PPV 0.935 ACC-----Kindler2320232 (74–288)0.940 AUC 0.820 Sens. 0.980 Spec. 0.906 PPV 0.952 ACC✓----Giammanco9920247 (652–4911)1.000 Sens. 0.990 Spec. 0.991 ACC-✓---Huang2220241 (137)0.8877F1b 0.8024 Sens.b 0.969 ACCb✓-✓✓-a187 CRC WSI.bPathologist performance assisted by model. Table 1Extracted studies for lymph node status and pNStudyYearValidation Cohorts (WSI)Reported Resultsclinical validationExternal ValidationDataCodeWeightsChuang9420211 (1000)0.997 AUC-----Mainovskaya9520221 (-)0.863 DICE-----Tsuneki9620221 (928a)0.911 Sens. 0.931 Spec. 0.928 ACC-✓---Bándi9720231 (44)0.979 AUC--✓--Khan2420234 (172–2803)1.000 Sens. 1.000 Spec.✓✓---Tan9820231 (169)0.977 AUC 0.953 Sens. 0.935 PPV 0.935 ACC-----Kindler2320232 (74–288)0.940 AUC 0.820 Sens. 0.980 Spec. 0.906 PPV 0.952 ACC✓----Giammanco9920247 (652–4911)1.000 Sens. 0.990 Spec. 0.991 ACC-✓---Huang2220241 (137)0.8877F1b 0.8024 Sens.b 0.969 ACCb✓-✓✓-a187 CRC WSI.bPathologist performance assisted by model. Extracted studies for lymph node status and pN a187 CRC WSI. bPathologist performance assisted by model. Huang et al.22 used a human-in-the-loop AI system (nuclei.io) to personalize ML models for colorectal LN metastasis detection. The approach improved accuracy, F1 score, sensitivity, and especially detection of isolated tumor cells, while significantly reducing evaluation time for fellows and negative cases. Similarly, Kindler et al.23 trained a Deep Neural Network tool achieving high pixel-level accuracy. Clinical testing showed high sensitivity (0.990), strong interobserver agreement (κ = 0.94), and significantly reduced review times for pathologists. Furthermore, Khan et al.24 demonstrated 100% agreement between their ensemble model and expert pathologists in a study of 217 cases. Clinical employability The reviewed studies show high sensitivity and specificity across the board for metastasis detection. However, the automation of pN staging requires the additional step of counting the number of positive lymph nodes in a slide. This can be difficult due to tissue preparation steps where some large lymph nodes may be cut in half and may be present twice in the same slide, or on multiple different slides. Correctly re-identifying the same lymph node and ensuring positive lymph nodes not being counted twice therefore remains an unsolved problem. One approach to addressing this issue could be a better link between macroscopic tissue preparation and AI-based image analysis. For example, lymph node placement information could be additionally stored, but consistent tissue inking may also be an avenue to solve the problem automatically. Histological tumor grade Tumor grade describes the level of differentiation of tumor cells where a higher grade indicates a loss of glandular organization14,25. In accordance with the WHO classification, the ICCR CRC guidelines classify the grade by the least differentiated component of the neoplastic lesion which then results in either low (previously low to moderately differentiated) or high (previously poorly-differentiated) tumor grade. Data and methods The retrieved studies use both private and public datasets, but with the exception of one study26, none validate their results externally (Table 2). Given the subjectivity and high interobserver variability of tumor grade assessment27, a consensus label based on multiple pathologists might be more reliable as ground truth. Some studies28,29 used two raters, with a third pathologist to resolve any conflicts. Yet, none of the retrieved studies reported agreement between pathologists and deep learning models. Most studies rely on tile classification26,28,29 to directly predict tumor grade. Rathore et al.30 use morphological and texture-based image features aggregated in a support vector machine classifier and majority voting. They achieve comparable performance to tile classifiers on their dataset. For tile classification methods, the overall strategies include multiple instance learning26 for end-to-end training, and supervised tile-based training28,29. No studies aggregate at the patient level, although clinical practice typically considers the highest grade across multiple slides. This is particularly relevant for TCGA-based studies28, as WSIs were not selected for grading and may not be representative of the overall tumor grade.Table 2Extracted studies for tumor grade. Normal (NORM), Well-differentiated (WD), Moderately-differentiated (MD), Poorly-differentiated (PD)StudyYearClassesValidation Cohorts (WSI)Reported ResultsExternal ValidationDataCodeWeightsRathore302019WD, MD, PD1 (174)0.98 AUC 0.928 Sens. 0.958 Spec. 0.95 Acc-✓--Schrammen262021NORM, WD + MD, PD1 (805)0.751 AUC 0.877 Sens. 0.528 Spec. 0.599 Acc 0.513 F1✓-✓-Chen282022NORM, WD, MD, PD1 (108)0.981,0.983, 0.963, 0.963 AUC-✓--Soldatov292022NORM, WD, MD, PD1 (357)0.981, 1.0, 0.933, 1.0 Spec. 0.553, 0.0, 0.577, 0.0 Sens. 0.838, 0871, 0.876, 0.974 Acc 0.696, 0.0, 0.6, 0.0 F1---- Table 2Extracted studies for tumor grade. Normal (NORM), Well-differentiated (WD), Moderately-differentiated (MD), Poorly-differentiated (PD)StudyYearClassesValidation Cohorts (WSI)Reported ResultsExternal ValidationDataCodeWeightsRathore302019WD, MD, PD1 (174)0.98 AUC 0.928 Sens. 0.958 Spec. 0.95 Acc-✓--Schrammen262021NORM, WD + MD, PD1 (805)0.751 AUC 0.877 Sens. 0.528 Spec. 0.599 Acc 0.513 F1✓-✓-Chen282022NORM, WD, MD, PD1 (108)0.981,0.983, 0.963, 0.963 AUC-✓--Soldatov292022NORM, WD, MD, PD1 (357)0.981, 1.0, 0.933, 1.0 Spec. 0.553, 0.0, 0.577, 0.0 Sens. 0.838, 0871, 0.876, 0.974 Acc 0.696, 0.0, 0.6, 0.0 F1---- Extracted studies for tumor grade. Normal (NORM), Well-differentiated (WD), Moderately-differentiated (MD), Poorly-differentiated (PD) Clinical employability Data accessibility varies significantly across studies. While Rathore et al.30 use the publicly available GLaS dataset and Chen et al.28 TCGA-COAD and TCGA-READ, and Soldatov et al.29 only make their dataset available upon request. Beyond TCGA and challenge datasets, independent validation is limited due to restricted access to additional clinical data. Reproducibility is another concern, as most studies do not share code or model weights. Only Schrammen et al.26 have a publicly available repository. Despite these limitations, the availability of public datasets (e.g., GLaS) enables further research and tool development. It should also be noted that the publications included in this review follow the three-tiered grading system (including well-, moderately-, and poorly-differentiated categories). This system can be directly mapped to the two-tier system, since according to the ICCR guidelines, low-grade corresponds to well- and moderately-differentiated tumors, while high-grade includes poorly differentiated cases. One study26 is the exception, where no reclassification is possible. pTNM Pathological TNM staging is the combined assessment of pT, pN, and pM. The TNM stage directly informs treatment decisions including whether a resection is necessary, or whether additional adjuvant therapy needs to be performed. It remains the most important prognostic factor in CRC diagnosis, with Stage I patients’ 5-year survival being over 90% for colon and rectum, but Stage IV 11-15% respectively31. While TNM Staging could be algorithmically constructed out of evaluated T, N, and M, all three retrieved approaches directly predict TNM stage from the primary, with only one publication also considering lymph node slides32 (Table 3). Two publications rely on TCGA COAD (and READ) for the evaluation, and Kumar et al.33 additionally include an internal cohort for training and validation. Two methods used graph convolution to spatially integrate tissue tile embeddings into slide level predictions32,34, whereas the last followed the more traditional method of tumor area detection, and subsequent tile classification and averaging based on morphological features of nuclei33.Table 3Extracted studies for pTNMStudyYearValidation Cohorts (WSI)Reported ResultsActionable ResultsExternal ValidationDataCodeWeightsLevy3220211 (17)0.92 AUC (Stage I, II vs III, IV)---✓-Kumar3320222 (130, 197)0.78–0.81 AUC (Stage II vs IV)✓--✓-Pei3420221 (459)0.89 Acc, 0.89 F1 (tile level)--✓-- Table 3Extracted studies for pTNMStudyYearValidation Cohorts (WSI)Reported ResultsActionable ResultsExternal ValidationDataCodeWeightsLevy3220211 (17)0.92 AUC (Stage I, II vs III, IV)---✓-Kumar3320222 (130, 197)0.78–0.81 AUC (Stage II vs IV)✓--✓-Pei3420221 (459)0.89 Acc, 0.89 F1 (tile level)--✓-- Extracted studies for pTNM Clinical employability Only Kumar et al.33 generate clinically actionable results, but they use their model for the specific task to differentiate between metastatic colon cancer and local colon cancer. While Pei et al.32 also show slide-level results, they never evaluate them or report a tile-aggregation strategy. Levy et al.32 predict low (I, II) vs high (III, IV) stage, but their approach does evaluate the slide for invasion depth so further development could lead to a comprehensive TNM analysis. Taken together, none of the retrieved papers demonstrate an end-to-end pipeline that can reliably assess the TNM stage (I–IV) from histological slides. While there are cases where directly predicting the risk of metastasis from the primary might be justified (e.g., what is the risk of a missed metastasis), simpler approaches following the UICC TNM guidelines by aggregating information about local, regional and distant spread may be more clinically relevant for routine reporting. Perineural invasion Perineural Invasion (PNI) is defined as tumor growth along a nerve, where the tumor surrounds at least one-third of the nerve’s perimeter and invades any of its layers35. PNI is reported as either present or absent, and its presence has been linked to poor prognosis, particularly in Stage II36. Data and methods Both included studies employ a similar approach for detecting PNI (Table 4). First, nerve and tumor regions are separately segmented, and subsequently a boundary between the two regions is identified. While Jung et al.37 use a private dataset for training, Han et al.38 trained on the PAIP2021 dataset comprising of 240 WSIs from multiple cancer types. The dataset was used for the PAIP2021 challenge39 to detect PNI.Table 4Extracted studies for perineural invasionStudyYearValidation cohort (WSI)Reported ResultsExternal validationDataCodeWeightsHan3820221 (50)ACC: 0.81-✓--Jung3720221 (-)AUC: 0.92---- Table 4Extracted studies for perineural invasionStudyYearValidation cohort (WSI)Reported ResultsExternal validationDataCodeWeightsHan3820221 (50)ACC: 0.81-✓--Jung3720221 (-)AUC: 0.92---- Extracted studies for perineural invasion Clinical employability Neither of the included studies provide the code or the weights of the DL algorithms that produced the reported results. Moreover, the validation cohorts, compared to studies on other topics, e.g., the previously discussed lymph node metastasis detection, are comparatively small. Beyond this reproducibility gap, the methods do report a binary output for the perineural invasion in accordance with the ICCR CRC guidelines. pT and extent of invasion The extent of invasion of the primary tumor is the first part of the TNM staging and reflected in pT. It is assessed by considering the different components of the colon wall and the presence of tumor within them. pT ranges from pT0 (no tumor) to pT4 (tumor invades other organs or perforation into the visceral peritoneum) and has a direct impact on the treatment of a patient20. Only one study that automates the prediction of pT in CRC passed the exclusion criteria40 (Table 5). In this study, the authors use two validation cohorts, one internal with 38 patients, one external with 42 patients. For classification into pT1-4, they first segment the tumor area, then use a patch-based classifier to predict pT for each tile to finally choose the highest pT value as the final prediction. The model reaches AUCs up to 0.93 on the internal test set and 0.90 on the external validation cohort.Table 5Extracted studies for extent of invasion and pTStudyYearClassesValidation Cohorts (Patients)Reported ResultsExternal validationDataCodeWeightsJu402022pT 1–42 (38, 42)0.87, 0.90, 0.93, 0.75 AUC 0.80, 0.83, 0.88, 0.71 Sens. 0.85, 0.86, 0.88, 0.75 Spec.✓--- Table 5Extracted studies for extent of invasion and pTStudyYearClassesValidation Cohorts (Patients)Reported ResultsExternal validationDataCodeWeightsJu402022pT 1–42 (38, 42)0.87, 0.90, 0.93, 0.75 AUC 0.80, 0.83, 0.88, 0.71 Sens. 0.85, 0.86, 0.88, 0.75 Spec.✓--- Extracted studies for extent of invasion and pT Yet the validation sets only include few patients, while data, code, and weights are not publicly available. They do show results with explicit cutoffs, but only in 1-vs-all settings without multiclass evaluation. Therefore, more work is needed to develop robust approaches for automatic pT assessment with larger validation cohorts and investigating challenges in real world datasets. Response to neoadjuvant therapy Tumor regression gradings (TRGs) are used to assess the response to neoadjuvant chemoradiotherapy (nCRT) by measuring the extent of tissue changes caused by the treatment41. This is a common assessment step in locally advanced rectal cancer treated with neoadjuvant therapies. Multiple TRGs exist but all rely on the assessment of tumor to fibrosis ratio within the tumor bed41. Data and methods We retrieved only one study that predicted regression after nCRT in a fully automated way42 (Table 6). The authors used an MIL approach with gated attention weight normalization with a final bilinear attention for multi-scale feature fusion to classify responders from non-responders according to slide-level labels. Yet, they validated the model on a breast cancer metastasis dataset rather than rectal cancer which reduces the value of the validation.Table 6Extracted studies for response to neoadjuvant therapyStudyYearClassesValidation cohort (Patients)Reported ResultsExternal validationDataCodeWeightsZhang422022Good response (TRG0-1), bad response (TRG2-3)1 (30)0.759 AUC 0.717 ACC---- Table 6Extracted studies for response to neoadjuvant therapyStudyYearClassesValidation cohort (Patients)Reported ResultsExternal validationDataCodeWeightsZhang422022Good response (TRG0-1), bad response (TRG2-3)1 (30)0.759 AUC 0.717 ACC---- Extracted studies for response to neoadjuvant therapy Clinical employability There exist many different systems for TRG, and in the included study, the AJCC system43 was used. However, the four AJCC TRG categories were merged into a binary classification task which does not correspond to the ICCR CRC guidelines. The reviewed study trained for slide-level predictions but does not discuss aggregation strategies for patient-level predictions. However, this post-processing aggregation step is needed for successful clinical deployment, as pathologists report a patient-level TRG. Due to the challenges caused by the TRG definitions, we did not find any study that presents a clinically ready algorithm to assess tumor regression in rectal cancer, yet a tissue type segmentation model fine-tuned to predict remaining tumor and fibrosis could replicate every TRG scheme. ICCR CRC guidelines: non-core elements The non-core elements of the ICCR CRC dataset assess additional features that provide valuable prognostic insights and are clinically relevant but are not frequently used in patient management or not yet validated. They include measurement beyond the muscularis propria, MLH1 promoter methylation, tumor budding, coexistent pathology, MMR status, and BRAF V600E mutation. Of these six elements, only four have been addressed in the published studies included in this review. MMR status and BRAF mutation, in particular, have received significant attention, with most MMR-related (28/39) and BRAF-related (11/16) studies using TCGA as their primary data source. For these elements, code to reproduce results is often available (21/48, 44%), and some studies provide access to trained model weights (8/48, 17%). Interestingly, since similar methods are applied for MMR- and BRAF-status prediction, 14 studies investigate both biomarkers simultaneously. In contrast, tumor budding and coexistent pathology remain comparatively unexplored and overall lack publicly available data, code, or access to weights (Fig. 5). In the following section, we examine the current research landscape for each of the non-core elements.Fig. 5Overview of included articles for non-core elements.A Number of articles covering a specific non-core element of the ICCR guidelines, split by the data, code, and weight availability for each topic. B Number of published articles over time, aggregated by half-year. Not shown are elements where no publications passed exclusion criteria. Overview of included articles for non-core elements. A Number of articles covering a specific non-core element of the ICCR guidelines, split by the data, code, and weight availability for each topic. B Number of published articles over time, aggregated by half-year. Not shown are elements where no publications passed exclusion criteria. MMR status The DNA mismatch repair (MMR) system is responsible to correct DNA replication mismatches44. The mutation of its major genes and the resulting instability of microsatellite repeat sequences is referred to Microsatellite Instability (MSI)45. MSI is a biomarker in CRC, guiding clinical decisions for intermediate risk Stage II and Stage IV CRCs46,47, and represents an indicator for lynch syndrome screening—a hereditary condition characterized by mismatch repair gene mutations. Routine diagnostic testing is typically performed using Immunohistochemistry (IHC) and both ASCO and ESMO guidelines mandate dMMR testing for all new CRC diagnoses48. Recent DL models have demonstrated that they can predict MSI status directly from H&E slides without the need for IHC. Data and methods Common datasets for evaluation are the publicly available TCGA COAD/READ cohorts (80.4%), the PAIP2020 challenge dataset (26.3%), and DACHS (21.05%) (Table 7). Of note here is that different publications use different subsets of TCGA COAD/READ so while results have been generated on some of the same slides metrics cannot be properly compared. At the same time, many papers only used one (33.3%) or two (35.3%) cohorts in their analysis and the remainder used more (29.4%).Table 7Extracted studies for microsatellite instabilityStudyYearValidation Cohorts (Patients)Reported AUCTCGA COAD/READ (FFPE) AUCaActionable ResultsExternal ValidationDataCodeWeightsKather5020193 (360, 378)0.840.77-✓✓✓-Schmauch10020201 (465)0.810.81--✓✓-Lee10120201 (57)--✓-✓--Yamashita10220213 (100, 479)0.930.84✓✓✓--Ke10320211 (747)----✓--Pai10420212 (136, 230)0.852-✓----Lee10520213 (91, 500)0.970.89✓✓✓--Schrammen2620212 (889, 2448)0.91-✓✓-✓-Saillard5520213 (47, 555)0.970.88-✓✓--Bilal6020213 (47, 428)0.980.86-✓✓--Bustos10620211 (1788)0.9-✓----Echle10720229 (51, 2452)0.960.91✓✓✓✓✓Saldanha10820225 (632, 2448)0.89--✓✓✓-Laleh10920222 (632, 2448)0.930.89--✓✓-Fujii6720223 (248, 986)0.92----✓✓Jiang11020225 (78, 441)0.880.88✓✓✓--Guo11120221 (396)0.760.76-----Lou11220223 (5, 144)0.94-✓----Teichmann6120221 (840)0.94---✓--Lv11320221 (868)0.920.92--✓--Rubinstein11420221 (313)0.710.71--✓--Ding11520223 (94, 306)0.840.84-✓✓✓-Cao11620232 (47, 360)0.960.91✓✓✓--Guo6220232 (423, 1065)0.910.91✓✓---Chang11720232 (305, 1579)0.950.72✓✓✓✓-Niehues11820232 (2190, 2448)0.94-✓--✓✓Yuan11920231(491)0.93---✓--Tsai6820232 (628, 927)0.880.88✓✓✓✓-Wagner56202317 (35, 2452)0.960.83✓✓✓✓-Cen6320232 (425, 1138)0.920.92--✓✓-Kim12020231 (118)0.92---✓--Saillard5520234 (47, 554)0.960.93✓✓✓--Zamanitajeddin6420242 (47, 428)0.990.88-✓✓--Roth5220242 (110, 632)0.890.89--✓✓-Liu12120241 (352)0.69b------Shen12220241 (100)0.850.85--✓--Gustav12320244 (27, 2039)0.940.87✓✓✓✓-Vorontsov5320242 (360, 482)0.96---✓✓✓Wang5420243 (75, 437)0.880.87-✓✓✓✓aResults based on some form of training only on TCGA if available.bPublication uses multimodal data to improve results, reported are pathology only. Table 7Extracted studies for microsatellite instabilityStudyYearValidation Cohorts (Patients)Reported AUCTCGA COAD/READ (FFPE) AUCaActionable ResultsExternal ValidationDataCodeWeightsKather5020193 (360, 378)0.840.77-✓✓✓-Schmauch10020201 (465)0.810.81--✓✓-Lee10120201 (57)--✓-✓--Yamashita10220213 (100, 479)0.930.84✓✓✓--Ke10320211 (747)----✓--Pai10420212 (136, 230)0.852-✓----Lee10520213 (91, 500)0.970.89✓✓✓--Schrammen2620212 (889, 2448)0.91-✓✓-✓-Saillard5520213 (47, 555)0.970.88-✓✓--Bilal6020213 (47, 428)0.980.86-✓✓--Bustos10620211 (1788)0.9-✓----Echle10720229 (51, 2452)0.960.91✓✓✓✓✓Saldanha10820225 (632, 2448)0.89--✓✓✓-Laleh10920222 (632, 2448)0.930.89--✓✓-Fujii6720223 (248, 986)0.92----✓✓Jiang11020225 (78, 441)0.880.88✓✓✓--Guo11120221 (396)0.760.76-----Lou11220223 (5, 144)0.94-✓----Teichmann6120221 (840)0.94---✓--Lv11320221 (868)0.920.92--✓--Rubinstein11420221 (313)0.710.71--✓--Ding11520223 (94, 306)0.840.84-✓✓✓-Cao11620232 (47, 360)0.960.91✓✓✓--Guo6220232 (423, 1065)0.910.91✓✓---Chang11720232 (305, 1579)0.950.72✓✓✓✓-Niehues11820232 (2190, 2448)0.94-✓--✓✓Yuan11920231(491)0.93---✓--Tsai6820232 (628, 927)0.880.88✓✓✓✓-Wagner56202317 (35, 2452)0.960.83✓✓✓✓-Cen6320232 (425, 1138)0.920.92--✓✓-Kim12020231 (118)0.92---✓--Saillard5520234 (47, 554)0.960.93✓✓✓--Zamanitajeddin6420242 (47, 428)0.990.88-✓✓--Roth5220242 (110, 632)0.890.89--✓✓-Liu12120241 (352)0.69b------Shen12220241 (100)0.850.85--✓--Gustav12320244 (27, 2039)0.940.87✓✓✓✓-Vorontsov5320242 (360, 482)0.96---✓✓✓Wang5420243 (75, 437)0.880.87-✓✓✓✓aResults based on some form of training only on TCGA if available.bPublication uses multimodal data to improve results, reported are pathology only. Extracted studies for microsatellite instability aResults based on some form of training only on TCGA if available. bPublication uses multimodal data to improve results, reported are pathology only. There are mixed approaches to ground truth, as for MSI/dMMR testing, several different methods are clinically appropriate. 33.3\% relied on both IHC and PCR for the same or different datasets, 49.0% only rely on PCR, 11.8% only on IHC, 3.9% on other methods. None of the papers using IHC as gold standard are reporting whether these were single observer results, even though there is some disagreement in manual IHC dMMR protein analysis49. Most MSI prediction models rely on tile level embeddings that are aggregated to a slide-level score. Before that step, many approaches pre-detect the tumor area (43.6%), and then subsequently usually add an average (41.1%), majority (23.5%) or top-k (17.6%) aggregation on top. Models that did not pre-segment the tumor tissue (56.4%) commonly relied on an attention mechanism (50%) to weight tile results for the final slide-level MSI score. Foundation models have included MSI prediction in CRC on datasets like TCGA-CRC-DX50,51 as a benchmark task52–54, which will lead to more results in this topic, though the small scope of the evaluation may not bring additional insights on clinical translatability. Clinical employability In MSI/dMMR prediction, the DL model either needs to predict MSI as well as a diagnostic test (e.g., at least as well as IHC) or needs to be used as a screening tool. In both cases, a cutoff needs to be defined. In this review, 47.1% of papers defined some cutoff at least on a subset of the validation data, but only 13.7% demonstrated some applicable cutoff viable for screening (e.g., at least 90% sensitivity) in a clinical setting. One method is clinically approved as a pre-screening tool with CE-IVD certification55, yet so far, no tool can reach the same specificity as IHC or PCR testing while maintaining 95% sensitivity (Saillard et al.55 46% Spec. at 98% Sens., Wagner et al.56 61% Spec. at 95% Sens.). The current state of papers therefore indicates that the MSI category is advanced, yet tools can only be employed as screening tools with gold-standard testing necessarily following the screening as the specificity is comparatively low. Therefore, either DL models can be further improved to have similar sensitivity and specificity as gold-standard testing, or this step necessarily needs to be deferred to a non-H&E evaluation. BRAF V600E BRAF mutation testing plays an important role in Lynch syndrome identification after MSI testing. On the other hand, microsatellite stable BRAF mutant colorectal cancers are particularly aggressive, yet the mutation also increases responsiveness to EGFR inhibitor therapy57. The non-core element in the CRC ICCR guidelines specifically concerns the oncogenic V600E mutation, which represents ~90% of BRAF mutant cases in a large trial58. Notably, BRAF and MSI cases share similarities in both morphological features and epidemiological patterns59. Data and methods As there is considerable overlap between the setup, task, and patients with MSI and BRAF mutation, publications frequently predict both (Table 8). Therefore, different subsets of the TCGA cohort are the also most prevalent datasets (75%), followed by the DACHS cohort (31.3%).Table 8Extracted studies for BRAF V600E mutationStudyYearValidation Cohorts (Patients)Reported AUCTCGA COADREAD (FFPE) AUCActionable ResultsExternal ValidationDataCodeWeightsFu6520201 (146)0.5710.571--✓✓✓Bilal6020211 (502)0.790.79--✓--Schrammen6220211 (2075)0.82-✓--✓-Chen6620221 (414)0.7290.729--✓--Fujii6720223 (248–986)0.859-✓--✓✓Guo11120221 (396)0.8970.897-----Laleh10920222 (534, 2081)0.860.808-✓✓✓-Saldanha10820224(500, 2075)0.77--✓✓✓-Teichmann6120221 (268)0.750.75--✓--Cen6320232 (500, 1026)0.780.78-✓✓✓-Guo6220232 (500, 1026)0.80.77✓✓---Niehues11820232 (1478, 2081)0.82--✓-✓✓Tsai6820232 (591, 906)0.710.71✓✓✓✓-Wagner5620239 (107, 2081)0.880.78-✓✓✓-Zamanitajeddin6420231 (502)0.790.79--✓--Wang5420242 (233, 559)0.7320.732-✓✓✓✓ Table 8Extracted studies for BRAF V600E mutationStudyYearValidation Cohorts (Patients)Reported AUCTCGA COADREAD (FFPE) AUCActionable ResultsExternal ValidationDataCodeWeightsFu6520201 (146)0.5710.571--✓✓✓Bilal6020211 (502)0.790.79--✓--Schrammen6220211 (2075)0.82-✓--✓-Chen6620221 (414)0.7290.729--✓--Fujii6720223 (248–986)0.859-✓--✓✓Guo11120221 (396)0.8970.897-----Laleh10920222 (534, 2081)0.860.808-✓✓✓-Saldanha10820224(500, 2075)0.77--✓✓✓-Teichmann6120221 (268)0.750.75--✓--Cen6320232 (500, 1026)0.780.78-✓✓✓-Guo6220232 (500, 1026)0.80.77✓✓---Niehues11820232 (1478, 2081)0.82--✓-✓✓Tsai6820232 (591, 906)0.710.71✓✓✓✓-Wagner5620239 (107, 2081)0.880.78-✓✓✓-Zamanitajeddin6420231 (502)0.790.79--✓--Wang5420242 (233, 559)0.7320.732-✓✓✓✓ Extracted studies for BRAF V600E mutation Most approaches apply stain normalization (68.8%) and tile extraction (100%) as preprocessing steps. Many workflows (31.3%) subsequently identify tumor tissue using a tissue classification algorithm60–64. The majority of publications implement MIL-based approaches (81.3%), differing in their tile aggregation approaches (averaging: 23.1%, top k tiles: 30.7%, other: 30.9%). These MIL-based approaches are commonly built on either CNN-based (31.3%) or attention-based (31.3%) architectures. Recent publications leverage foundation models (18.8%) with either averaging-based65,66 or attention-based54 aggregation strategies. Clinical employability Only 43.8% of publications conducted an external validation of their algorithm, while 56.3% utilized more than one cohort. Openly available datasets were included in 68.8% of all studies. The code is available in 62% of all publications, and model weights are only available in 31.25% of studies. Furthermore, most studies did not differentiate between BRAFV600E mutations and all other BRAF mutations (87.5%). In contrast, Fuji et al. utilized a BRAFV600E-specific cohort, whereas Tsai et al. reported both BRAFV600E-specific and unspecific results. Only four studies reported actionable results26,62,67,68, meaning that a label (BRAFWT/BRAFmut) rather than a mutation probability was assigned to each WSI. Only two studies report prediction sensitivities higher than 90%26. Of those, only one study reached an accuracy higher than 0.526. Of all 16 included publications, only Guo et al.62 and Schrammen et al.26 report fully integrated pipelines with results based on a clinically appropriate cutoff (sensitivity >90%). However, Guo et al.62 neither shared code nor weights, and Schrammen et al.26 did not use any openly available datasets on which results could be reproduced. The current state of BRAFV600E prediction is unsolved and mostly serves as a validation task for multi-purpose models. Almost all (88%) studies reporting BRAFmut -status report MSI-status as well. As those features are highly correlated, it is expected that models will perform similarly for both tasks. However, there exist cases that are non-correlating, which are especially interesting to pathologists69. None of the studies investigated performance in such cases, although previous work has demonstrated a confounding effect between the biomarkers with impact on predictive performance70. Furthermore, most studies do not utilize a clinically appropriate cutoff, making it difficult to evaluatetheir performance in a clinical setting. Coexistent pathology Under coexistent pathology, all other identified lesions should be listed. This could be additional synchronous carcinomas, polyps or e.g., IBD, other dysplastic normal tissue, subsumed under other lesions. Published work usually focuses on precursor lesions or the correct classification of polyps, but those models could be applied to e.g., correctly classify additional polyps in the same case. Additionally, because of the diversity of possible findings within this section, the models that have been developed so far only focus on one of the items that should be reported. Data and methods The reviewed studies utilized datasets derived from biopsy or polyp resections rather than oncologic resection specimens (Table 9). Across studies, there is lack of access to the development datasets.Table 9Extracted studies for response to coexistant pathologyStudyYearTaskClassesValidation Cohorts (WSI)Reported ResultsExternal ValidationDataCodeWeightsKorbar1242017Polyp class.NORM, HP, SSP, SA, TA, TVA1 (239)0.96, 0.81, 0.82, 0.90, 0.90, 0.92 Sens. 0.90, 0.90, 0.95, 0.93, 0.96, 0.95 ACC✓---Wei732020Polyp class.TA, TVA, HP, SSA, (NORM)2 (157,238)0.89, 0.97, 0.82, 0.78 Sens. 0.95, 0.95, 0.96, 0.98 Spec.✓-✓-Perlo712021Dysplasia gradingHP, LG, HG1 (72)0.80, 0.73, 0.78 Sens. 0.99, 0.78, 0.88 Spec. 0.90, 0.76, 0.83 ACC-✓--Neto722022Dysplasia gradingNORM, LG, HG3 (100,1009)1.0 Sens. 0.99 ACC✓✓--Soldatov292022Polyp class.NORM, TA, VA1 (357)0.55, 0.32, 0.28 Sens. 0.98, 0.79, 0.96 Spec. 0.84, 0.66, 0.87 ACC----NORM normal tissue, HP Hyperplastic Polyp, TA Tubular Adenoma, TVA Tubulo-Villous Adenoma, SSP Sessile Serrated Polyp, SA Serrated Adenoma, SSA Sessile Serrated Adenoma, LG low grade dysplasia, HG high grade dysplasia. Table 9Extracted studies for response to coexistant pathologyStudyYearTaskClassesValidation Cohorts (WSI)Reported ResultsExternal ValidationDataCodeWeightsKorbar1242017Polyp class.NORM, HP, SSP, SA, TA, TVA1 (239)0.96, 0.81, 0.82, 0.90, 0.90, 0.92 Sens. 0.90, 0.90, 0.95, 0.93, 0.96, 0.95 ACC✓---Wei732020Polyp class.TA, TVA, HP, SSA, (NORM)2 (157,238)0.89, 0.97, 0.82, 0.78 Sens. 0.95, 0.95, 0.96, 0.98 Spec.✓-✓-Perlo712021Dysplasia gradingHP, LG, HG1 (72)0.80, 0.73, 0.78 Sens. 0.99, 0.78, 0.88 Spec. 0.90, 0.76, 0.83 ACC-✓--Neto722022Dysplasia gradingNORM, LG, HG3 (100,1009)1.0 Sens. 0.99 ACC✓✓--Soldatov292022Polyp class.NORM, TA, VA1 (357)0.55, 0.32, 0.28 Sens. 0.98, 0.79, 0.96 Spec. 0.84, 0.66, 0.87 ACC----NORM normal tissue, HP Hyperplastic Polyp, TA Tubular Adenoma, TVA Tubulo-Villous Adenoma, SSP Sessile Serrated Polyp, SA Serrated Adenoma, SSA Sessile Serrated Adenoma, LG low grade dysplasia, HG high grade dysplasia. Extracted studies for response to coexistant pathology NORM normal tissue, HP Hyperplastic Polyp, TA Tubular Adenoma, TVA Tubulo-Villous Adenoma, SSP Sessile Serrated Polyp, SA Serrated Adenoma, SSA Sessile Serrated Adenoma, LG low grade dysplasia, HG high grade dysplasia. It is important to highlight that the same task has different GT across studies. While only two studies address dysplasia grading, they incorporate different classes. Perlo et al.71 add Hyperplastic Polyp (HP) to their dysplasia categories, which is not used by Neto et al.72. Similar observations apply for polyp classification. Slide-level classification is achieved by Multiple Instance Learning by Neto et al.72, while Perlo et al.71 aggregated tile classification results from a CNN. Wei et al.73 defined class-specific thresholds to categorize whole slide images (WSIs). Clinical employability Variability in ground truth definitions limits model comparability and clinical integration. Differences in annotation standards hinder consistent evaluation, while a lack of publicly available datasets and independent validation cohorts restricts reproducibility. Only Neto et al.72 provide accessible validation data, but broader external validation remains scarce. Existing studies focus on polypectomy specimens rather than oncologic resection samples, since the studies are designed for application to polyp classification in the colorectal cancer screening workflow. While existing models can assess isolated polyps, their performance on primary tissue slides, which encompass all layers of the colon, and thereby unseen tissue types, is uncertain. Furthermore, several coexisting pathologies remain unexplored. For instance, no studies were identified that automatically evaluate (IBD) using H&E-stained slides. While Rymarczyk et al.74 present a model to score Crohn’s disease and ulcerative colitis, it relies on prior classification into one of these two categories, highlighting a lack of methods for fully automatic reporting of this section of the guidelines. Tumor budding Tumor budding (TB) is defined as presence of small clusters of up to four cancer cells at the tumor invasive front in a tissue section75. TB assessment in cancer follows the standardized guidelines of the International Tumor Budding Consensus Conference (ITBCC) and is performed in hotspot regions (0.785 mm²) of H&E-stained tissue sections76. This scoring has clinical implications, particularly for pT1 and stage II CRC patients, as high TB is associated with poor prognosis75,77. Identifying and counting TBs on H&E slides is time-consuming and prone to inter-observer variability making it an ideal task to be automated by DL models. Data and methods Most published studies on AI-assisted TB assessment rely on IHC staining, patch-level, single-center training and validation of the model. After exclusion criteria, two studies following the standardized ITBCC assessment pipeline were left (Table 10).Table 10Extracted studies for tumor buddingStudyYearValidation cohorts (WSI)Reported ResultsExternal validationTumor bud detection and countDataCodeWeightsLu7820221 (40)0.96 AUC 0.94 Sens. 0.83 Spec. 0.86 PPV-✓---Bokhorst8020232 (15, 18)0.98 Sens. 0.44 PPV 0.58 F1✓✓--- Table 10Extracted studies for tumor buddingStudyYearValidation cohorts (WSI)Reported ResultsExternal validationTumor bud detection and countDataCodeWeightsLu7820221 (40)0.96 AUC 0.94 Sens. 0.83 Spec. 0.86 PPV-✓---Bokhorst8020232 (15, 18)0.98 Sens. 0.44 PPV 0.58 F1✓✓--- Extracted studies for tumor budding The study by Lu, et al.78 uses faster RCNN79 for bud detection. Ground truth was established manually on H&E tissue slides and the model’s evaluation time was significantly faster than manual assessment. On the other hand, Bokhorst et al.80 created a pipeline integrating a U-Net81 model for tissue segmentation and HoVer-Net82 for nuclei detection at the invasive front. The tumor bulk and invasive front were identified using a convex-hull algorithm. Their strategy was validated on data from four different medical centers. Clinical employability Both models present different approaches in TB detection and counting in H&E WSIs that seem promising to reduce time and inter-observer variability among pathologists. However, there are a number of limitations to be addressed before being able to deploy such models in the diagnostic routine, such as small validation datasets, an overestimation of TB count especially in necrotic areas, and an inability to distinguish pseudo budding83. Discussion In this extensive systematic review, we have analyzed published work on deep- and machine learning for digital pathology image analysis by putting each study into context of the ICCR CRC guidelines, which considers primary carcinoma from the colon and rectum, neuroendocrine carcinomas (NECs) and neuroendocrine-non-neuroendocrine neoplasms (MiNENs). Particular focus was placed on whether studies could be implemented in a clinical workflow to assist pathologists by automating the CRC pathology report. Despite the breadth of available research, our findings reveal substantial gaps: numerous ICCR CRC guideline elements remain completely unaddressed. Moreover only 38% of the studies provided access to code, and just 12% to model weights. This underscores a broad challenge in the field: without transparent and accessible implementations, independent validation on external datasets and direct comparison between methods remain limited. In this sense, ensuring availability of code, model weights, and data is to accelerate progress toward clinical integration. One of the main observations from our analysis is that despite the critical role of core elements of the ICCR CRC guidelines for clinical decision making, most research efforts focus on automating the reporting of non-core elements. This discrepancy highlights the challenge in aligning the research objectives of the CPath community with the practical needs of the clinic. Many core tasks are not simple, and some are not investigated at all. Instead, they are often overlooked due to a perceived lack of methodological novelty, despite being central to real-world diagnostic workflows. In this context, it is important to acknowledge that the recent advances in AI have powered the development of chatbots84, or computer vision tools that can work as an end-to-end diagnostic system or even predict patient survival85. While these models offer the appeal of simplified outputs by bypassing intermediate diagnostic variables, they risk overlooking the interpretability and clinical validation that the elements of the ICCR guidelines offer. In contrast, building trustworthy and reliable models for ICCR-defined elements aligns more closely with current diagnostic practice. Such an approach may facilitate adoption by pathologists and laboratories, as more transparent model outputs can be verified and causally understood86. On the other hand, in established diagnostic criteria, it can take decades to alter established criteria or add additional ones, such that end-to-end prognostic models may not have a chance to ever be adopted as standard-of-care. Another pressing issue is the generalized lack of multi-cohort evaluation. Models are often developed and validated on single-institution datasets or within closed consortia, which limits generalizability. Differences in staining protocols, scanning equipment, and population demographics are known to significantly affect performance, and are a source of bias87,88, yet multi-site evaluation that would better reflect real-world conditions is rarely done, primarily due to data access barriers. As a result, many studies use TCGA data, but H&E slides in TCGA were never selected for diagnostic evaluation. Therefore, validation results on TCGA have only limited transferability to other real-world cohorts. This review excluded studies which could not report slide level results, but frequently research groups do not have access to clinical data or whole slide annotations to verify their results. Efforts such as BIGPICTURE89 could help facilitate more clinically relevant research by making such data available. Challenges have been crucial in pushing the boundaries of deep learning research in computational pathology39,90,91. Even after challenges have been completed, a number of studies still use the provided benchmark for further evaluation of new approaches. Yet, many challenges and benchmarks focus on a specific problem which may not be translatable to a diagnostic element and often do not require methods to develop deployable tools but rather highly specific methods to beat the benchmark. Moreover, inference time, economic considerations and required infrastructure are usually not addressed either, such that these challenges are often only useful for deep learning researchers, but not for clinicians and laboratory staff. On a positive note, challenges provide a standardized evaluation dataset which enables fair method comparison, whereas even validation on TCGA is often done non-transparently without publishing the used data split, list of included slides in train and test set, or the exclusion criteria. This review focused on the microscopic evaluation of CRC resection specimens and exclusively on H&E based evaluation, yet a diagnostic report would also contain macroscopic information or use other data modalities. Macroscopic analysis includes taking photos of the specimen and textual information exists, so theoretically, deep learning models could be developed for this purpose, too. The integration of extra information becomes relevant for addressing some core elements such as margin status, where information from macroscopic reports is needed to identify the relevant slides that must be analyzed. The use of LLM models is already being discussed to extract this type of structured information from unstructured macroscopic pathology reports92. Furthermore, modalities such as immunohistochemistry or transcriptomics, as well as techniques not yet incorporated into standardized reporting guidelines, may offer important diagnostic or prognostic insights. Deep learning-based analysis of IHC, while more expensive since additional stainings are necessary, may be more reproducible and would still be automatically assessed92. Novel machines analyzing tissue before embedding could also further pave the way for additional deep learning tools improving diagnostic routine by creating different imaging data93. Multimodal strategies represent an exciting direction for future research, and we encourage further work to explore how integrating different biological layers might enhance diagnostic accuracy and clinical decision-making. However, our decision to exclude other modalities was guided by a desire to evaluate clinically relevant models that align with current diagnostic workflows and internationally recognized reporting standards. Nonetheless, we recognize that future work should explore how computational pathology can incorporate emerging features and modalities to further enhance clinical decision-making. Given the high incidence of CRC across pathology departments, the development of computational tools for its diagnostic assessment can be impactful. The relative abundance of CRC cases enables the collection of large, diverse datasets suitable for training and validating robust algorithms. Implementing such tools could support standardization and efficiency in routine diagnostics, thereby reducing pathologist workload and allowing greater focus on diagnostically complex or rare cases. Nevertheless, despite the potential for clinical impact, key diagnostic tasks remain underexplored. Morphologic subtyping is one example. While some studies attempt subtype prediction, none yet capture the full spectrum of histologic variants. Future work should focus on leveraging large, diverse datasets and advanced modeling strategies to enable accurate and reproducible subtype classification, including rare morphologies. There is also a lack of automated approaches for pT3 sub-stratification, a feature with prognostic value that could support more precise staging. Automated approaches using tissue segmentation may help quantify this feature consistently, but further research is warranted to assess its utility and feasibility in large-scale clinical applications. Beyond model performance, the successful implementation of computational pathology tools in clinical practice requires careful consideration of practical factors such as infrastructure, integration with laboratory information systems, maintenance, and the need for specialized expertise. These elements can involve significant costs and resource allocation. As such, any clinical adoption should be guided by a balanced evaluation of potential benefits versus operational demands. Highlighting not just technical feasibility but also cost-effectiveness and scalability is essential to ensure meaningful and sustainable clinical impact. Regardless, the major results of this review are the identification of several gaps in the ICCR criteria that current research has not yet covered. Table 11 gives an overview over all topics not yet comprehensively addressed by current work. For each topic, a potential approach is outlined, together with the required data and the potential benefit from automatically analyzing this element.Table 11Addressing the remaining ICCR CRC guideline elementsCore ElementsWhatHowDataBenefitsHistological SubtypeClassifying histological subtypes could be feasible using slide-level aggregation models to capture relevant morphological patterns. Although some studies have predicted individual subtypes, no existing algorithm differentiates between all subtypes. Rare subtypes could benefit from anomaly detection as a screening tool.Evaluation data needs to sufficiently cover all subtypes, which may be challenging due to rarity of some subtypes15. Given a heterogeneous dataset, a pretrained foundation model could reduce data requirements.Many subtypes require a percentage (e.g., at least 50% mucinous) of the tumor volume for subtype assigment14. Computer aided quantification improves reproducibility and can highlight rare morphologies or calculate ratios.Lymphatic and venous invasion (LVI)Detecting LVI on H&E can be challenging, and pathologists often rely on elastin stains to improve accuracy125. Simple MIL models may fall short, as contextual cues, such as an artery without a nearby vein, aid detection14. Leveraging the spatial relationship of vessels and tumor regions may be more promising.ICCR CRC guidelines differentiate between intramural and extramural venous invasion. Therefore, datasets should include diverse vessel sizes, invasion patterns, and anatomical locations.LVI is a critical risk factor for stage I/II colorectal cancer126,127, but suffers from low interobserver agreement, particularly on H&E125. Sensitive DL models could improve diagnosis in early-stages.Margin StatusMargin status is typically assessed only when tumor is macroscopically identified within 30 mm of the resection margins. In such cases, slides from the margin are examined for tumor presence. A simple tile-based classifier may be sufficient for this task, although identifying the cutting structure could be necessary to estimate the tumor-to-margin distance.Generally, in most pathology archives, many blocks covering resection margins should be available, but a positive resection margin is rare. Actively annotating the cutting edge may be helpful for correct identification.This procedure is often highly relevant for interoperative consultation when pathologists assess cryosections to assess whether a tumor has been completely removed.Tumor DepositsTumor deposits are located in the fatty tissue on primary or lymph node WSI. To distinguish them from lymph node metastases, the boundary complexity could help, and primary tissue should not be misclassified either.A diverse dataset including tumor deposits, lymph node metastases, and samples with and without primary tissue on the same section.Detecting tumor deposits within fatty tissue is part of pN-staging, and leads to pN1 classification even without lymph node metastases.Distant metastasis (pM)The metastasis needs to be correctly identified regardless of the tissue it is in. A tile-based classifier could already be sufficient to detect the metastasis and confirm whether the morphology indeed matches a colorectal cancer morphology.A dataset spanning tissue from multiple organs including healthy examples and samples with other pathologies that are not CRC metastases. This would help the model learn to identify metastasis and not other changes in healthy tissue.pM is often only a confirmatory step, as the tissue with the believed metastasis has then already been extracted. But for a end-to-end second opinion by an AI model, pM is a central block of CRC diagnosis.Neuroendocrine confirmationFor neuroendocrine neoplasms, the CRC ICCR guidelines ask for specific markers (synaptophysin and chromogranin A), and a Ki-67 proliferation index, which could also be predicted individually, though detecting the subtypes successfully would be a first step.As neuroendocrine neoplasms are quite rare, even among colorectal neoplasms128, it may be difficult to gather enough examples to train a classifier and have the ground truth for both the stains and the Ki-67 index.Neuroendocrine neoplasms are rare but have poor prognosis129. However, identifying neuroendocrine neoplasms in the first place is unsolved.Non-Core ElementsWhatHowDataBenefitsMeasurement of invasion beyond muscularis propriaTo measure the invasion depth cases with tumor invasion beyond the muscularis propria (MP) must first be identified. Then, a tissue segmentation model can detect tumor regions beyond the MP and measure the orthogonal distance to it.The necessary samples for this type of project are common, as any CRC > pT2 can be included. Pathologist annotations for the continuous measurement would be necessary on a sufficiently large cohort.Evidence suggests130–132 that pT3 can be stratified into pT3a and pT3b with additional prognostic value. A large-scale investigation using an automatic method could provide further insight.MLH1 promoter methylationGiven the current state of MSI prediction where high specificity with clinical grade sensitivity cannot be reached, trying to make an even more specific prediction than the general MSI label, is a high-risk project. Table 11Addressing the remaining ICCR CRC guideline elementsCore ElementsWhatHowDataBenefitsHistological SubtypeClassifying histological subtypes could be feasible using slide-level aggregation models to capture relevant morphological patterns. Although some studies have predicted individual subtypes, no existing algorithm differentiates between all subtypes. Rare subtypes could benefit from anomaly detection as a screening tool.Evaluation data needs to sufficiently cover all subtypes, which may be challenging due to rarity of some subtypes15. Given a heterogeneous dataset, a pretrained foundation model could reduce data requirements.Many subtypes require a percentage (e.g., at least 50% mucinous) of the tumor volume for subtype assigment14. Computer aided quantification improves reproducibility and can highlight rare morphologies or calculate ratios.Lymphatic and venous invasion (LVI)Detecting LVI on H&E can be challenging, and pathologists often rely on elastin stains to improve accuracy125. Simple MIL models may fall short, as contextual cues, such as an artery without a nearby vein, aid detection14. Leveraging the spatial relationship of vessels and tumor regions may be more promising.ICCR CRC guidelines differentiate between intramural and extramural venous invasion. Therefore, datasets should include diverse vessel sizes, invasion patterns, and anatomical locations.LVI is a critical risk factor for stage I/II colorectal cancer126,127, but suffers from low interobserver agreement, particularly on H&E125. Sensitive DL models could improve diagnosis in early-stages.Margin StatusMargin status is typically assessed only when tumor is macroscopically identified within 30 mm of the resection margins. In such cases, slides from the margin are examined for tumor presence. A simple tile-based classifier may be sufficient for this task, although identifying the cutting structure could be necessary to estimate the tumor-to-margin distance.Generally, in most pathology archives, many blocks covering resection margins should be available, but a positive resection margin is rare. Actively annotating the cutting edge may be helpful for correct identification.This procedure is often highly relevant for interoperative consultation when pathologists assess cryosections to assess whether a tumor has been completely removed.Tumor DepositsTumor deposits are located in the fatty tissue on primary or lymph node WSI. To distinguish them from lymph node metastases, the boundary complexity could help, and primary tissue should not be misclassified either.A diverse dataset including tumor deposits, lymph node metastases, and samples with and without primary tissue on the same section.Detecting tumor deposits within fatty tissue is part of pN-staging, and leads to pN1 classification even without lymph node metastases.Distant metastasis (pM)The metastasis needs to be correctly identified regardless of the tissue it is in. A tile-based classifier could already be sufficient to detect the metastasis and confirm whether the morphology indeed matches a colorectal cancer morphology.A dataset spanning tissue from multiple organs including healthy examples and samples with other pathologies that are not CRC metastases. This would help the model learn to identify metastasis and not other changes in healthy tissue.pM is often only a confirmatory step, as the tissue with the believed metastasis has then already been extracted. But for a end-to-end second opinion by an AI model, pM is a central block of CRC diagnosis.Neuroendocrine confirmationFor neuroendocrine neoplasms, the CRC ICCR guidelines ask for specific markers (synaptophysin and chromogranin A), and a Ki-67 proliferation index, which could also be predicted individually, though detecting the subtypes successfully would be a first step.As neuroendocrine neoplasms are quite rare, even among colorectal neoplasms128, it may be difficult to gather enough examples to train a classifier and have the ground truth for both the stains and the Ki-67 index.Neuroendocrine neoplasms are rare but have poor prognosis129. However, identifying neuroendocrine neoplasms in the first place is unsolved.Non-Core ElementsWhatHowDataBenefitsMeasurement of invasion beyond muscularis propriaTo measure the invasion depth cases with tumor invasion beyond the muscularis propria (MP) must first be identified. Then, a tissue segmentation model can detect tumor regions beyond the MP and measure the orthogonal distance to it.The necessary samples for this type of project are common, as any CRC > pT2 can be included. Pathologist annotations for the continuous measurement would be necessary on a sufficiently large cohort.Evidence suggests130–132 that pT3 can be stratified into pT3a and pT3b with additional prognostic value. A large-scale investigation using an automatic method could provide further insight.MLH1 promoter methylationGiven the current state of MSI prediction where high specificity with clinical grade sensitivity cannot be reached, trying to make an even more specific prediction than the general MSI label, is a high-risk project. Addressing the remaining ICCR CRC guideline elements This overview may serve as a starting point for a number of investigations, yet it is important to mention that each element may have its own set of specific problems that only arise once researchers investigate the element. Limitations Limiting the scope of the review to papers with slide-level results and automated analysis ignores a sizable number of papers that could be translated to functional pipelines at the WSI-level with little additional work. Yet, these excluded articles are also not less important for the progress in the field. Moreover, the employed search terms potentially excluded relevant papers if the specified terms were not present in the abstract. By selecting ICCR CRC guidelines over alternatives like CAP, we emphasized certain topics while omitting emerging biomarkers such as tumor-stroma ratio, tumor border configuration, and immune response. Additionally, the focus on H&E-based algorithms ignores immunohistochemistry-based approaches, which represent promising alternatives in CRC diagnostics. Strengths This systematic review includes publications on deep- and machine-learning AI-models for image analysis in pathology according to elements of the ICCR CRC dataset15 - a globally recognized, evidence-based and standardized pathology reporting guideline. It provides the readers with the current landscape of research in CPath towards automating routine tasks of pathologists during CRC assessment according to ICCR guideline. In the results section we provide a general analysis on the trend of publications over the years and per each element of the guideline. We also present information on the data, code and weight availability among included publications. Following the general overview, we dissect publications individually, summarize their key results in separate tables as well as critically evaluate the clinical readiness of the proposed models, for each core and non-core element of the ICCR CRC guideline. Moreover, to guide future research efforts, we present a dedicated table containing all underexplored core and non-core elements, where we propose a training approach, the dataset types and the anticipated benefits of utilizing such models. Finally, we emphasize practical aspects for researchers, such as the necessity for transparency in computational requirements, availability of code, weights and data - essential for reproducibility and improvement of AI tools with clinical applicability. All in all, this review should facilitate alignment between future research efforts in CPath and clinical practices as well as encourage new studies to address current gaps and underexplored research areas. Conclusion This systematic review provides a comprehensive overview of the current state of CPath research to address elements of the diagnostic report in CRC. It reveals the limited maturity of the field and lack of alignment between research and clinical practices and emphasizes the need for developing approaches for various gaps while evaluating existing methods in clinical contexts. The findings could also lead to new benchmarks for novel models that assess entire diagnostic workflows rather than isolated tasks. Finally, this review underscores the inherent complexity of CRC diagnosis performed by pathologists, which represents just one of many diseases they routinely evaluate every day. Supplementary information Supplementary Information Supplementary Information