Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registry

Lovedeep Gondara, Gregory Arbour, Raymond Ng, Jonathan Simkin, Shebnum Devji · 2025

Read Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registry by Lovedeep Gondara, Gregory Arbour, Raymond Ng, Jonathan Simkin, Shebnum Devji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09991.txt.

Introduction

Lovedeep Gondara and co-authors share lessons from bringing language AI into a cancer registry in a 2025 paper. The British Columbia Cancer Registry holds records that staff once read and coded by hand. Software that pulls facts from clinical text could save large amounts of time. Putting such software into daily service, though, proved harder than building the models.

The team distills lessons from the full project cycle. Frame the problem around office goals, not just model scores, they advise. Build in small steps with doctors, users, and model builders at one table. Plain models and mixed approaches often beat fancy ones, and data quality needs constant care. Human checks and audits catch errors, and staff training builds trust in the tools.

Catalog record

CREATORLovedeep Gondara, Gregory Arbour, Raymond Ng, Jonathan Simkin, Shebnum Devji
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-09991

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging AI Innovation and Healthcare Needs: Lessons Learned from Incorporating Modern NLP at The BC Cancer Registry Lovedeep Gondara, Gregory Arbour, Raymond Ng, Jonathan Simkin, Shebnum Devji Abstract Automating data extraction from clinical documents offers significant potential to improve efficiency in healthcare settings, yet deploying Natural Language Processing (NLP) solutions presents practical challenges. Drawing upon our experience implementing various NLP models for information extraction and classification tasks at the British Columbia Cancer Registry (BCCR), this paper shares key lessons learned throughout the project lifecycle. We emphasize the critical importance of defining problems based on clear business objectives rather than solely technical accuracy, adopting an iterative approach to development, and fostering deep interdisciplinary collaboration and co-design involving domain experts, end-users, and ML specialists from inception. Further insights highlight the need for pragmatic model selection (including hybrid approaches and simpler methods where appropriate), rigorous attention to data quality (representativeness, drift, annotation), robust error mitigation strategies involving human-in-the-loop validation and ongoing audits, and building organizational AI literacy. These practical considerations, generalizable beyond cancer registries, provide guidance for healthcare organizations seeking to successfully implement AI/NLP solutions to enhance data management processes and ultimately improve patient care and public health outcomes. 1 Introduction Cancer registries are vital in healthcare systems, providing critical data for cancer surveillance, research, and public health policies [ 1 ] . However, the manual processes involved in data extraction from pathology reports and clinical notes are often labor-intensive, time-consuming, and prone to inconsistencies, leading to backlogs and potential delays in data availability [ 2 ] . With the recent successes in machine learning (ML) [ 3 ] , especially in Natural Language Processing (NLP) [ 4 ] , Language models (LMs) offer a promising foundation to automate and enhance these processes. Our team has decades of experience in ML and NLP. For the last four years, we have been working on training and deploying various NLP models at British Columbia Cancer Registry (BCCR) for a wide array of tasks, including report classification, information extraction, and text segmentation. Our work primarily covers pathology reports but also includes imaging documents such as CT and PET scan modalities. We have published multiple papers focused on this area, such as for tumor reportability classification [ 5 ] , tumor group classification [ 6 ] , relapse detection [ 7 ] , report segmentation [ 8 ] , auditing language models in a production setting [ 9 ] , etc. Even with the extensive experience, we faced several challenges bringing the solutions from inception to production and deployment. This paper presents lessons learned from our experience implementing different NLP techniques to automate information extraction from pathology reports at BCCR. We aim to provide practical insights for healthcare organizations considering similar deployments, focusing on key considerations across the project lifecycle. Our findings are not limited to cancer registries, but are generalizable across the healthcare landscape, where we emphasize the crucial role of clear problem definition, taking an iterative approach to the final solutions, interdisciplinary co-design and collaboration, and a pragmatic approach to model selection and deployment. While NLP is technically a subfield of AI focusing on text and language data, for the purposes of this discussion on applying various NLP techniques in healthcare, the terms “AI”, “ML” and “NLP” are often used interchangeably. 2 The Project Goals and Milestones The success of any applied AI project hinges on a clearly defined problem statement and well-articulated project goals and milestones. In settings where many people from different backgrounds, roles and organizations are involved, aligning all stakeholders on the project as early as possible to have a common understanding is paramount. 2.1 Success Beyond Accuracy: Focus on Business Goals It is crucial to recognize that the ultimate goal of implementing automated NLP approaches in a healthcare setting is not solely to maximize model accuracy, but to achieve specific business objectives such as improving efficiency, reducing costs, or enhancing patient care. For instance, in a cancer registry context, the initial, poorly defined goal might be: Create a NLP solution that is at least 99% accurate for identifying reportable tumors. This solely focuses on accuracy and lacks specificity. A more effective problem statement, aligned with business needs, would be: Address the existing two-year backlog in pathology coding, a labor-intensive and expensive process, by automating a method to achieve accuracy similar to current processes in an automated pipeline for reportable tumor identification in pathology reports. This revised example clearly articulates the motivation (backlog reduction), the desired outcome tied to reality (accuracy similar to current processes), and the specific task (reportable tumor identification). Such goals are best established when the end-users (like the cancer registry staff in the example above) and the ML experts collaborate to understand the limitations of current methods and propose a viable target for the ML system. A real example from BCCR is that the discussions between subject matter experts and ML experts lead to the creation of an NLP solution that minimizes false positives while keeping false negatives below 2%. This shows that strategic alignment of projects is fundamental to realizing business value and achieving desired outcomes [ 10 ] . Furthermore, ML experts measure “model” success by false negatives, false positives, area under the receiver operating characteristic (ROC) curve, etc. However, to a business organization, the success of a “solution” based on a “model” can be measured by other metrics such as the amount of waiting time to complete the task, and/or the amount of actual time to complete the task. For waiting time, an AI tool can operate 24/7, while human-centric workflows may be interrupted by staff holidays, shortage, etc. For completion time, an NLP solution may significantly reduce the amount of time taken, even if human post-processing is needed. Let us return to the BCCR reportable tumor identification example. Let us say that for the sake of illustration, for every 1000 true positive pathology reports, the current system adds 400 false positive reports. Thus, tumor registrars would need to review 1400 reports to filter out the false positives. With an NLP tool, let us say that the false positives are reduced to 100 reports, and the tumor registrars would need to review 1100 reports. But the gain is more because the NLP tool marks up the sentences on why the report is flagged as reportable. Thus, a tumor registrar instead of spending, say, 1 minute to read one report, may now only need to spend 30 seconds on it. In other words, without the tool, human experts spend 1400 minutes to process 1400 reports; with the NLP tool, this task is now completed in \dfrac{1100}{2} = 550 minutes. Thus, to BCCR, while reducing the false positive rate from 40% to 10% is great, the time reduction from 1400 minutes to 550 minutes is even more valuable. 2.2 Inclusion/Exclusion Criteria and Outcome Variables A well-defined problem statement needs to be clear regarding inclusion and exclusion criteria for data, as well as a precise definition of the outcome variable. For example, when extracting information from pathology reports, specifying the types of reports included (example: breast cancer, invasive surgeries only, for specific date ranges) and the exact entities to be extracted (example: tumor grade, stage, biomarkers) is essential.

Read the complete text