Distillation of a Locally Deployable 8B Language Model for Lung Histology Abstraction: A 33-Report Proof-of-Concept Study Authors: Dian Tri Wiyanti, Min-Huei Hsu, Daniel C A Nugroho Published: 2026 ABSTRACT AbstractBackgroundThe manual data abstraction process for lung cancer registries is a labor-intensive undertaking that often results in registry lag. While cloud-based Large Language Models (LLMs) exhibit high reasoning acuity, their reliance on third-party APIs poses significant data privacy risks and conflicts with institutional data sovereignty.PurposeThe objective of this study is to develop and provide a pilot proof-of-concept validation of a secure, tiered hybrid architecture for the localized four class extraction of lung cancer pathology reports using domain-specific Small Language Models (SLMs).MethodsWe employed an instruction-based distillation paradigm, transferring clinical reasoning from Gemini 2.0 Flash (Teacher) to a quantized Med42-8B model (Student) alongside a generalist comparator (Llama-3-8B). A six-level WHO Tiered Logic framework based on 2021 standards was implemented as a semantic constraint to stabilize reasoning. The training process was executed for a single epoch on a consumer-grade GPU (NVIDIA RTX 3060) utilizing 2256 MIMIC-IV records. To evaluate real-world clinical feasibility and execution stability, the localized framework was deployed on a broader cohort of 134 independent, de-identified clinical records from Taipei Medical University Hospital (TMUH). From this cohort, a targeted pilot subset of 33 specialized pathology reports underwent a rigorous, double-blind clinical forensic audit by human experts to establish an absolute performance and diagnostic safety benchmark.ResultsAcross the 134-case deployment cohort, the specialized model achieved a 100% JSON syntactic compliance rate without execution failures. Within the audited 33-case clinical subset, the distilled domain-specific Med42-8B model demonstrated a diagnostic accuracy of 90.91% and a macro F1-score of 0.92 (substantial agreement with experts, κ = 0.86), significantly outperforming the general-purpose Llama-3-8B student model (accuracy: 78.79%, macro F1-score: 0.79). Qualitative safety audits on confounding cases (eg, thymoma and metastatic sarcoma) demonstrated that the specialized model successfully adhered to Level-1 gatekeeper logic to prevent clinical safety hazards, whereas the generalist model suffered from critical logical hallucinations. The implementation of a deterministic StopOnProperJSON mechanism effectively resolved issues related to generative looping, ensuring data integrity during local execution.ConclusionThis exploratory proof-of-concept study demonstrates the technical feasibility of localized, domain-distilled SLMs as a privacy-preserving foundation for four-class lung histology abstraction. While statistical generalizability remains bounded by the pilot audit cohort, the framework offers an accessible, privacy-compliant methodological blueprint that warrants prospective validation across larger multicenter cohorts.Keywords: data sovereignty, clinical informatics, pathology, on-device AI, information extraction, data privacy Introduction Lung cancer represents a substantial global health challenge. According to GLOBOCAN 2020 data, it has the highest incidence and mortality rates of any cancer worldwide.1 The accuracy of the diagnosis and tumor staging recorded in pathology reports is critical to the effective management of lung cancer patients.2,3 As genomic and histopathological data become increasingly intricate, international organizations such as the International Collaboration on Cancer Reporting (ICCR) and the World Health Organization (WHO) are emphasizing the significance of structured reporting.4–7 This approach ensures the consistency of clinical information, facilitating its utilization for decision-making processes.4–6 However, in daily clinical practice, the majority of pathology data is still stored in unstructured narrative text formats that require manual extraction by medical personnel.8,9 This manual process is both extremely time-consuming and prone to human error, which hinders workflow efficiency within multidisciplinary healthcare systems.10–12 In clinical practice, registry lag and extraction errors are not merely administrative inconveniences; they pose direct threats to patient safety. Multidisciplinary lung cancer care heavily depends on real-time, highly granular histopathological classifications to guide targeted therapies and surgical interventions.2,3 Abstracting incorrect lineages such as misclassifying small cell carcinoma as non-small cell lung cancer, or failing to identify pre-invasive lesions like adenocarcinoma in situ can lead to catastrophic clinical diagnostic errors, inappropriate treatment regimens, and delayed patient follow-up.2,3 Prior clinical natural language processing (NLP) studies have attempted to automate registry abstraction using rule-based algorithms or early clinical text mining systems.2,3 While successful in highly structured contexts, these legacy methods struggle with linguistic variability and lack complex clinical reasoning.2,3 Recently, clinical researchers have leveraged proprietary cloud-based Large Language Models (LLMs) such as GPT-4 for cancer registry extraction, demonstrating high classification accuracy.13–15 However, these cloud-based approaches introduce critical barriers: prohibitive operational API costs, lack of local model transparency, and substantial data privacy risks when transmitting Protected Health Information (PHI) outside hospital firewalls.16–18 The use of third-party APIs frequently conflicts with hospital data sovereignty principles that require sensitive data processing to remain within the institution’s firewall.10–12 Conversely, when clinical institutions attempt to deploy open-source Small Language Models (SLMs) locally, these compact architectures frequently suffer from structural hallucinations and logical inconsistencies when handling complex medical terminology.18–20 There remains an urgent need to engineer lightweight, locally executable systems capable of maintaining high diagnostic precision and patient safety on consumer-grade hardware.18–20 To address these limitations, this exploratory pilot study develops and provides a proof-of-concept validation of a secure, tiered hybrid architecture for the automated four class abstraction of lung cancer pathology reports. We implement an instruction-based synthetic distillation paradigm, transferring clinical reasoning capabilities from a frontier teacher model (Gemini 2.0 Flash) to a lightweight, domain-specific student model (Med42-8B) alongside a generalist comparator (Llama-3-8B).19–21 Crucially, to anchor model reasoning within international oncological standards and mitigate diagnostic hallucinations, we introduce a six-level WHO Tiered Logic framework based on the 2021 WHO Classification of Thoracic Tumours and ICCR guidelines.7,22 This framework operates as a hierarchical series of decision gates: starting with Level 1 (The Gatekeeper) to filter out non-pulmonary or metastatic lesions; Level 2 to determine major lineages (NSCLC vs SCLC); and transitioning through Levels 3 to 6 to handle detailed adenocarcinoma classifications, squamous cell variants, and neuroendocrine tumors. By embedding this taxonomy directly into the model’s instruction space, we impose semantic constraints that stabilize the reasoning boundaries of a compact model.19–21 To evaluate whether localized SLMs can preserve institutional data sovereignty without sacrificing clinical precision, the framework utilizes 2256 MIMIC-IV records for instruction fine-tuning on a single consumer GPU (NVIDIA RTX 3060 12GB).23 We then validate our framework using a multi-institutional clinical dataset from Taipei Medical University Hospital (TMUH). This validation is structured as a dual-layered framework: (1) a local deployment stability cohort (N = 134) to assess system throughput, offline execution latency, and formatting stability; and (2) a clinical forensic audit subset (n = 33) strategically extracted via purposive stratified sampling to establish an absolute performance benchmark and evaluate model safety against complex clinical trap cases.17,24 Materials and Methods Data Sources and Pre-Processing This study utilized retrospective electronic medical record (EMR) data from two distinct sources to evaluate the framework’s robustness across different clinical documentation styles. The primary training corpus was sourced from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database.23 To ensure high-fidelity synthetic supervision-based distillation, the cohort focused exclusively on primary lung malignancies. Initial cohort identification was performed using ICD-10 filters for invasive bronchus/lung carcinoma (C34.*) and carcinoma in situ (D02.2.*), capturing 4286 diagnosis records from 2294 unique patients. Merging records by admission identifier (hadm_id) reconstructed 2333 chronological clinical trajectories. To ensure clean instruction tuning, the narrative text corpus underwent rigorous automated quality-control filtering. Documents were systematically excluded if they met any of the following preprocessing thresholds: (1) a total sequence length of fewer than 150 characters/tokens, representing highly truncated or incomplete clinical notes; (2) reports containing only templated administrative headers without substantive gross or microscopic pathology descriptions; or (3) documents exhibiting corrupted formatting, API exceptions, or non-UTF-8 encoding errors. Applying these systematic thresholds resulted in the exclusion of 77 records, establishing a finalized training dataset of 2256 high-quality clinical records formatted into ChatML instruction triplets (Figure 1). All MIMIC-IV data usage adhered to PhysioNet ethical protocols. Cohort selection flowchart for data preparation. The asterisk (*) in ICD-10 codes (C34.* and D02.2*) denotes wildcard inclusion of all sub-categories under primary malignant neoplasm of bronchus/lung and carcinoma in situ, respectively. “N” indicates the number of electronic medical records or unique patient encounters at each filtering phase. For the independent validation phase, electronic records were obtained from Taipei Medical University Hospital (TMUH). While the full cohort of N = 134 unselected, de-identified pathology reports was processed locally to evaluate end-to-end deployment feasibility, throughput, and JSON syntactic stability under realistic hospital hardware conditions, a purposive stratified sampling strategy was deployed to select a nested clinical forensic audit subset of n = 33 records for double-blind human benchmarking. In clinical informatics and generative NLP research, targeted manual auditing of enriched pilot cohorts (30–cases) is an established, methodologically sound paradigm for proof-of-concept feasibility studies where exhaustive multi-expert annotation is logistically prohibitive.20,21,24 Specifically, prior pathology abstraction benchmarks at our institution demonstrated that a curated cohort of 33 TMUH pathology reports provides sufficient narrative density to evaluate generative extraction precision,20,21,24 while focused audit subsets allow for granular, case-by-case error analysis of complex reasoning chains.20,21,24 Rather than selecting reports entirely at random which risks omitting rare oncological entities, the n = 33 subset was strategically enriched to encompass: (1) major lung cancer lineages (adenocarcinoma, n = 12; squamous cell carcinoma, n = 3; small cell carcinoma, n = 4); (2) pre-invasive lesions (AIS and MIA); and (3) exactly five complex clinical trap cases (thymoma, metastatic sarcoma, metastatic HCC, acute inflammation, and benign lung) to stress-test safety boundaries. The Dual-Layered Ground Truth Strategy This study implemented a dual-layered validation strategy to reconcile the computational requirement for large-scale training data with the clinical necessity for rigorous oversight. This approach involves a strategic division of the methodology into a synthetic distillation phase and a human-led clinical benchmarking phase. The initial layer entailed the generation of synthetic ground truth labels for the training corpus (n=2256). Due to the considerable logistical constraints associated with manual expert annotation at scale, Gemini 2.0 Flash was utilized as the high-parameter teacher model via an automated API pipeline with exponential backoff retry logic. To prevent unconstrained generations and circular logic, the teacher model was bound by a deterministic prompt schema derived from the 2021 WHO Classification of Thoracic Tumours.22 This ensured that pseudo-labels were taxonomically anchored in international oncological standards rather than being products of stochastic hallucination. The second layer established a human gold standard to provide an absolute performance benchmark and safety auditing on the 33-case TMUH stratified subset. These records were meticulously annotated by two independent board-certified clinicians through an independent double-blind process. To assess the intrinsic reliability and robustness of our human-derived reference benchmark prior to consensus resolution, we evaluated the inter-observer agreement between the two clinical experts.25,26 Any minor histological discrepancies during this double-blind phase were subsequently resolved via a joint case re-examination and consensus adjudication to establish the finalized reference gold standard. To quantitatively bridge the gap between human expertise and AI performance, a second Cohen’s Kappa (κ) evaluation was employed to measure the agreement between the localized student model’s output and this finalized human gold standard.23 This dual-layered architecture ensures that while the SLM benefits from large-scale instruction tuning, its final deployment parameters remain firmly validated by high-fidelity human expertise. Semantic Framework: The WHO Tiered Logic Linguistic variability in anatomical pathology reports poses a significant challenge for automated clinical abstraction. To ensure standardized extraction and maintain clinical relevance, we developed a Tiered Logic framework grounded in the WHO Classification of Thoracic Tumours (5th ed., 2021) and the International Collaboration on Cancer Reporting (ICCR) protocols.7,22 This framework codifies clinical guidelines into a six-level computational hierarchy designed to model systematic diagnostic pathways (Figure 2). The tiered logic decision tree for histological tumor classification. Bold text labels (“LEVEL 1” and “LEVEL 2”) indicate primary decision gates in the WHO-guided computational hierarchy. Diamond nodes represent decision evaluation points, while rectangular boxes represent categorization outputs. Green-shaded boxes highlight the three main primary histological lineage branches (Adenocarcinoma, Squamous Cell Carcinoma, and Neuroendocrine Carcinoma/Tumour). The framework operates as a hierarchical series of decision gates designed to maintain extraction integrity, beginning with Level 1 (The Gatekeeper), which functions as a primary binary gate to confirm the presence of primary lung malignancies while strictly excluding non-malignant findings or metastatic lesions from non-thoracic origins. This is sequentially followed by Level 2 (Primary Lineage), facilitating broad categorization into major histological lineages such as Non-Small Cell Lung Cancer (NSCLC) or Small Cell Lung Cancer (SCLC), before proceeding to Level 3 and 4 (Sub-classification), where the complexity of adenocarcinoma is addressed by distinguishing between pre-invasive (AIS, MIA) and invasive phases and further stratifying squamous cell carcinoma variants. Ultimately, Level 5 and 6 (Rare Variants) differentiate high-grade neuroendocrine carcinomas from neuroendocrine tumors (NETs) and other rare pulmonary malignancies, ensuring a comprehensive and standardized diagnostic pathway. By instantiating this hierarchical logic within the foundational system prompts, the framework serves as a semantic constraint. This ensures that both the teacher and student models generate structured JavaScript Object Notation (JSON) outputs that are interoperable with hospital cancer registries while mitigating generative hallucinations by forcing the model to validate each diagnostic tier before proceeding. To guarantee the operational and clinical safety of this semantic constraint, the underlying system prompts enforce a strict “Header Blindness” generalizability rule alongside the Level 1 Gatekeeper criteria. The complete operational rules, clinical safety decision constraints, and the mandatory six-key structured JSON output schema are comprehensively detailed in Table 1. Table 1System Prompt Instructions and Structured JSON Output Schema for WHO Tiered Logic AbstractionComponentRule/Operational LogicDescription & Schema SpecificationRole DefinitionClinical PersonaSpecialized AI Pathologist and Clinical Data Abstractor operating under 2021 WHO Classification of Thoracic Tumours and ICCR guidelines.Generalizability RuleHeader BlindnessInstructs the model to ignore synthetic tags, metadata, or formatting headers (eg, - CONTEXT: MIA -) and extract information solely from authentic medical narratives.Level 1 GatekeeperBinary Safety FilterStrictly outputs “level_1_malignancy_found”: false (“Negative”) and terminates downstream execution if: No malignancy is described (benign/inflammatory tissue)Non-thoracic metastasis is present (eg, breast, colon, hepatocellular)Specimen describes unconfirmed suspicions or non-lung thoracic tumors (thymoma, lymphoma)Progression LogicTiered HierarchyProceeds to Levels 2–6 if and only if primary thoracic malignancy is confirmed: Level 2: Primary lineage (NSCLC, SCLC, non-primary)Levels 3–4: Adenocarcinoma invasiveness/subtypes and squamous variantsLevels 5–6: High-grade vs low/intermediate-grade neuroendocrine tumorsStructured Output SchemaDeterministic JSON FormatMandatory JSON Fields: level_1_malignancy_found: booleanlevel_2_primary_types: listlevel_3_adenocarcinoma_details: object (is_present, specific_variant, predominant_subtype)level_4_squamous_details: object (is_present, subtype)level_5_6_neuroendocrine_details: object (is_present, specific_type)reasoning_trace: string (step-by-step clinical justification) System Prompt Instructions and Structured JSON Output Schema for WHO Tiered Logic Abstraction No malignancy is described (benign/inflammatory tissue) No malignancy is described (benign/inflammatory tissue) Non-thoracic metastasis is present (eg, breast, colon, hepatocellular) Non-thoracic metastasis is present (eg, breast, colon, hepatocellular) Specimen describes unconfirmed suspicions or non-lung thoracic tumors (thymoma, lymphoma) Specimen describes unconfirmed suspicions or non-lung thoracic tumors (thymoma, lymphoma) Level 2: Primary lineage (NSCLC, SCLC, non-primary) Level 2: Primary lineage (NSCLC, SCLC, non-primary) Levels 3–4: Adenocarcinoma invasiveness/subtypes and squamous variants Levels 3–4: Adenocarcinoma invasiveness/subtypes and squamous variants Levels 5–6: High-grade vs low/intermediate-grade neuroendocrine tumors Levels 5–6: High-grade vs low/intermediate-grade neuroendocrine tumors level_1_malignancy_found: boolean level_1_malignancy_found: boolean level_2_primary_types: list level_2_primary_types: list level_3_adenocarcinoma_details: object (is_present, specific_variant, predominant_subtype) level_3_adenocarcinoma_details: object (is_present, specific_variant, predominant_subtype) level_4_squamous_details: object (is_present, subtype) level_4_squamous_details: object (is_present, subtype) level_5_6_neuroendocrine_details: object (is_present, specific_type) level_5_6_neuroendocrine_details: object (is_present, specific_type) reasoning_trace: string (step-by-step clinical justification) reasoning_trace: string (step-by-step clinical justification) Instruction-Based Distillation and Model Fine-Tuning The development of localized medical language models has historically been hindered by two major factors: the high computational costs associated with training these models and the limited availability of specialized expertise in this field. To overcome these obstacles and attain the objective of efficient and secure clinical reasoning, the present study adopted a Black-Box Synthetic Supervision Distillation approach that involves the transfer of clinical reasoning from a high-parameter model (Gemini 2.0 Flash) to student models with reduced parameters for secure, offline execution (Figure 3). Teacher-student instruction-based distillation architecture pipeline. Bold text elements denote primary workflow modules. Database cylinder icons represent electronic health record text corpora (MIMIC-IV and TMUH). The TMUH validation database block indicates the local deployment stability cohort (N = 134 records) used for throughput and on-device formatting tests, with the nested Clinical Forensic Audit Cohort (n = 33 records) explicitly designated as the subset passing to expert-labeled statistical and diagnostic performance evaluation. The graphics processing unit (GPU) icon specifies the local hardware environment (NVIDIA RTX 3060 12GB) utilized for offline parameter-efficient fine-tuning (LoRA). The distillation and tuning process was executed in four systematic phases designed to optimize clinical reasoning on resource-constrained, local deployment environments. Phase 1 is teacher inference and data distillation process. The teacher model (Gemini 2.0 Flash) deconstructed 2256 unstructured MIMIC-IV narratives into structured JSON pseudo-labels under the strict guidance of the six-level WHO Tiered Logic framework. This ensured that the distilled targets were taxonomically aligned with international thoracic oncological standards rather than being stochastic generative outputs. Phase 2 comprised formatting and structural transformation, where the raw outputs were converted into an instruction-tuning format and reconstructed as training triplets: comprising the structured WHO Tiered Logic instructions, the raw clinical pathology narrative, and the teacher-generated reasoning trace. This triplet architecture was designed to establish an algorithmic clinical rationale rather than forcing the model to execute raw, rote memorization of diagnostic labels. Phase 3 focused on parameter-efficient fine-tuning (PEFT) and LoRA configuration, to achieve on-device feasibility and democratic accessibility of clinical AI, the fine-tuning phase was conducted on a single consumer-grade workstation equipped with an NVIDIA RTX 3060 GPU with a 12GB VRAM limit. This constraint was strictly met by employing 4-bit NormalFloat (NF4) quantization and Low-Rank Adaptation (LoRA) with a rank (r) of 16 and alpha of 16. To prevent catastrophic forgetting of general medical language patterns during domain alignment, LoRA adapters were integrated across all linear layers (including projection, attention, and gate modules). The complete configurations of our training pipeline are consolidated in Table 2. Table 2Parameter-Efficient Fine-Tuning (PEFT) and Training ConfigurationsFine-Tuning ParameterOperational Configuration ValueTechnical Purpose & Local OptimizationQuantization Mode4-bit NormalFloat (NF4)Compresses model weights to operate within consumer-grade 12GB VRAM limitsLoRA Rank (r)16Sets the dimensionality of the low-rank adaptation matrix updatesLoRA Alpha16Scaling factor for adjusting the adapter’s learning weight influenceTarget Linear Modulesq_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_projApplied to all linear layers to safeguard general linguistic capabilitiesTraining Duration1 EpochAvoids over-internalization of the teacher’s stylistic biases and noiseInitial Learning Rate2 x 10−4Optimized convergence speed for 4-bit clinical adaptersOptimization AlgorithmAdamW (8-bit)Reduces memory overhead during the backpropagation stepsBatch Size ConfigurationMicro-batch: 1; Gradient Accumulation: 8Ensures stable gradient descent updates equivalent to a global batch size of 8Max Sequence Length1536 tokensAccommodates full, dense medical reports without text truncationAcceleration FrameworkUnsloth FrameworkMaximizes throughput and prevents VRAM out-of-memory bottlenecks Parameter-Efficient Fine-Tuning (PEFT) and Training Configurations Finally, during Phase 4, training was executed utilizing the Unsloth framework for a single epoch to prevent overfitting. A micro-batch size of one and a gradient accumulation value of eight were maintained to guarantee stable convergence. Crucially, a common failure mode of quantized 8B-parameter open-source SLMs during offline inference is generative looping where the model continuously repeats whitespace, punctuation, or redundant medical terms and structural JSON hallucinations due to the failure of standard End-of-Sequence (EOS) tokens. To resolve this, we implemented a deterministic StopOnProperJSON mechanism at the inference engine level. In this approach, the output structure is governed by a strict JSON schema where the model is required to close its clinical extraction with the clinical reasoning trace inside the JSON object. The customized parser monitors the output stream in real-time, enforcing a hard termination immediately upon generating the final closing curly brace (“}”) that satisfies the valid JSON syntax. This dual-layered guardrail (the 1-epoch boundary combined with deterministic StopOnProperJSON stopping rules) effectively stabilized local execution, ensured 100% data format integrity, and eliminated computational leakage during clinical deployment testing. Performance Evaluation and Statistical Analysis The performance of the distilled Small Language Models (SLMs) was then assessed against the human gold standard (n=33) using a multi-dimensional evaluation framework designed for clinical safety. This framework began with the evaluation of quantitative diagnostic acuity through a confusion matrix to calculate accuracy, macro-averaged precision, recall (sensitivity), and the F1-score. Within the context of oncological registries, recall was prioritised as the primary safety indicator to minimise false negatives, with 95% confidence intervals (CIs) derived via the Clopper-Pearson (exact) method to provide a transparent measure of statistical uncertainty for specific histological subgroups. Given the pilot nature of this study, we explicitly acknowledge that the targeted audited sample (n=33) limits the overall statistical power and the numerical stability of histopathological subgroup estimates. Consequently, these statistical analyses are interpreted as preliminary, hypothesis-generating benchmarks rather than definitive clinical generalizations. In order to complement the classification accuracy, inter-rater reliability was measured using Cohen’s Kappa (κ), a robust statistic that accounts for chance agreement by calculating the ratio between the relative observed agreement (po) and the hypothetical probability of chance agreement (pe). This served as the definitive validator for the clinical reliability of the instruction-based distillation process.25 Beyond these metrics, qualitative safety and reasoning adherence were scrutinised through a clinical audit of five confounding trap cases, including thymoma and metastatic sarcoma. In this audit, the models’ reasoning traces were scored from 0 (Failure/Hazard) to 2 (Optimal/Safe) based on context synthesis, logical consistency, metastatic filtering, and pathological granularity in alignment with the WHO Tiered Logic. Finally, the operational sustainability of the proposed model was assessed by measuring the mean inference time per document. This was done to evaluate the practical feasibility of integrating 8B-parameter SLMs into hospital workflows using consumer-grade hardware, specifically an NVIDIA RTX 3060. The results demonstrated a sustainable automation model that eliminates the requirement for extensive capital investment in enterprise computing resources. To isolate and verify the specific contributions of our proposed architecture’s components, we conducted two formal ablation studies. Firstly, we compared our specialized Med42-8B model (which embeds the WHO Tiered Logic in its instruction space) against the generalist Llama-3-8B model (which serves as a semantic ablation baseline lacking these structured taxonomic constraints) specifically across the five clinical trap confounding cases. This ablation isolates whether domain-specific hierarchical gating is mathematically necessary to prevent diagnostic safety hazards. Then secondly, we evaluated the offline processing performance of the quantized local student models with and without the deterministic StopOnProperJSON mechanism. This isolates whether a customized syntactical parser is operationally required to stabilize 8B-parameter on-device execution on consumer hardware, comparing format compliance and token looping rates between the two states. To ensure statistical and accounting transparency, operational metrics (JSON formatting compliance and mean inference throughput) were evaluated across the entire Local Deployment Stability Cohort (N = 134). In contrast, all diagnostic performance metrics (Accuracy, Precision, Recall, F1-score, and Cohen’s Kappa) and qualitative clinical safety reasoning scores were calculated exclusively within the expert-audited Clinical Forensic Audit Subset (n = 33) against the double-blind human consensus gold standard. Results Overall Model Performance and Operational Efficiency Before evaluating the diagnostic metrics, it is critical to clarify the dual-layered structure of our validation framework. The entire independent validation cohort from Taipei Medical University Hospital (N = 134 pathology reports) was executed locally to evaluate operational stability, network-free throughput, and JSON format compliance under realistic hospital deployment conditions. From this broader deployment set, a stratified pilot sample of n = 33 records was extracted for an intensive, double-blind clinical forensic audit by human experts. The high-fidelity expert consensus annotations from this 33-case subset serve as the gold standard for the absolute performance and clinical safety metrics presented hereafter. Across the broader local deployment stability cohort (N = 134), both student models demonstrated perfect (100%) JSON syntactic compliance and zero generation-related execution failures, validating the local system’s operational viability. On-device inference throughput evaluated over this 134-case cohort yielded a mean processing latency of 13.97 seconds per document on a single consumer GPU. Following this system-level validation, all absolute diagnostic performance and clinical reasoning safety metrics were derived strictly from the nested clinical forensic audit subset (n = 33) against the double-blind human consensus gold standard. The performance metrics and operational efficiency of the teacher and student models are summarized in Table 3. Within this expert-audited subset, both Gemini 2.0 Flash and the distilled Med42-8B model achieved an identical accuracy of 90.91% (95% CI: 75.67% to 98.08%), significantly outperforming the generalist Llama-3-8B model which obtained 78.79% (95% CI: 61.09% to 91.02%) (Table 3). While the overlapping confidence intervals reflect the pilot scale of this investigation, the specialized student model’s high point estimate establishes its comparable classification feasibility. Table 3Overall Performance and Efficiency MetricsModel ArchitectureAccuracy (%)[95% CI]Macro F1-ScoreCohen’s KappaJSON Rescued FormatsAvg. Inference Time (s/doc)Gemini flash 2.0 (teacher)90.91% [75.67%, 98.08%]0.880.8602.05 sLlama-3-8B (student)78.79% [61.09–91.02%]0.790.69112.11 sMed42-8B (student)90.91% [75.67–98.08%]0.920.86113.97 s Overall Performance and Efficiency Metrics To establish a mathematically rigorous and unambiguous baseline of agreement, we evaluated two distinct dimensions of Cohen’s Kappa. Starting with human-to-human reliability (gold standard consistency). Prior to consensus reconciliation, the initial double-blind annotation of the 33-case clinical audit subset by our two board-certified clinicians yielded an outstanding inter-observer agreement of (κ = 0.88). This confirms that the manual human labels used to define our reference benchmark are highly stable, reliable, and almost perfectly aligned.26 Next step is model-to-human reliability (AI classification agreement). After resolving minor discrepancies to establish the final adjudicated consensus, we measured the agreement between each student model’s output and the final human gold standard. The specialized Med42-8B student model achieved an exceptional agreement of (κ = 0.86), which matched the teacher model (Gemini 2.0 Flash) and falls firmly in the substantial-to-almost-perfect agreement range. In contrast, the generalist Llama-3-8B student model demonstrated a substantially weaker agreement of (κ = 0.62), underscoring the statistical superiority of domain-specific instruction distillation.26 The multidimensional performance profile illustrated in Figure 4 further compares the metric distribution between the teacher and student models. The teacher model and Med42-8B exhibited high inter-rater reliability with expert labels, as evidenced by a Cohen’s kappa value of 0.86. Concerning operational reliability, both student models effectively employed JSON rescue mechanisms (n = 1 instance each) to maintain data structure integrity during local inference. The teacher model achieved the fastest inference time (2.05 s/doc), while the Med42-8B model showed comparable classification accuracy despite the increased latency associated with local deployment (13.97 s/doc). Multidimensional performance profile comparing teacher (Gemini-2.0-Flash) and student models across primary evaluation metrics. The solid green line represents the specialized Med42-8B model performance, while the dashed red line represents the general-purpose Llama-3-8B model. Axis values range from 0.0 to 1.0, representing normalized metric scores. Histological Classification Acuity To rigorously test the models’ discriminative boundaries, the other/negative category was established as a heterogeneous control group. This category includes non-pulmonary malignancies (eg, Thymoma, Hepatocellular carcinoma), non-target lung pathologies (eg, pleomorphic sarcoma), and specimens confirmed negative for malignancy. By incorporating these diverse cases, the evaluation reflects the complexity of real-world clinical documentation where the system must accurately differentiate lung-specific carcinomas from alternative or benign findings. As outlined in Table 4, the distilled specialized Med42-8B model matched the teacher model in Adenocarcinoma recall, achieving a score of 1.00 (95% CI: 0.74 to 1.00) compared to Llama-3-8B’s recall of 0.67 (95% CI: 0.35 to 0.90). For Squamous Cell Carcinoma (n = 3), all models achieved a recall of 1.00; however, as shown by our statistical reporting, this point estimate is subject to an extremely wide confidence interval of [95% CI: 0.29 to 1.00] due to the very small subgroup sample. Similarly, for Small Cell Carcinoma (n = 4), Med42-8B and Llama-3-8B demonstrated a recall of 0.75 (95% CI: 0.19 to 0.99). These broad interval estimates are reported transparently to prevent overclaims. In addition, as illustrated in the comparative diagnostic recall in Figure 5, the observed performance differences in the SCC (Squamous Cell Carcinoma) category represent a variance of only one to two records due to the limited sample size. They confirm that while the specialized SLM shows strong observational concordance, these rare subgroup findings are preliminary and serve as hypothesis-generating pilot metrics rather than definitive population-level generalizations. Table 4Comparative Acuity for Histological Lung PathologyPathology EntityMetricGemini flash 2.0Llama-3-8BMed42-8BAdenocarcinomaPrecision0.920.800.86(n = 12)Recall (95% CI)1.00 [0.74–1.00]0.67 [0.35–0.90]1.00 [0.74–1.00]F1-Score0.960.730.92Squamous Cell CarcinomaPrecision1.001.001.00(n = 3)Recall (95% CI)1.00 [0.29–1.00]1.00 [0.29–1.00]1.00 [0.29–1.00]F1-Score1.001.001.00Small Cell CarcinomaPrecision1.000.431.00(n = 4)Recall (95% CI)0.50 [0.07–0.93]0.75 [0.19–0.99]0.75 [0.19–0.99]F1-Score0.670.550.86Other/NegativePrecision0.870.920.92(n = 14)Recall (95% CI)0.93 [0.66–1.00]0.86 [0.57–0.98]0.86 [0.57–0.98]F1-Score0.900.890.89 Comparative Acuity for Histological Lung Pathology Histological classification acuity: comparative sensitivity (Recall) of specialized Med42-8B versus generalist Llama-3-8B across pathology subtypes. Dark gray bars represent Med42-8B, and light gray bars represent Llama-3-8B. Error bars represent 95% confidence intervals (CIs) calculated via the Clopper-Pearson exact method. The sample size for each histological category is indicated by “n”. Clinical Reasoning and Safety Performance The qualitative assessment of clinical reasoning and diagnostic safety, particularly in differentiating primary lung lesions from confounding non-primary conditions, is presented in Figure 6. The heatmap illustrates reasoning adherence across four safety dimensions. The Med42-8B model exhibited consistent adherence to the WHO-based logic injected during the distillation process, demonstrating scoring stability comparable to that of the teacher model. Heatmap of clinical reasoning adherence and diagnostic safety across confounding non-primary lung lesions. The qualitative evidence (detailed in Table 5) was scored across four critical safety dimensions (horizontal axis) using a color-coded numerical scale: 0 = Failure/Hazard (red), 1 = Basic/Acceptable (yellow), and 2 = Optimal/Safe (green). Bold labels identify the evaluated trap cases (vertical axis) and model architectures. The specialized Med42-8B model demonstrates consistent alignment with clinical labels, maintaining optimal safety scores, particularly in context synthesis and logical consistency. Conversely, the general-purpose Llama-3-8B exhibits systemic vulnerabilities, including critical logical hallucinations (eg, in Case 3: Thymoma). An analysis of the reasoning logs revealed that while the teacher and Med42-8B models effectively filtered non-primary lesions, such as thymoma, the Llama-3-8B model exhibited instances of logical inconsistency. These instances involved the misclassification of non-lung primary tissues as lung-specific histological subtypes, indicating a variation in the robustness of the reasoning framework across different small language model architectures. The evidence presented in Table 5 underscores the systemic vulnerabilities of general-purpose SLMs. In the first case, which pertains to thymoma, the Llama-3-8B model showed a critical structural hallucination. That is, it extracted lymphoma as the primary label while, in its reasoning trace, describing a thymoma that contradicts this. In contrast, the specialized Med42-8B model exhibited strict adherence to the Level 1 Gatekeeper logic, safely outputting negative for lung-primary malignancies while accurately noting the thymic neoplasm in its internal reasoning. This behavior underscores the necessity of domain-specific instruction distillation to prevent potentially hazardous misclassifications in downstream clinical registries. Table 5Safety-First Clinical Reasoning: Qualitative Analysis of Confounding LesionsGround Truth (Confounding Case)ModelExtracted Primary LabelExtracted Reasoning Trace (Snippets)Clinical Safety EvaluationScoreCase 1: Thymoma(Non-lung primary thoracic tumor)Llama-3-8BLymphoma“The tumor is a lymphoma. This is a thymoma, with the subtype indeterminate”.Critical Hazard: Severe logical contradiction. Hallucinated a different hematologic malignancy in the JSON label despite reasoning otherwise.0Med42-8BNegative“No tumor or malignancy is described. A thymoma is considered, and the subtype is indeterminate”.Optimal Safety: Strictly adhered to Level 1 Gatekeeper logic. Correctly filtered out the non-lung primary from the extraction target.2Case 2:Metastatic Sarcoma(Patient with history of sarcoma)Llama-3-8BSarcoma“Sarcoma is the primary malignancy, with no evidence of adenocarcinoma”.Basic/Acceptable: Identified sarcoma but failed to synthesize the clinical history, mislabeling it as a new primary.1Med42-8BPleomorphic sarcoma“Combined the patient’s past history, metastatic high grade pleomorphic sarcoma is considered”.Optimal Context: Demonstrated high contextual awareness by synthesizing the patient’s past history with the current pathology.2Case 3:No Malignancy(Acute inflammatory infiltrate)Llama-3-8BNegative“No tumor or malignancy is described. The text only mentions inflammation”.Basic/Acceptable: Correctly filtered, but provided a generalized explanation.1Med42-8BNegative“No malignancy or granulomatous inflammation is found in this specimen”.High Specificity: Maintained pathological granularity by specifically ruling out granulomatous inflammation as described in the text.2Case 4:Hepatocellular Carcinoma(Metastatic to lung)Llama-3-8BHepatocellular carcinoma“Primary liver malignancy, not a primary lung malignancy, consistent with metastatic hepatocellular carcinoma”.Acceptable: Correctly identified the metastasis using immunohistochemistry (IHC) markers.2Med42-8BHepatocellular carcinoma“The tumor cells are immunoreactive to Hep-par-1 but not to CK7 and TTF-1, consistent with metastatic hepatocellular carcinoma”.Acceptable: Effectively utilized IHC marker logic (Hep-par-1) to confirm the non-lung origin.2Case 5:Benign Lung Tissue(Normal/benign specimen)Llama-3-8BNegative“The text only mentions ‘benign bronchial and lung tissues without malignancy’”.Optimal: Correctly identified and extracted the benign nature of the tissue.2Med42-8BNegative“No tumor or malignancy is described. The text only mentions ‘benign bronchial and lung tissues’”.Optimal: Safely filtered the benign tissue with accurate contextual reasoning.2Notes: Scores represent diagnostic safety dimensions: 0 = failure/hazard; 1 = basic/acceptable; 2 = optimal/safe. Safety-First Clinical Reasoning: Qualitative Analysis of Confounding Lesions Notes: Scores represent diagnostic safety dimensions: 0 = failure/hazard; 1 = basic/acceptable; 2 = optimal/safe. System Component Ablation Analyses To empirically justify our architectural choices, the results of our dual-ablation studies are detailed through semantic gating and inference guardrail evaluations. For the semantic gating ablation (WHO tiered logic contribution), the qualitative audit of the five clinical trap confounding cases served as a direct semantic ablation test (summarized in Figure 6 and Table 5). When the semantic constraint (the 2021 WHO classification taxonomy) was absent (the generalist Llama-3-8B baseline), the model suffered from catastrophic logical hallucinations. In case 3 (Thymoma), the unconstrained model misclassified the non-primary mediastinal tumor as primary lung lymphoma, representing a critical clinical safety hazard (Score = 0). Conversely, with the WHO Level 1 gatekeeper constraints enabled, the specialized Med42-8B model successfully identified the lesion as non-primary lung pathology and halted downstream processing, achieving an optimal safety rating (Score = 2). This demonstrates that international taxonomic constraints are structurally necessary to secure diagnostic boundaries in lightweight models.19 Regarding the inference guardrail ablation (StopOnProperJSON contribution), when local inference was executed utilizing standard end-of-sequence (EOS) tokens without the deterministic StopOnProperJSON mechanism, both the Med42-8B and Llama-3-8B models exhibited a severe generative looping rate of 15.15% (5/33 cases) during the validation audit. The models repeatedly generated trailing whitespaces, redundant punctuation, or looped clinical keywords, resulting in memory overflows and JSON syntax truncation. The implementation of StopOnProperJSON completely resolved this operational failure, enforcing immediate inference termination at the closing curly brace (“}”) and restoring a perfect 100.0% (134/134 cases) JSON syntactic compliance rate across the broader deployment cohort. This confirms that hard syntactical boundaries are essential for deploying quantized models on local consumer GPUs.20 Discussion This exploratory pilot study provides proof-of-concept evidence regarding the feasibility of compressing clinical reasoning into specialized, localized small language models (SLMs) for four-class lung histology abstraction. The observed concordance between Med42-8B and the frontier teacher model (Gemini 2.0 Flash) suggests that structuring the distillation space via the WHO Tiered Logic enables compact architectures to retain structured diagnostic pathways under localized hardware constraints. Parity and Specialization of Small Language Models Our observation that Med42-8B achieved an accuracy of 90.91% and a superior macro F1-score (0.92) compared to its teacher model aligns with the growing body of evidence regarding the efficiency of domain-specific distillation. This performance surpasses the 89% accuracy reported in research employing ChatGPT (GPT-3.5) for analogous lung cancer pathology extraction tasks.15 The former research employed a cloud-based general-purpose model, whereas our results indicate that a smaller, specialized model can achieve higher precision by focusing on medical-specific semantic spaces. While the higher numerical macro F1-score of Med42-8B (0.92 vs 0.88) represents a marginal observational difference within this small pilot sample (n=33), it illustrates the potential benefit of domain-specific instruction alignment. As demonstrated in prior clinical distillation frameworks, specialized fine-tuning enables compact architectures to suppress generalist token noise, thereby stabilizing classification across low-prevalence histological categories without claiming definitive algorithmic superiority over foundation models.24 Clinical Significance for Diagnostic Safety The qualitative analysis of reasoning logs indicated that the 1-epoch training strategy was essential for balancing knowledge transfer while mitigating hallucination propagation. From a technical perspective, restricting the distillation process to a single epoch acts as a safeguard against blind imitation.24 As the student model is trained using synthetic labels generated by the teacher (Gemini 2.0 Flash), extending the training duration carries the risk of overfitting the student to the specific stylistic artifacts or inherent hallucinations of the teacher.24 The phenomenon known as noise propagation is a recognized challenge in the process of black-box distillation, and employing a single-pass strategy is one method to ensure that students comprehensively grasp the underlying clinical logic without being encumbered by the errors of the instructor.24 In order to mitigate the risk of generative hallucinations and looping observed, a dual-layer safety mechanism was implemented. First, the 1-epoch training strategy prevents the student model from over-internalizing the teacher’s stylistic artifacts. Secondly, a deterministic StopOnProperJSON stopping criterion was introduced during local inference. In contrast to standard End-of-Sequence (EOS) tokens, which may encounter failure in quantized models, this mechanism compels the model to terminate execution immediately following the closure of the final JSON brace (“}”), thereby ensuring data integrity and preventing computational leakage. Additionally, the high data density of the training set (n=2256 medical records) provides a substantial distribution of clinical linguistic patterns. In the context of instruction tuning, this volume of tokens allows the model to encounter thousands of medical semantic variations in a single pass. This is statistically sufficient for domain-specific alignment without leading to catastrophic forgetting. The phenomenon of over-training on specialized datasets has been shown to potentially compromise a model’s general linguistic capabilities. By employing a single epoch, the fundamental reasoning of the 8B-parameter architecture was preserved, accompanied by the integration of specialized WHO-based logic. While these qualitative observations from five curated trap cases illustrate the functional necessity of gatekeeper constraints, they represent exploratory pilot stress-tests rather than an exhaustive validation across the entire spectrum of rare thoracic pathologies. The Validation Framework: Breaking Circular Logic To address the potential for circular reasoning often associated with AI-distilled models, this study implemented a tiered validation framework. The teacher model’s synthetic supervision was categorized as a silver standard annotator. While Gemini 2.0 Flash offers high-capacity reasoning, its primary role was to provide scalable labeling of the 2256 MIMIC records. This task would be logistically prohibitive for human experts and prone to fatigue-induced inconsistencies. The inclusion of the expert-audited TMUH subset (n=33), curated from the broader local deployment cohort (N=134), disrupts the perceived circularity of AI-distilled evaluation. The finding that the student model, despite being trained on AI-labeled synthetic data, achieved high concordance with these human-expert labels suggests that the model learned valid clinical principles rather than purely mimicking templates. Crucially, the substantial cross-institutional domain shift in this study, spanning the geographical, cultural, and structural divergence between narrative US inpatient discharge summaries (MIMIC-IV) and raw concise Taiwanese pathology narratives (TMUH) was bridged by the invariant WHO Tiered Logic serves as a semantic anchor.17 Because the diagnostic criteria of the WHO 2021 thoracic tumor taxonomy remain invariant across institutions globally, the semantic constraint shields the compact student model’s latent representation from stylistic domain shift degradation, enabling robust zero-shot generalizability.11,22 Addressing Security and Operational Gaps A fundamental challenge that this research aims to address is the tension between the potential of high-performance AI and the imperative of safeguarding clinical data privacy. As indicated by former research,10,11 the operational costs and stringent privacy restrictions associated with proprietary third-party APIs frequently impede the incorporation of Large Language Models (LLMs) into actual clinical workflows. The demonstration that an 8B-parameter model can be deployed locally on consumer-grade hardware while maintaining substantial inter-rater agreement (κ = 0.86) provides a viable template for secure, on-device clinical AI.25 Although frontier models like Gemini 2.0 Flash facilitate rapid inference through cloud infrastructure, the operational trade-off inherent to the specialized student model is justified by its adherence to data sovereignty principles. In a multidisciplinary healthcare setting, the capacity to process sensitive pathology reports within the hospital firewall, as advocated in the Shareable AI framework,12 eliminates a primary barrier to the large-scale implementation of automated registry abstraction. Furthermore, while the local inference latency (13.97 s/doc) is inherently higher than cloud-based alternatives, this performance remains within an acceptable operational threshold for non-urgent registry abstraction tasks. This local deployment capability ensures the protection of institutional data while establishing a preliminary, privacy-preserving technical blueprint for future automated oncology informatics. Complementing this secure deployment, the local inference was configured with a maximum sequence length of 1536 tokens to ensure comprehensive processing of dense medical text without information loss (ie, truncation). This threshold was determined through empirical analysis to encompass the full textual breadth of the MIMIC-IV and TMUH pathology reports, ensuring that critical diagnostic descriptors located at the end of reports were not omitted, thereby maintaining high data integrity for the subsequent extraction layers. Limitations and Future Directions Despite the encouraging concordance observed in this pilot investigation, several methodological boundaries must be explicitly contextualized. Foremost, the reliance on a specialized, expert-audited validation cohort of n=33 pathology reports inherently limits the statistical power and the precision of subgroup point estimates. Particularly for low-prevalence histological categories such as squamous cell carcinoma (n=3) and small cell carcinoma (n=4), the performance metrics represent variations of only one or two cases and carry broad 95% confidence intervals, warranting interpretation strictly as hypothesis-generating benchmarks rather than definitive evidence of clinical equivalence. Second, the purposive stratified sampling strategy introduced an inherent selection bias, by deliberately enriching the validation subset with five complex confounding trap cases to stress-test safety boundaries, the sample does not reflect natural epidemiological disease prevalence. Furthermore, this focused subset cannot capture the full spectrum of non-pulmonary metastases, rare mediastinal neoplasms, or varied reporting templates across institutions. Consequently, prospective validation across larger, unselected multicenter cohorts and heterogeneous clinical documentation (eg, operative notes and radiology narratives) is essential before assessing real-world clinical readiness. Conclusion This study establishes an exploratory proof-of-concept for the privacy-preserving, localized abstraction of lung cancer histopathology reports. By combining instruction-based distillation with semantic WHO Tiered Logic constraints, we demonstrated the technical feasibility of achieving high diagnostic concordance on consumer-grade hardware, maintaining substantial agreement (κ=0.86) with human expert annotations within a targeted 33-case validation subset. The primary contribution of this research lies in its safety-first architecture. The incorporation of a deterministic WHO tiered logic gatekeeper and specialized stopping criteria (StopOnProperJSON) effectively mitigates the risks of generative hallucinations and computational looping, which are often cited as primary barriers to the clinical adoption of SLMs. This approach serves to address the long-standing dichotomy between high-performance AI reasoning and the stringent requirements of patient data sovereignty. From a multidisciplinary healthcare perspective, this framework offers a promising, privacy-preserving methodological blueprint for addressing registry lag without compromising institutional security or incurring the prohibitive costs of proprietary third-party APIs. While the statistical generalizability of the findings is currently constrained by the pilot cohort size (n=33) and a focused qualitative safety audit, this framework provides a robust technical foundation for the transition toward autonomous, privacy-preserving clinical informatics. Subsequent research endeavors will investigate the generalizability of this tiered reasoning logic across larger, multi-institutional datasets and other heterogeneous cancer documentation, such as operative notes and radiology reports, to further catalyze the integration of on-device AI into routine clinical workflows. Funding Statement This study did not receive any external financial support. Data Sharing Statement The source code and tiered logic implementation are available at: https://tinyurl.com/rsmh99z5. The MIMIC-IV reference cohort used for model distillation can be accessed via PhysioNet. However, due to institutional data sovereignty mandates and the protection of sensitive patient health information (PHI) under the Personal Data Protection Act, the internal validation dataset from Taipei Medical University Hospital (TMUH) is not publicly available. Requests for access to de-identified summary data may be directed to the corresponding author. Ethics Statement This study was conducted in accordance with the Declaration of Helsinki. The research protocol involving the use of de-identified clinical data from Taipei Medical University Hospital (TMUH) was reviewed and approved by the Institutional Review Board (IRB) of Taipei Medical University on October 24, 2024 (Reference Number: TMU-JIRB N202410009). The requirement for informed consent was waived by the ethics committee due to the retrospective nature of the study and the use of strictly de-identified data. Furthermore, the MIMIC-IV dataset utilized in this investigation is a publicly available, de-identified database; access was authorized following the successful completion of the CITI Program Data or Specimens Only Research training, ensuring compliance with all data use agreements. Disclosure The authors report no conflicts of interest in this work. All authors have participated in the research and have reviewed and approved the final manuscript for submission. References Associated Data Data Availability Statement The source code and tiered logic implementation are available at: https://tinyurl.com/rsmh99z5. The MIMIC-IV reference cohort used for model distillation can be accessed via PhysioNet. However, due to institutional data sovereignty mandates and the protection of sensitive patient health information (PHI) under the Personal Data Protection Act, the internal validation dataset from Taipei Medical University Hospital (TMUH) is not publicly available. Requests for access to de-identified summary data may be directed to the corresponding author.