Guideline-augmented large language models for contraindication screening in interventional spine care Authors: Jack Lott, Nicholas Dietrich Published: 2026 ABSTRACT BackgroundLarge language models (LLMs) are increasingly used for clinical decision support in spine care, but their ability to classify contraindications to interventional procedures according to safety guidelines is unknown.ObjectiveThis study evaluated LLM accuracy using the International Pain and Spine Intervention Society (IPSIS) Safety Practices and assessed whether retrieval-augmented generation (RAG) improves performance.MethodsA database of 318 clinical scenarios was curated from IPSIS Safety Practice guidelines spanning 12 procedure categories (102 absolute contraindications, 57 relative contraindications, 159 matched controls). Three LLMs (GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6) classified each case as no contraindication, relative, or absolute under baseline and RAG conditions. McNemar tests with Bonferroni correction compared conditions within models. Between-model comparisons used Cochran's Q test with post-hoc pairwise analysis.ResultsBaseline accuracy ranged from 85.8% to 87.1% across models and improved to 92.5% to 98.4% with RAG (all p < 0.05). Weighted kappa improved from 0.885–0.913 to 0.949–0.990. Safety catch rates for absolute contraindications were 99.0% to 100.0% across all conditions. Relative contraindications showed the largest improvements with RAG, increasing from 42.1%-56.1% to 59.6%-93.0%.ConclusionLLMs demonstrated high reliability in identifying absolute contraindications but were less consistent for relative contraindications. RAG significantly improved classification accuracy across all models, supporting further evaluation of guideline-augmented LLMs for pre-procedural safety screening. Introduction Low back pain remains one of the leading causes of disability worldwide, and interventional spine procedures are among the most common treatments offered in musculoskeletal practice [1,2]. These procedures, which include epidural steroid injections, facet joint interventions, radiofrequency neurotomy, sacroiliac joint injections, and neuromodulation, require careful pre-procedural screening to identify contraindications that may place patients at risk for serious adverse events [3,4]. The International Pain and Spine Intervention Society (IPSIS) publishes Safety Practices for Interventional Pain Procedures, which provide evidence-informed consensus recommendations on absolute and relative contraindications, procedural sedation, antithrombotic management, imaging guidance, and injection technique across multiple procedure categories [5]. Adherence to these safety practices is a cornerstone of quality assurance in interventional pain medicine. However, the growing number of procedure-specific contraindication rules creates complexity. For instance, pregnancy is classified as a relative contraindication for epidural steroid injection but an absolute contraindication for radiofrequency neurotomy and spinal cord stimulation [5]. Similarly, coagulopathy is relative for most injection procedures but absolute for neuromodulation implantation [5]. These nuances demand familiarity with each individual procedure document. Large language models (LLMs) such as ChatGPT, Gemini, and Claude have shown promise as clinical decision-support tools across medical specialties [6,7]. In musculoskeletal medicine and spine care, LLMs have been evaluated for answering patient questions about low back pain, generating clinical guidelines for spine surgery, triaging surgical referrals, and providing patient education about epidural steroid injections [[8], [9], [10]]. However, to our knowledge, no study has evaluated whether LLMs can accurately classify contraindications to interventional spine procedures according to evidence-informed safety practice guidelines. Retrieval-augmented generation (RAG) is a technique in which relevant reference documents are retrieved and provided to the LLM at inference time, supplementing the model's parametric knowledge with authoritative text [11]. RAG has been shown to improve factual accuracy and reduce hallucinations, defined as plausible-sounding but factually incorrect or unsupported model-generated information, in medical question answering and guideline adherence tasks [7,12]. However, the application of RAG to procedure-specific safety screening in interventional pain research has not been well studied. The purpose of this study was to: (1) evaluate the accuracy of three contemporary LLMs in classifying contraindications to interventional spine procedures according to evidence-informed safety practice guidelines, and (2) determine whether providing guideline context through RAG improves classification performance. These decisions are relevant not only to interventional pain physicians but also to musculoskeletal physiotherapists and other musculoskeletal clinicians involved in triage, referral, patient education, and pre-procedural screening for spine interventions. Methods Study design This was a cross-sectional concordance study comparing LLM classification of derived clinical cases against the IPSIS Safety Practices for Interventional Pain Procedures (2025) as the reference standard [5]. The study involved no human subjects, patient data, or protected health information and was exempt from Institutional Review Board approval. Clinical scenario development Clinical scenarios were derived from the contraindication and safety practice statements of the IPSIS Safety Practices for 12 procedure categories: epidural steroid injection, medial branch block, intra-articular facet joint injection, medial branch radiofrequency neurotomy, lateral atlantoaxial joint injection, sacral lateral branch block, sacroiliac joint injection, sacral lateral branch radiofrequency neurotomy, spinal cord stimulation and dorsal root ganglion stimulation, peripheral nerve stimulation, provocation discography, and vertebral augmentation. Each listed contraindication was presented as a brief case scenario specifying patient age, sex, relevant history, and the proposed procedure. A matched control with no contraindication was constructed for each case, following established approaches for LLM evaluation [7,13]. Two physician reviewers independently assessed all scenarios using prespecified criteria requiring that each scenario: (1) contained sufficient clinical information to determine contraindication status without relying on unstated assumptions; (2) was mapped to the appropriate IPSIS procedure category; (3) reflected a contraindication statement explicitly described in the relevant IPSIS Safety Practice document, when applicable; (4) was assigned the correct reference label of no contraindication, relative contraindication, or absolute contraindication; and (5) had a matched control scenario that removed the contraindicating feature while preserving the same procedure context. For each scenario, reviewers independently assigned a final reference classification of 0, 1, or 2, corresponding to no contraindication, relative contraindication, or absolute contraindication. Initial exact agreement was calculated as the proportion of scenarios for which both reviewers assigned the same final classification. Initial exact agreement was 96.9% (308 of 318 scenarios). Discrepancies were resolved through consensus discussion to establish the final reference standard. The final dataset comprised 318 scenarios: 102 absolute contraindications (score 2), 57 relative contraindications (score 1), and 159 controls (score 0). The sample size of 318 was determined by the total number of unique contraindication items across all 12 IPSIS procedure documents, each paired with a matched control. The distribution of clinical scenarios by procedure is presented in Table 1. Representative example scenarios are also provided in Supplementary Table S1.Table 1Distribution of derived clinical scenarios by procedure category.Table 1ProcedureAbsoluteRelativeControlTotalEpidural Steroid Injection1331632Medial Branch Block731020Facet Joint Injection831122Medial Branch RFN1061632LAAJ Injection941326Sacral Lateral Branch Block731020Sacroiliac Joint Injection731020Sacral Lateral Branch RFN1051530SCS/DRGS Implantation1061632Peripheral Nerve Stimulation581326Provocation Discography1051530Vertebral Augmentation681428Total10257159318RFN = radiofrequency neurotomy; LAAJ = lateral atlantoaxial joint; SCS = spinal cord stimulation; DRGS = dorsal root ganglion stimulation. Table 1Distribution of derived clinical scenarios by procedure category.Table 1ProcedureAbsoluteRelativeControlTotalEpidural Steroid Injection1331632Medial Branch Block731020Facet Joint Injection831122Medial Branch RFN1061632LAAJ Injection941326Sacral Lateral Branch Block731020Sacroiliac Joint Injection731020Sacral Lateral Branch RFN1051530SCS/DRGS Implantation1061632Peripheral Nerve Stimulation581326Provocation Discography1051530Vertebral Augmentation681428Total10257159318RFN = radiofrequency neurotomy; LAAJ = lateral atlantoaxial joint; SCS = spinal cord stimulation; DRGS = dorsal root ganglion stimulation. Distribution of derived clinical scenarios by procedure category. RFN = radiofrequency neurotomy; LAAJ = lateral atlantoaxial joint; SCS = spinal cord stimulation; DRGS = dorsal root ganglion stimulation. LLM selection and experimental conditions Three LLMs were evaluated: GPT-5.4 (OpenAI), Gemini 3.1 Pro (Google), and Claude Sonnet 4.6 (Anthropic). All models were accessed programmatically through their application programming interfaces (APIs). When supported, temperature (a model setting that controls response randomness, with lower values producing more deterministic outputs) was set to 0 to reduce response variability, and outputs were limited to 300 tokens (approximately 200–250 words). Each scenario was submitted as a separate API request, so the model could not use information from preceding scenarios. Each scenario was presented under two conditions. In the baseline condition, the LLM received only the scenario and classified the contraindication level using its pre-trained knowledge. In the RAG condition, a vector-based retrieval system was used to provide relevant guideline context. The IPSIS Safety Practices documents were extracted and segmented into overlapping text chunks (800 tokens per chunk, 200-token overlap). Each chunk was converted into an embedding, a numerical vector representation of its semantic content, using text-embedding-3-large (OpenAI), and stored in a vector database. For each clinical scenario, the scenario text was embedded using the same model, and the five guideline chunks with the greatest semantic similarity were retrieved using cosine similarity, a mathematical measure of similarity between numerical vectors. These retrieved guideline excerpts were then provided as additional context within the prompt (Fig. 1). Under both conditions, models were instructed to respond with a numeric score (0, 1, or 2), corresponding to absent, relative, or absolute contraindications, respectively. Supplementary Table S2 provides the prompts for baseline and RAG model inference.Fig. 1Baseline and retrieval-augmented generation pipeline. Baseline and retrieval-augmented generation pipeline. Outcomes The primary outcome was exact classification accuracy, defined as the proportion of scenarios for which the LLM score matched the IPSIS reference standard. Accuracy was additionally stratified by contraindication category (absolute, relative, and control) to evaluate differential performance across classification types. Secondary outcomes included the safety catch rate (proportion of absolute contraindications assigned a score of 1 or 2 by the LLM), the number of missed absolute contraindications (absolute contraindications assigned a score of 0), the number of over-refusals, defined as control scenarios with no contraindication that the LLM incorrectly classified as an absolute contraindication (score of 2), and weighted accuracy based on mean absolute error (MAE) on the ordinal 0–2 scale, calculated as (1 − MAE/2) × 100. All model responses were evaluated as initially generated. If an over-refusal occurred, the model was not reprompted or instructed to reconsider or override its initial classification. For within-model baseline versus RAG comparisons, the number of discordant classifications (scenarios corrected by RAG and scenarios worsened by RAG) was reported alongside statistical testing. Statistical analysis Within-model comparisons of baseline versus RAG accuracy were performed using McNemar's exact test for paired proportions, with Bonferroni correction. The Wilcoxon signed-rank test compared absolute error magnitudes between conditions. Between-model comparisons within each condition used Cochran's Q test; when significant, post-hoc pairwise McNemar tests were conducted with Bonferroni-adjusted significance thresholds. Agreement between LLM scores and IPSIS reference classifications was assessed using weighted Cohen's kappa. Statistical significance was set at α = 0.05 after correction. Analyses were performed using Python 3.12 with SciPy, statsmodels, and scikit-learn. Results Overall classification accuracy A total of 1908 classifications were completed (318 scenarios × 2 conditions × 3 models). Baseline accuracy was 85.8% for both GPT-5.4 (273/318) and Claude Sonnet 4.6 (273/318) and 87.1% for Gemini 3.1 Pro (277/318), with a mean accuracy across models of 86.3% (SD 0.7) (Table 2). With RAG, accuracy improved to 94.7% for GPT-5.4 (301/318), 98.4% for Gemini 3.1 Pro (313/318), and 92.5% for Claude Sonnet 4.6 (294/318), with a mean of 95.2% (SD 3.0). McNemar's test confirmed that RAG significantly improved accuracy for all three models (GPT-5.4, p < 0.001; Gemini 3.1 Pro, p < 0.001; Claude Sonnet 4.6, p = 0.002; all Bonferroni-corrected). RAG corrected more classifications than it worsened for each model (Table S3). Weighted kappa ranged from 0.885 to 0.913 at baseline and improved to 0.949–0.990 with RAG (Table 2).Table 2Overall accuracy and agreement with evidence-informed reference standard.Table 2ModelConditionCorrectNAccuracy (%)MAEWeighted κGPT-5.4Baseline27331885.80.1570.885GPT-5.4RAG30131894.70.0600.954Gemini 3.1 ProBaseline27731887.10.1350.913Gemini 3.1 ProRAG31331898.40.0160.990Claude Sonnet 4.6Baseline27331885.80.1540.890Claude Sonnet 4.6RAG29431892.50.0790.949MAE = mean absolute error on the 0-2 ordinal scale (lower is better). Weighted κ = weighted Cohen's kappa. RAG = retrieval-augmented generation. Table 2Overall accuracy and agreement with evidence-informed reference standard.Table 2ModelConditionCorrectNAccuracy (%)MAEWeighted κGPT-5.4Baseline27331885.80.1570.885GPT-5.4RAG30131894.70.0600.954Gemini 3.1 ProBaseline27731887.10.1350.913Gemini 3.1 ProRAG31331898.40.0160.990Claude Sonnet 4.6Baseline27331885.80.1540.890Claude Sonnet 4.6RAG29431892.50.0790.949MAE = mean absolute error on the 0-2 ordinal scale (lower is better). Weighted κ = weighted Cohen's kappa. RAG = retrieval-augmented generation. Overall accuracy and agreement with evidence-informed reference standard. MAE = mean absolute error on the 0-2 ordinal scale (lower is better). Weighted κ = weighted Cohen's kappa. RAG = retrieval-augmented generation. Accuracy by contraindication category Performance varied substantially by contraindication type (Table 3). Absolute contraindications were classified with the highest accuracy at baseline, ranging from 88.2% (Claude Sonnet 4.6) to 97.1% (Gemini 3.1 Pro). With RAG, accuracy for absolute contraindications increased to 95.1% (GPT-5.4), 99.0% (Gemini 3.1 Pro), and 100.0% (Claude Sonnet 4.6).Table 3Classification accuracy by contraindication category (%).Table 3ModelConditionAbsolute (n = 102)Relative (n = 57)Control (n = 159)GPT-5.4Baseline89.256.194.3GPT-5.4RAG95.182.598.7Gemini 3.1 ProBaseline97.142.196.9Gemini 3.1 ProRAG99.093.0100.0Claude Sonnet 4.6Baseline88.252.696.2Claude Sonnet 4.6RAG100.059.699.4Values represent the percentage (%) of cases correctly classified within each category. Table 3Classification accuracy by contraindication category (%).Table 3ModelConditionAbsolute (n = 102)Relative (n = 57)Control (n = 159)GPT-5.4Baseline89.256.194.3GPT-5.4RAG95.182.598.7Gemini 3.1 ProBaseline97.142.196.9Gemini 3.1 ProRAG99.093.0100.0Claude Sonnet 4.6Baseline88.252.696.2Claude Sonnet 4.6RAG100.059.699.4Values represent the percentage (%) of cases correctly classified within each category. Classification accuracy by contraindication category (%). Values represent the percentage (%) of cases correctly classified within each category. Relative contraindications showed the lowest baseline accuracy across all models: 42.1% for Gemini 3.1 Pro, 52.6% for Claude Sonnet 4.6, and 56.1% for GPT-5.4. RAG improved relative contraindication accuracy to 93.0% for Gemini 3.1 Pro, 82.5% for GPT-5.4, and 59.6% for Claude Sonnet 4.6. Control scenarios were accurately classified in 94.3% to 96.9% of scenarios at baseline, improving to 98.7% to 100.0% with RAG. Safety metrics Safety catch rates for absolute contraindications were 99.0% to 100.0% across all models and conditions (Table S4). Claude Sonnet 4.6 was the only model to miss an absolute contraindication at baseline (1 of 102, 1.0%); no absolute contraindications were missed under RAG by any model. McNemar's test on the safety catch rate was not significant for any model (all p > 0.99), reflecting high baseline performance. Over-refusals, in which a control scenario was incorrectly classified as an absolute contraindication, ranged from 2–5 at baseline and decreased from 0–2 with RAG. Distance-based accuracy Mean absolute error on the ordinal scale decreased from 0.135 to 0.157 at baseline to 0.016–0.079 with RAG. Corresponding weighted accuracy improved from 92.1% to 93.2% at baseline to 96.1%–99.2% with RAG. The Wilcoxon signed-rank test confirmed that RAG significantly reduced error magnitude for all models (GPT-5.4, p < 0.001; Gemini 3.1 Pro, p < 0.001; Claude Sonnet 4.6, p < 0.001; all Bonferroni-corrected). Between-model comparisons At baseline, Cochran's Q test did not detect a significant difference in accuracy among the three models (Q = 0.62, p = 0.73). Under RAG, a significant difference was observed (Q = 18.47, p < 0.001). Post-hoc pairwise analysis revealed that Gemini 3.1 Pro (98.4%) significantly outperformed both Claude Sonnet 4.6 (92.5%, p < 0.001) and GPT-5.4 (94.7%, p = 0.01) under RAG (all Bonferroni-corrected). The GPT-5.4 versus Claude Sonnet 4.6 comparison did not reach significance (Table S3). Discussion This study evaluated the accuracy of three contemporary LLMs in classifying contraindications to interventional spine procedures against evidence-informed IPSIS Safety Practices. All three models achieved baseline accuracy exceeding 85% and demonstrated high accuracy for absolute contraindications. Guideline augmentation through RAG significantly improved overall accuracy and ordinal agreement for all models. A notable finding was the differential performance across contraindication categories. All models reliably identified absolute contraindications at baseline, with safety catch rates of 99% to 100%. While the internal representations of the evaluated LLMs are not directly observable, this pattern is consistent with strong performance on broadly recognized safety principles, such as avoiding procedures in the setting of active infection, hypertensive emergency, or inability to obtain informed consent [14]. These high-level contraindications are widely emphasized in clinical training and literature and are therefore more likely to be reflected in model outputs. Relative contraindications presented a considerably greater challenge, with baseline accuracy as low as 42.1%. Many relative contraindications require knowledge of procedure-specific nuances. For example, pregnancy is classified as a relative contraindication for epidural steroid injections but an absolute contraindication for radiofrequency neurotomy and neuromodulation procedures. These distinctions are explicitly defined in the evidence-informed guidelines but may not be consistently represented in general LLM training data [11]. RAG was effective in addressing this gap, with improvements observed for all models. These findings extend the growing body of literature on LLMs in musculoskeletal and spine care. Prior studies have evaluated LLM performance in answering patient questions about low back pain, generating clinical guidelines for spine surgery, and providing patient education about epidural steroid injections [[8], [9], [10],15,16]. The present study differs in its focus on clinical safety decision-making rather than patient communication, and in its use of evidence-informed safety documents as the explicit reference standard. Under RAG, a significant between-model difference emerged. Gemini 3.1 Pro achieved the highest accuracy (98.4%) and significantly outperformed both Claude Sonnet 4.6 and GPT-5.4 in post-hoc analysis, while no models differed significantly at baseline. This suggests that models may vary in their ability to integrate and apply externally retrieved guideline text, even when their baseline medical knowledge is comparable. Model selection should therefore be an important consideration when deploying RAG-based clinical decision support systems. In addition, future work should evaluate more structured approaches, including advanced prompting strategies, tool-augmented retrieval, and domain-specific fine-tuning, which may further improve reliability beyond standard RAG [11]. The role of multimodal inputs and integration within existing clinical systems also warrants investigation, particularly with respect to data privacy considerations [17]. Prospective clinical validation will be important to determine whether these systems meaningfully improve decision-making while preserving patient privacy in real-world practice. Several limitations should be considered. First, the case scenarios were derived from guideline statements and may not capture the full complexity of real clinical encounters, in which multiple comorbidities, ambiguous presentations, and patient preferences interact. However, this design was intentional and is consistent with prior LLM evaluation studies that use structured clinical scenarios to assess guideline adherence [7]. As a result, the findings should be interpreted as performance on structured contraindication recognition rather than as evidence of real-world clinical decision-making accuracy. Second, the study evaluated a single prompt format under controlled conditions; variations in prompt phrasing, model version, and temperature settings may yield different results. Third, the IPSIS Safety Practices represent one society's consensus and may not be universally adopted [5]. Lastly, the scenarios were evaluated at a single time point, and model performance may change with future updates. Conclusion LLMs demonstrated high baseline accuracy and strong classification performance for absolute contraindications to interventional spine procedures per evidence-informed safety practice guidelines. Classification of relative contraindications was lower at baseline but improved significantly with RAG. These findings support the potential of guideline-augmented LLMs as a component of pre-procedure safety screening in interventional pain practice. Conflict of interest statement The authors declare no conflicts of interest. No funding was received for this work. The authors have no financial or other relationships that could be perceived as influencing the objectivity of this research.