Artificial Intelligence for American Society of Anesthesiologists Physical Status Classification: Agreement with Clinician Consensus and Temporal Stability Analysis Authors: Anne Lykke Soerensen, Christina Froeslev-Friis, Gunhild Kjaergaard Andersen, Swati Bhavsar, Lisbeth Holmgaard Quitzau, Vinayak Desurkar, Anil Rao, Sri Krishna, Reshma Bhosale, Rajesh Prabhakar Bhavsar Published: 2026 ABSTRACT Background: The American Society of Anesthesiologists Physical Status (ASA-PS) classification is widely used for perioperative communication and research with known variation in agreement amongst assessors. Large language models (LLM) are increasingly considered for uniform decision making due to agreement challenges within clinicians under identical inputs. The study compared four contemporary LLMs against clinician-derived consensus and quantified within-model stability across repeated assessments. Methods: In a cross-sectional vignette study, 228 anesthesiologists from Denmark, England, and India classified 20 standardized perioperative vignettes through online survey. The modal response per vignette was considered as clinician consensus. Four LLMs (ChatGPT-5.2 Plus, Gemini 3 Pro, Perplexity AI Pro, Claude 4 Sonnet) received same set of vignettes through identical prompts. Temporal stability was assessed by repeating each vignette query nine times per model (three-time windows across three nonconsecutive days) in fresh sessions. Primary outcome was exact agreement with clinician consensus. Results: Consensus agreement for modal LLM classifications was 18/20 (90%) for ChatGPT, 17/20 (85%) for Gemini, 17/20 (85%) for Claude, and 15/20 (75%) for Perplexity. Disagreement is clustered in vignettes with weak or split clinician consensus. Stability differed by model: Claude was fully stable across all vignettes (20/20), Gemini 19/20, ChatGPT 18/20, and Perplexity 14/20; instability typically involved adjacent-class shifts. Conclusions: Contemporary LLMs often match clinician modal judgement and are largely temporally stable, with discordance concentrated in clinically ambiguous boundary cases. 1. Introduction The American Society of Anesthesiologists Physical Status (ASA-PS) classification system is a globally used scale intended to describe preoperative health status and systemic disease burden of the patient prior to anesthesia. It categorizes patients from ASA-PS I to VI and allows an “E” modifier for emergency procedures. Although frequently interpreted as a perioperative “risk score,” ASA-PS was originally developed as a descriptive classification for standardization and reporting, and its simplicity remains both a strength and a limitation [1]. Despite its conceptual simplicity, ASA-PS has demonstrated meaningful associations with perioperative outcomes and is therefore embedded in routine pre-anesthetic assessment, perioperative communication, and observational research [2]. In contemporary practice, ASA-PS is routinely recorded in anesthesia information systems and is widely used for case-mix description and risk adjustment in registries and clinical studies [3]. In several healthcare systems, it is also applied in operational and administrative contexts, including preoperative triage pathways and, in some settings, reimbursement modifiers that assume increasing ASA-PS class reflects increasing anesthetic workload and resource utilization. This extended use implicitly requires that ASA-PS be assigned accurately, consistently and reproducibly [4]. Using ASA-PS as a proxy for patient severity to guide resource allocation has intuitive appeal because higher ASA-PS classes correlate, on average, with worse outcomes. However, ASA-PS does not explicitly incorporate procedure-specific risk, frailty, functional capacity, or surgical invasiveness. Consequently, reliance on ASA-PS for high-stakes downstream decisions such as triage level, postoperative destination, benchmarking, or administrative categorization, may be particularly vulnerable to misclassification [5]. Administrative reliance may also introduce incentive structures around scoring. Work on potential payer-linked upcoding illustrates why reproducibility matters when ASA-PS influences operational or financial decisions, even in the absence of overt systematic fraud [6]. There is extensive literature demonstrating, at best, moderate inter-rater reliability for ASA-PS in clinical practice, underscoring its dependence on clinical judgment [7]. Disagreement is most pronounced at common category boundaries (II vs. III; III vs. IV), where clinicians may differ in their interpretation of systemic disease severity, disease control, and functional limitation [8]. Variability has also been observed across provider groups and clinical contexts, including discordance between surgeons and anesthesiologists, underscoring that ASA-PS may function in practice as a clinician-dependent judgment, particularly in borderline cases [9]. Efforts to improve consistency, such as education and consultation of official definitions, can increase accuracy but do not eliminate disagreement [10]. Recent advances in machine learning and natural language processing have enabled development of models that assign or predict ASA-PS using structured electronic health record variables or pre-anesthetic text summaries, explicitly aiming to address limitations in human reproducibility [11]. At the same time, large language models based on artificial intelligence (AI) systems, have demonstrated vulnerabilities including hallucinations, instruction sensitivity, and hidden biases [12]. As these models evolve rapidly, it remains unclear whether newer iterations achieve improved agreement with clinicians, greater internal stability, or simply different patterns of inconsistency when compared with human judgment. Accordingly, this study evaluates multiple contemporary AI models under identical conditions, comparing their ASA-PS classifications with a clinician-derived consensus and assesses the temporal stability of AI-generated assignments across repeated assessments. 2. Materials and Methods This was a cross-sectional, survey-based study designed to evaluate inter-clinician variability in ASA-PS classification and to compare clinician-derived consensus with outputs from multiple contemporary LLMs. In addition, AI models were evaluated using a repeated-measures design to assess the temporal stability of ASA-PS assignments under identical input conditions. The clinical vignettes were intentionally constructed by the author group as standardized test cases rather than as an exhaustive representation of perioperative scenarios. The objective was to enable controlled assessment of inter-clinician agreement and temporal variability in AI outputs under comparable conditions, rather than to assess performance across the full spectrum of real-world clinical complexity. The study protocol was prospectively registered on the Open Science Framework (OSF). Twenty clinical case vignettes were developed to reflect a broad range of preoperative scenarios commonly encountered in anesthesia practice. Each vignette included sufficient demographic and clinical information to permit ASA-PS classification (I–IV, with or without emergency modifier [E]). ASA-PS I and V cases were intentionally excluded to focus on routine elective and semi-urgent perioperative assessments. Vignettes were prepared in both English and Danish (Supplementary Material Files S1 and S2). Content equivalence between language versions was ensured through forward translation followed by independent clinical review. Six senior anesthesiologists with more than 10 years of clinical experience (two each from Denmark, England, and India) independently reviewed all vignettes to confirm clinical realism, clarity, and sufficiency of information for ASA-PS assignment. Final vignette versions were agreed upon by consensus among the author group following iterative review. These reviewers did not contribute to outcome determination, and their assessments were not used as reference classifications. Anesthesiologists from multiple hospitals across India, Denmark, and England were invited to participate through professional networks. Recruitment followed a pragmatic online approach using snowball sampling, whereby initial recipients were encouraged to forward the questionnaire to eligible colleagues within their professional circles. Data collection was conducted over a predefined period from 1 November 2025 to 1 December 2025. Eligible participants who completed the survey within this period were practicing anesthesiologists involved in perioperative care. No formal exclusion criteria were applied. All responses received during this period were included in the analysis, and no formal sampling frame or random selection was applied. Participants were drawn from multiple countries and practice settings, introducing a degree of heterogeneity in clinical background. However, detailed demographic variables such as years of experience, subspecialty, and institutional characteristics were not systematically collected and are acknowledged as limitations. Participants completed an anonymous online survey in which they independently assigned an ASA Physical Status (ASA-PS) class (I–IV, ±E) to each of the 20 standardized clinical vignettes. Participants were blinded to AI-generated classifications and to the responses of other clinicians. All respondents assessed the same set of vignettes, ensuring that comparisons were based on identical clinical information across participants. The use of standardized vignettes was intended to minimize variability related to local clinical environments and enhance comparability of ASA-PS assessments. No formal a priori sample size calculation was performed. The study was designed as a pragmatic cross-sectional survey and participant inclusion was based on responses received during a predefined recruitment period. All eligible anesthesiologists who completed the survey within this period were included in the final analysis. The study aim was descriptive and comparative, focusing on agreement between ASA-PS classifications assigned to standardized vignettes rather than testing a single prespecified hypothesis. The reference standard for analysis was defined as the modal ASA-PS classification assigned by the participating anesthesiologists (n = 228) for each vignette. This approach reflects collective clinical judgement rather than a definitive measure of clinical accuracy. Response options included most relevant ASA-PS categories (I–IV), together with two additional verification options (VO): (1) request for additional information and (2) request for physical patient assessment. Selection of multiple options was permitted to reflect real-world clinical decision-making under uncertainty. This response structure was finalized following consensus discussion among the six senior anesthesiologists and was applied identically to human respondents and AI models. For each vignette, the majority clinician responses were used to define the consensus ASA-PS classification. In cases of tied responses between ASA-PS classes, the lower class was selected to reflect conservative clinical judgment. In the event of a tie between emergency (E) and non-emergency classifications within the same class, the emergency designation was selected. The resulting clinician consensus served as the reference comparator for AI-generated classifications. Four contemporary web-based large language model platforms were evaluated: ChatGPT-5.2 Plus, OpenAI, San Francisco, CA, USA; Gemini 3 Pro, Google LLC, Mountain View, CA, USA; Claude 4 Sonnet, Anthropic PBC, San Francisco, CA, USA; and Perplexity AI Pro, Perplexity AI, Inc., San Francisco, CA, USA. The models were accessed through their publicly available web interfaces between 16 and 21 January 2026. Because these platforms are web-based systems and exact backend version identifiers and generation parameters are not fully user-controlled, the model names displayed in the user interface at the time of access were recorded as the software/model versions. These models were selected to represent widely accessible, general-purpose large language models from independent developers that are commonly used by clinicians in real-world settings. They differ in underlying design philosophy, safety orientation, and information retrieval strategies, allowing comparison across heterogeneous AI systems rather than variants of a single architecture. Model selection was therefore based on clinical relevance, diversity of AI approaches, and methodological feasibility rather than on theoretical performance benchmarking. The models were accessed via publicly available web-based interfaces, where generation parameters and exact version identifiers are not user-controlled. To address potential variability, each vignette was evaluated repeatedly at predefined time points using identical prompts. This design enabled assessment of temporal stability under real-world conditions. Each model received the same 20 vignettes using a fully standardized prompt (Table 1). The prompt instructed the model to assign at least one ASA-PS category unless classification was impossible and allowed parallel selection of VOs. Agreement proportions were calculated for each model, and 95% confidence intervals were estimated using a binomial distribution. To evaluate temporal stability of AI-generated ASA-PS classifications, each model was queried nine times per vignette across three predefined time-of-day windows on three nonconsecutive days (3 × 3 design), using identical prompts in new, independent sessions to avoid contextual carryover. Queries were scheduled according to Copenhagen time and distributed across periods of presumed lower and higher system usage to approximate real-world conditions (Supplementary Material File S6). For each vignette and model, a modal ASA-PS class was defined as the most frequently selected classification across the nine runs. Run-level outputs, including verification option selections, were recorded, and agreement with clinician consensus was quantified per vignette and model (0–9 concordant runs). Complete run-level outputs and agreement metrics are provided in Supplementary Materials Files S3 and S4. ASA-PS classification and VOs were treated as orthogonal outputs. ASA-PS consensus calculations and agreement analyses were based exclusively on selected ASA-PS categories, independent of any VOs. VOs were analyzed separately as markers of residual uncertainty. Primary outcomes were agreement between AI-generated ASA-PS classifications and clinician-derived consensus, expressed as percentage agreement. Secondary analyses included temporal consistency of AI responses across repeated assessments, patterns of disagreement between adjacent ASA-PS classes (e.g., II vs. III), frequency of emergency modifier assignment, and differences in agreement and stability across models. Vignettes were categorized based on the level of clinician consensus, defined pragmatically as strong (≥70% agreement), moderate (55–69%), and weak/split (<55%). These thresholds were selected for descriptive purposes within the dataset and do not represent validated or universally established cut-offs. The whole flow of study is shown in Figure 1. Although the study was not prospectively designed using the CHERRIES checklist, key recommended elements were addressed in the design and reporting of this online survey. 3. Results The analyses were structured hierarchically. Clinician-derived consensus served as the reference standard (Table 2). Intrinsic AI response behavior was characterized independently (Table 3). Agreement between AI and clinician consensus was assessed at the vignette level using modal ASA-PS classifications (Table 4). The internal reliability of AI modal classifications and their agreement with clinician consensus across repeated runs are reported in the Supplementary Materials (Supplementary Material File S4). A total of 228 anesthesiologists completed the survey and were included in the analysis, comprising 180 non-Danish and 48 Danish respondents. All participants assessed all 20 vignettes. For each vignette, the most frequently selected ASA-PS class constituted the clinician consensus (Table 1). VOs, defined as requests for additional information or physical examination, were analyzed separately and could co-occur with ASA-PS grading. Based on the proportion of respondents selecting the modal ASA-PS class, five vignettes (1, 4, 12, 13, and 15) demonstrated strong consensus (≥70%), eight (2, 3, 8, 9, 10, 11, 16, and 18) moderate consensus (55–69%), and seven (5, 6, 7, 17, 19, and 20) weak or split consensus (<55%). Full distributions of human responses, including VO selection, are provided in Supplementary Material File S2. Each AI model generated an ASA-PS classification for every vignette in every run. For each vignette and model, a modal ASA-PS class was defined as the most frequently selected class across the nine repeated runs (Table 2). In many vignette–model combinations, the modal ASA-PS class was selected in all 9/9 runs, indicating fully stable internal AI consensus. In contrast, a small number of borderline vignettes showed lower internal agreement, with modal selections occurring in as few as 5/9 runs (≈55%), reflecting sensitivity to interpretive ambiguity. Complete run-level AI outputs and VO selections across all nine assessments are provided in Supplementary Material File S3. Run-level agreement with clinician consensus for each model and vignette (0–9/9 concordant runs) is provided in Supplementary Material File S4. Exact agreement between the AI modal ASA-PS class and the clinician consensus classification across all 20 vignettes is shown in Supplementary material File S5. ChatGPT agreed with the clinician consensus in 18/20 vignettes (90%; 95% CI 68–99%), Gemini and Claude both in 17/20 vignettes (85%; 95% CI 62–97%), and Perplexity in 15/20 vignettes (75%; 95% CI 51–91%). When stratified by clinician consensus strength, agreement was highest in vignettes with strong consensus, remained high in moderate-consensus vignettes, and decreased in weak or split-consensus scenarios (Table 4). Accordingly, AI–human discordance was largely confined to cases in which clinicians themselves demonstrated limited agreement. Confidence intervals overlapped across models, indicating that differences in agreement should be interpreted descriptively rather than as statistically significant. The overlap of confidence intervals suggests that observed differences between models may reflect sampling variability rather than true performance differences. Across repeated assessments, AI output stability differed between models but was largely vignette-specific rather than random. Claude demonstrated complete vignette-level stability, assigning the same ASA-PS class in all nine runs for all 20 vignettes (20/20). Gemini was stable in 19/20 vignettes, ChatGPT in 18/20 vignettes, and Perplexity in 14/20 vignettes (Supplementary Material File S4). Where instability occurred, it most often reflected occasional shifts between adjacent ASA classes or parallel changes in VOs rather than large category transitions (Supplementary Material File S4). Importantly, in vignettes where AI models consistently disagreed with clinician consensus (notably vignettes 5, 10, and 20), disagreement was reproducible across all nine runs for each model, indicating stable alternative interpretations rather than stochastic variation. 4. Discussion This study examined how consistently contemporary large language models assign ASA-PS classifications and how closely their outputs align with the modal judgement of practicing anesthetists when applied to standardized written vignettes. Overall, agreement between AI models and clinician consensus was high, and most AI outputs were highly reproducible across repeated assessments. Importantly, disagreement between AI and humans was not randomly distributed but concentrated in the same vignettes where clinicians themselves showed limited agreement. A substantial body of literature has demonstrated that inter-rater reliability for ASA-PS classification in routine clinical practice is moderate at best, reflecting the system’s reliance on subjective clinical judgement. Disagreement is particularly common at boundaries between adjacent categories, most notably between ASA-PS II and III and between ASA-PS III and IV, where interpretation of systemic disease severity, disease control, and functional limitation may differ between clinicians [5]. In scenarios with strong human consensus, agreement between AI and clinicians was uniformly high across all models. Conversely, the vignettes that generated systematic AI–human discordance were those characterized by weak or split human consensus. This pattern indicates that disagreement primarily reflects intrinsic ambiguity in the vignette descriptions rather than gross model error. In other words, AI–human friction emerged precisely at points where ASA-PS classification is already challenging for clinicians [13,14]. Prior studies have similarly shown that ASA-PS classification variability increases in patients with multiple comorbidities, borderline functional capacity, or partially controlled disease, suggesting that disagreement is driven more by case complexity than by lack of familiarity with ASA-PS definitions. For example, a large German study including 684 adult general surgical patients likewise reported decreasing agreement with increasing numbers of systemic diseases. Older patient age (≥75 years) was likewise associated with lower agreement. Interestingly, agreement did not differ significantly between junior and senior anesthesiologists, indicating that ASA-PS classification need not be restricted to more experienced clinicians. A notable finding was that both clinicians and several AI models frequently paired a definite ASA-PS classification with explicit signals of uncertainty. In weak-consensus vignettes, clinicians still assigned ASA grades but distributed them across adjacent categories, while many simultaneously indicated a need for additional information or physical examination. Gemini, ChatGPT, and Perplexity showed similar surface behavior in a subset of these borderline cases, whereas Claude consistently produced definite classifications without uncertainty modifiers. However, the meaning of uncertainty differs fundamentally between humans and AI. For clinicians, requesting additional information or physical examination represents an actionable plan that may alter the final ASA-PS classification. For AI models, uncertainty modifiers cannot trigger further data acquisition and therefore function only as indicators that the provided text is insufficiently discriminative. In this context, AI uncertainty should be interpreted as a flag for human review rather than as a directive for a specific next step [15]. Future systems could enhance clinical usefulness by indicating which information domains—such as functional capacity, disease stability, or recent investigations—are driving the classification ambiguity. The observed instability in a minority of vignette–model combinations is best understood as sensitivity to borderline definitions rather than random fluctuation. When a vignette clearly satisfies the textual criteria for a single ASA-PS class, all models repeatedly return the same classification. When the description lies close to a class boundary, small differences in how individual words or phrases are weighted can occasionally shift the classification to an adjacent category on repeated runs. Thus, limited instability reflects responsiveness to ambiguity in the input rather than unreliability. Differences between models appear to reflect underlying design choices. More deterministic systems, exemplified here by Claude, commit to a single dominant interpretation and therefore achieve perfect reproducibility, even when that interpretation differs from the human modal response in some borderline cases. Other models, such as ChatGPT and Gemini, occasionally move between neighboring classes, sometimes aligning and sometimes diverging from the clinician consensus. Perplexity both disagreed more frequently and fluctuated more in ambiguous scenarios. These patterns suggest stable, model-specific interpretations of incomplete descriptions rather than erratic behavior. The three vignettes that showed complete and reproducible disagreement across all models are particularly informative. These cases likely represent borderline ASA-PS constructs where routine clinical gestalt and literal text-based interpretation diverge in a systematic manner. Such findings highlight that clinician consensus itself does not represent an objective ground truth but rather a collective professional judgement that may incorporate tacit assumptions not explicitly present in the vignette text [2,5]. From a practical perspective, two properties are central: Concordance with typical clinician judgement across cases and reproducibility of outputs when reassessed. Perfect reproducibility alone does not guarantee agreement with clinicians, as illustrated by stable but discordant outputs in some borderline vignettes. Conversely, limited shifts between adjacent classes can sometimes increase agreement with human consensus, albeit at the expense of strict repeatability. These trade-offs are important when considering how AI systems might be used in clinical practice. ASA-PS is increasingly used beyond its original descriptive purpose, including for case-mix adjustment, benchmarking, triage pathways, and administrative categorization. In such contexts, misclassification, or inconsistent assignment may have downstream operational or interpretive consequences, underscoring the importance of reproducibility, and transparency in ASA-PS assignment [16]. Within the scope of a vignette-based evaluation, the more reproducible models, particularly Claude, and to a large extent Chat GPT and Gemini, appear suitable as supportive tools for low-stakes applications such as education, documentation standardization, and preliminary flagging of potentially higher-risk cases. However, systematic differences in complex scenarios, frequent uncertainty signals in ambiguous cases, and sensitivity to incomplete textual descriptions indicate that fully autonomous ASA-PS assignment would be premature. AI-generated ASA-PS classifications should therefore be treated as provisional suggestions requiring human confirmation, especially in borderline or high-risk patients. Limitations This study has several limitations. First, no formal a priori sample size calculation was performed, as recruitment was based on a predefined survey period. However, the inclusion of 228 anesthesiologists assessing standardized vignettes generated a substantial number of observations for comparative analyses. Second, the ASA-PS classification lacks an objective gold standard; accordingly, the reference standard was defined as the modal classification among participating anesthesiologists, reflecting collective clinical judgement rather than a definitive measure of accuracy. Third, the use of standardized vignettes, while enabling controlled comparisons, does not capture the full complexity of real-world clinical assessment, including dynamic patient interaction, longitudinal information, and contextual decision-making, which may limit generalizability. Fourth, recruitment via snowball sampling and inclusion of participants from selected countries may introduce selection bias and limit representation of global practice patterns. Fifth, the analysis focused on exact agreement and descriptive comparisons. The thresholds used to define consensus strength were pragmatic and dataset-specific, and more advanced statistical approaches (e.g., weighted agreement metrics or pairwise model comparisons) were not performed. Both factors may influence interpretation and could provide additional insight in future studies. Sixth, the study did not provide standardized ASA-PS classification examples to participants during scoring. Prior studies have shown that the inclusion of examples can improve consistency of ASA-PS assignment; therefore, variability in clinician responses in the present study may partly reflect the absence of such guidance [17,18]. Finally, AI models were evaluated at predefined time points using fixed prompts and model behaviour may change over time with updates or differ under alternative deployment conditions; therefore, findings reflect model performance at the time of assessment. 5. Conclusions In summary, contemporary AI systems can generate clinically plausible ASA-PS classifications that often align with the modal judgement of experienced anesthesiologists and demonstrate high internal reproducibility. Importantly, AI–human disagreements were concentrated in scenarios where clinicians themselves showed limited consensus. These findings suggest that such systems may serve as assistive second opinions and detectors of ambiguity but do not support their use as replacements for clinical judgement. Future research should focus on prospective, real-time comparisons between clinician and AI-generated ASA-PS classifications in routine clinical practice, incorporating dynamic patient interaction, and contextual clinical information to better reflect real-world decision-making.