Roadmap for the evolution of monitoring: developing and evaluating waveform-based variability-derived artificial intelligence-powered predictive clinical decision support software tools Authors: Andrew J E Seely, Kimberley Newman, Rashi Ramchandani, Christophe Herry, Nathan Scales, Natasha Hudek, Jamie Brehaut, Daniel Jones, Tim Ramsay, Doug Barnaby, Shannon Fernando, Jeffrey Perry, Sonny Dhanani, Karen E A Burns Published: 2024 ABSTRACT AbstractBackgroundContinuous waveform monitoring is standard-of-care for patients at risk for or with critically illness. Derived from waveforms, heart rate, respiratory rate and blood pressure variability contain useful diagnostic and prognostic information; and when combined with machine learning, can provide predictive indices relating to severity of illness and/or reduced physiologic reserve. Integration of predictive models into clinical decision support software (CDSS) tools represents a potential evolution of monitoring.MethodsWe perform a review and analysis of the multidisciplinary steps required to develop and rigorously evaluate predictive clinical decision support tools based on monitoring.ResultsDevelopment and evaluation of waveform-based variability-derived predictive models involves a multistep, multidisciplinary approach. The stepwise processes involves data science (data collection, waveform processing, variability analysis, statistical analysis, machine learning, predictive modelling), CDSS development (iterative research prototype evolution to commercial tool), and clinical research (observational and interventional implementation studies, followed by feasibility then definitive randomized controlled trials), and poses unique challenges (including technical, analytical, psychological, regulatory and commercial).ConclusionsThe proposed roadmap provides guidance for the development and evaluation of novel predictive CDSS tools with potential to help transform monitoring and improve care.Keywords: Clinical decision support, Artificial intelligence, Waveform monitoring, Variability analysis, Heart rate variability, Respiratory rate variability, Critical illness Introduction Waveform monitoring is routinely performed for patients with impending or existing critical illness. Continuous vital sign monitoring typically involves recording of heart rate with single or multi-lead electrocardiogram (ECG), oxygen saturation with pulse oximetry, respiratory rate through chest impedance or capnography, and arterial blood pressure measured intermittently or continuously using an arterial catheter. Measured hundreds of times per second, routine vital sign waveform monitoring provides an immediate ability to detect change in a patient’s status, such as an alteration in cardiac rhythm, development of life-threatening abnormal vital signs requiring immediate intervention, and tracking responses to therapeutic interventions, and more. Monitoring is essential to evaluate and manage the status of patients who are critically ill or at risk for clinical deterioration. Continuous cardiopulmonary monitoring enables monitoring of what is occurring to enable timely and appropriate treatment. With over five decades of practice demonstrating its foundational value, it would be inconceivable to manage operative or critically ill patients without waveforms. Nevertheless, especially given its pervasiveness and longevity, it is remarkable that waveform monitoring has remained fundamentally unchanged since its inception. Greater integration of information over time and over populations of patients is required to improve monitoring. One approach to improve monitoring is to both accept and try to reduce uncertainty of the future, in addition to monitoring the present. Uncertainty in diagnosis, prognosis, and therapeutic response is undeniably present, and recognition of this scientific fact is both essential and helpful [1]. Indeed, uncertainty remains a particularly active problem in Emergency Departments (EDs) and intensive care units (ICUs). For example, uncertainty in diagnosis of infection that leads to late diagnosis and treatment increases mortality risk and healthcare costs [2–6], with sepsis (infection leading to organ failure) remaining a costly healthcare burden worldwide [7–11]. Moreover, uncertainty can increase medical costs as the most “expensive” ICU patients are those with indeterminate outcomes [12, 13]. Uncertainty also leads to medical inefficiencies, which in turn, impact quality of patient care. It is estimated that 21 to 47% of the healthcare spending in the US pertains to interventions that have no value [14]. Finally, uncertainty can greatly impact clinical decision making, ultimately impacting short-term and long-term health outcomes. For example, uncertainty in determining optimal timing for removal of an endotracheal tube lead to patient harm and wasted resources [15–24]. This is clinically important given that approximately 15%-20% of invasively ventilated patients fail extubation requiring emergency re-intubation [18]. To mitigate these pitfalls, a growing area of research seeks to derive greater value from monitoring by developing predictions from monitored data to help reduce future uncertainty. Integrating all information within intervals in time, variability analysis offers a means to track the underlying integrity of the whole complex system producing these dynamics [25]. Rather than discard the data, the analysis of patterns of variation over time represents a novel dimension of analysis, with potential to augment the value of routine data monitoring. Increasing evidence supports the association between variability analysis and health outcomes [26, 27]. For example, in ICUs, alterations in heart rate variability (HRV) have been linked with early warning of sepsis [28–30] and septic shock [31, 32]. In the ED, HRV analysis has also shown prognostic capabilities in adults with sepsis as an early marker of multiple organ dysfunction syndrome [33] and early predictor of death [34]. Similar findings have been noted with respiratory rate variability (RRV) in critically ill patients. Defined as the breath-to-breath changes in respiration, a reduction in RRV was found to be associated with organ failure [35], restrictive lung disease [36], extubation failure [44, 45] and a marker of stress during weaning from mechanical ventilation [37–39]. Although less well investigated, similar associations have been noted for loss of blood pressure, temperature, and glucose variability, and key outcomes including severity of organ damage, cardiovascular complications, and mortality [40, 41]. These studies and more demonstrate loss of degree and complexity of variability in association with illness, occurring with loss of function, reduced adaptability, reduced functional reserve. To help translate variability measures into clinically actionable information, variability may be converted into prediction to reduce uncertainty and help influence care. Given there are a broad array of variability metrics that evaluate many dimensions of variation (including overall degree of variation, proportion of high and low frequency variation, degree of irregularity or information or complexity), and different measures may be important in different clinical scenarios, variability metrics may be converted into clinically intelligible predictions or scores that relate to specific clinical scenarios and outcomes through artificial intelligence predictive modelling performed on large population studies. Predictive scores must then be integrated into clinical decision support tools to directly influence care. This integration over populations of patients (predictive modelling) is complimentary to the integration over time (variability analysis), and offers a means to help transform monitoring into actionable benefical information, representing a bourgening frontier in ED and ICU research. To further stimulate and support research and development in this novel domain, we propose a systematic roadmap outlining the methods to develop and evaluate waveform-based, variability-derived, artificial intelligence-powered clinical decision support tools (Fig. 1). Roadmap for Development and Evaluation of a Waveform-based Clinical Decision support Software (CDSS) tool (RCT = Randomized Controlled Trial) We propose specific scientific milestones that both assess progress and address barriers that are encountered in the translation of waveform-derived predictive analytics technologies to the bedside. We characterize the roadmap into (A) the multidisciplinary data science steps required to transform monitoring into prediction, followed by (B) clincial decision support software (CDSS) tool development, subsequent (C) clinical research steps and (D) anticipated challenges encountered along this path. Data science Individual patient data collection The first step in creating a waveform-based, variability-derived, artificial intelligence-powered clinical decision support tool is collecting monitored patient data. While there are no standardized methods to extract waveform data from multivendor bedside monitors or ambulatory physiological devices, data is accessible through a variety of direct (e.g. direct from monitor) and networked (e.g. from middleware software tool) methods. Waveform capture solutions depend on the brand and model of the monitors and have varied technical requirements and costs. Setting up waveform data acquisition at clinical institutions is usually not a trivial undertaking, requiring clinical and technical stakeholders [42–45]. Research studies involving multiple sites face the additional challenge of harmonizing the waveform data collection process emanating from different systems, and amalgamation into a single database. The choice of data repository, database and file formats, and need to develop customized software to easily review large amounts of clinical data and physiological recordings should be carefully considered for each study. Waveform processing Physiological waveforms acquired from patients must be then processed by a series of automated software algorithms. The first step in waveform processing is to translate the waveform into a time series of physiological events, such as individual heartbeats (e.g. R wave detection on ECG), onset of exhalation from a capnogram (expired breath CO2 waveform), or systolic/diastolic/mean pressures from an arterial pressure waveform. Events can also include other fiducial points extracted from physiological waveforms that are relevant to the clinical problem. Subsequently, time series of inter-beat or inter-breth intervals may be measured. A thorough automated assessment of the quality of the underlying physiological waveform signal and derived physiological events time series is needed at this stage to identify and deal with movement artefacts, noise, disconnections and signal saturations. Signal quality indices help determine the validity of the data on a beat-by-beat or breath-by-breath basis, using continuity and morphology analyses. Additionally, the times series of physiological events should be filtered to exclude non-sinus beats, abnormally shaped breaths and non-physiologically plausible data that would otherwise render subsequent variability results erroneous. Variability analysis Once waveform processing is complete, variability assessment can be performed on a cleaned time series of events. This is commonly performed through a moving window analysis, whereby a window of fixed duration or number of samples/events is shifted across the entire duration of monitoring. A comprehensive set of variability metrics are calculated within each window. An array of linear and non-linear variability analysis techniques characterize the degree and complexity of variation within physiological signals. Variability measures can include metrics that characterize (a) statistical properties (e.g. standard deviation, root mean square of successive differences between normal heartbeats (RMSSD)), (b) the informational complexity (e.g. entropy measures), (c) the pattern variations across time scales (e.g. fractal measures, power law exponents) or (d) the energy contained in the signal (e.g. spectral measures). These techniques are based upon statistics, non-linear dynamics, chaos theory and mathematical physics [27, 46]. Linear techniques include (a) statistical measures describing the properties of the time series viewed as a stochastic process, such as standard deviation or RMSSD, (b) energetic measures characterizing the energy contained in various portions of the frequency, or (c) high frequency (HF), low frequency (LF) or time–frequency spectra estimated via mathematical transforms such as the fast Fourier transform, Lomb-Scargle periodogram or Wavelet transform. Non-linear techniques include (a) geometric measures describing properties related to the shape of a dataset in a certain space, such as Poincaré plots, recurrence plot or grid counting, (b) information theory-based measures that describe the degree of irregularity, disorder or complexity of a biological signal through entropy measures and/or similarity indexes, or (c) scale-invariant measures that estimate possible fractal (i.e. multiscale self-similarity) properties or attributes that do not change over time scales [26]. Each variability analysis technique provides a unique and complementary perspective on the data. A combination of techniques should be used for complete evaluation as no single metric offers a definitive characterization of biologic signals [26, 46]. High-quality variability estimates should be used in subsequent processing steps with non-stationary trends being flagged or removed. The goal of this step is to create a comprehensive and time-tracked multivariate analysis of variability. Statistical analysis Prior to predictive modelling, standard descriptive statistical analysis can be used to assess the presence and strength of associations between available variability metrics and an outcome(s) of interest. It can also evaluate whether an a priori model explains the data within specified confidence bounds. Outcomes are categorized based on clinical criteria or the presence/absence of deterioration. If statistical relationships between a set of variables and the outcome of interest are uncovered, and large population studies are feasible, then an attempt to determine if variability also offers reliable reproducible prediction of patient outcomes is then warranted. Predictive modelling Predictive modelling is distinct and complementary to statistical analysis offering actionable rather than explanatory information [43–45, 47]. In developing a predictive model, a set of relevant features must first be determined. In this context, features refer to input parameters that are fed to a supervised machine learning software to train the model to learn patterns and relationships within the data which enable accurate predictions. Relevant input features are determined through a priori expert knowledge or a feature selection process. The discriminative power of each feature is assessed in a population of patients based on one or several predetermined performance metrics. Input features may include variability metrics in addition to other clinical parameters (e.g. vital signs, laboratory data). In addition, separate predictive models may be derived for a subset of features. The models can be combined later through weighted averaging of the scores, using a Bayesian framework or other ensembling methods, especially if features are recorded at different time intervals. There is a wide choice of machine learning techniques to choose from for classification and prediction; commonly used techniques include logistic regression, k-nearest neighbour analysis, decision trees, random forests, support vector machines, neural networks, and variations of aforementioned techniques. Reproducibility of prediction is assessed by segmenting the data set, thus the need for larger data sets than statistical analysis. Typically, a cross-validation scheme can be used, whereby the dataset is separated into training and validation sets for model and feature selection, and a third “unseen data” testing set for obtaining an unbiased estimate of the performance. Calibration of the model can be addressed during the model selection of through a post hoc process via a hold-out dataset. External validation of the predictive model on a subsequently acquired population or dataset can assesses generalizability. The choice of performance metrics used to derive the optimal model depends on the characteristics of the dataset and specifics of the clinical decision. The selection and performance assessment of the candidate predictive models should be unbiased and prioritized for reproducibility. Commonly used metrics include sensitivity/specificity, accuracy, area under the receiver operating characteristic curve (AUROC), and F-measures. Specificity is often prioritized over sensitivity, which leads to the creation of a model that captures more patients to ensure health outcomes are not missed. Fine-tuning of the predictive model is an ongoing process which involves optimizing decision thresholds and estimating net benefit for patients and health partners. More data leads to improved models and subgroup specifications, assuming the data training and validation gets better at reflecting underlying clinical conditions. In the process of model development, an important question is its value added, critical to further adoption and clinical value, may be measured through performance metrics, comparing to existing standards-of-care. If possible, the newly developed predictive model should be compared to as well as with existing methods. Even if no previous predictive model exists for the clinical problem under investigation, the added value can be determined during the predictive modelling or even model selection stage analysis by comparing the isolated or added performance of common clinical features used in practice, such as vital signs data or severity scores, along with the subset of features derived from the newly developed model. Clincial decision support software tool (CDSS) development For predictive models to impact care, they must be integrated into a CDSS tool suitable for clinical use. CDSS has been defined as “software designed to be a direct aid to clinical-decision making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician for a decision” [48–50]. Predictive CDSS offers individualized prediction in addition to relevant clinical information, intended to assist, not replace, clinical decision making. To maximize the adoption and practical use, clinical predictive models integrated into CDSS ttols should be as simple as possible without compromising predictive accuracy, with parameters that are physiologically and clinically relatable to the user, and results that can be practically actioned upon within a standard clinical workflow. Clinical decision support software (CDSS) prototype development Information gleaned from any predictive model needs to be delivered to clinicians in a strategic manner to improve care, depending on context and exisiting practices. A predictive model’s output should be presented to clinicians alongside all other relevant information typically used to aid the associated clinical decision. Means and process of presentation must be continuously re-evaluated and improved. Ideally, a novel CDSS tool does not disrupt or have a negative impact on established clinical workflows; instead, it ideally enhances existing practices with a focussed targeted outcome. Indeed, identification of all outcomes and stakeholders anticipated to be impacted is critical at an early stage. In the development stage, barriers and enablers relevant to each stakeholder group should be considered. In fact, actively involving and gathering feedback from healthcare staff, technical and research teams as well as patients and/or their family in the prototype development process can help ensure the CDSS is favourably received. This process involves iterative evolutions of a CDSS product enabling improvement and optimization. A patient or user-centered design approach may be employed such that feedback is implemented to create successively better iterations of the CDSS. In the clinical setting, research prototype CDSS products that directly influence care of patients are considered as medical devices, and interventional research requires investigational testing authorization (ITA). In doing so, the CDSS may help with standardization of patient care including workup, diagnosis, and treatment. In addition, research prototypes drive the software development process in a cost-efficient manner [51]. Commercialization and regulatory approval Software that directly and immediately impacts patient care requires commercial product development and regulatory approval (e.g., FDA, Health Canada, CE Mark, etc.), even if they assist and do not replace clinical decision making. Given the expense, this is performed only if research prototype product development and testing demonstrates successful milestones (see Fig. 1). If so, a commercialization program and regulatory approval are required to complete the creation of a finished commercial grade CDSS product capable of being used for widespread interventional implementation. Commercial product development requires industry partnerships, whom posess an externally validated and audited corporate software quality management system that manages and tracks all changes and testing of software tools, all required for ongoing regulatory approval. Subsequently, observational followed by interventional clinical research studies can evaluate technical and clinical feasibility, clinical perceptions of use, and impact on clinical health economic outcomes. Clinical research steps Observational and interventional mixed method studies During and following CDSS tool development, clinical evaluation must be undertaken to assess the robustness, safety, and clinical impressions of CDSS, further iteratively improving the tool (see Fig. 1). This evaluation includes observational followed by interventional mixed methods studies that require patient consent as well as ethic board approval. First, an observational study may be performed whereby the CDSS tool is implemented prospectively, yet with no impact on care, as predictive CDSS results are not shown to clinicians in real time, but in a delayed fashion (e.g. days later). As there is no impact on patient care, there is no need for regulatory approval. These observational studies nonetheless enable the evaluation of feasibility, utility, and clinical perceptions of the user interface as well as usefulness. Feasibility may be objectively evaluated by measuring time to complete clinical data entry, time, success and quality of monitored data collection, quality and rapidity of completing predictive model, and time to produce the CDSS tool. Utility and clinical perceptions may be assessed with an evaluation of how the product integrates into existing practice and pathways of care, assessed by questionnaires and/or interviews. Questions can gauge stakeholder’s experience of using the tool, their impressions surrounding its contribution to care, as well as gather relevant feedback for changes to the user interface. Questions can ask whether there is perception of added value with the use of a CDSS tool. This entire process helps refine the CDSS interface, including data input and report output [52, 53]. Following observational studies, an interventional implementation study similarly may evaluate feasibility, utility, and clinical perceptions, yet with clinicins experiencing the tool as they care for consenting patients. In addition to consent and ethics approval, interventional studies require investigations testing authority (ITA) from regulatory bodies if the CDSS tool directly influences care in the immediate future. Ideally, the iterative refinement of the CDSS tool is minimal at this stage; however ongoing improvement should be a product development objective. These studies lay the foundation for subsequent rigourous evaluation on clinical and health economic outcomes, namely randomized interventional studies. Randomized controlled trial A randomized controlled trial (RCT) is the gold standard for evaluating the efficacy of a therapy, intervention, or product. RCTs are intended to evaluate the impact on clinical and health economic outcomes based on well-defined criteria relevant to patients and clinicians. To conduct a RCT, treatment choices are assigned randomly, independent of the patient and physician decision process. This randomization allows for the different groups of patients to be comparable with respect to prognostic and other unknown factors that may affect the outcome. For CDSS, while early adopters may wish to trial CDSS technology as early adopters, RCTs are required for widespread adoption. An RCT is again a multistep process. First, a feasibility pilot RCT assesses the feasibility and practicality of consent, randomization, protocol adherence, and complete clinical and outcome data collection. Due to the need to ensure adequate power and generalizability, multicenter RCTs are generally required. Consent and enrolment rates, proportion of cross-overs and contamination, incidence of outcomes, and more are all required to inform planning and design of a definitive RCT. In order to obtain funding for a definitive RCT, one must first demonstrate adequate capacity to consent and enroll, deliver the intervention, and collect all data, ideally supported by favourable healthcare professionals’ opinions and attitudes regarding their experience of the intervention and study. For a predictive CDSS, a definitive RCT compares doctors’ decision-making alone versus doctor and CDSS-assisted decision-making. The structural equivalence between the patient groups makes the definitive RCT the gold standard for appraisal of therapeutic efficacy and a required step for broad adoption of new interventions. The decision to pursue an RCT is rooted in the hypothesis that an intervention will improve care, and with the belief that rigorous evaluation is required to evaluate that hypothesis. When there is uncertainty surrounding the impact of the intervention, particularly in complex healthcare settings, and a need to establish relationships between the CDSS and clinical outcomes, an RCT is warranted. Less commonly, if ample longitudinal evidence suggests a high therapeutic benefit of the CDSS and clinical experience is clearly positive, large RCT studies may not be needed. For example, continuous pulse oximetry in acute care has never been evaluated in an RCT, as it is so clearly superior to intermittent measurement. However, CDSS tools are not as obviously beneficial, and RCTs are recommended and to be expected. Challenges Despite decades of research, virtually no applications of variability monitoring exist in adult critical care medicine. There are several challenges unique to monitored data predictive CDSS that act as barriers to development and evaluation of CDSS tools. These include technological, analytical, psychological, regulatory and commercial barriers within this pioneering domain. Technological There remains the challenge to capture waveform data from monitors feasibly and reliably, either directly or through middleware. An increasing number of monitor platform software solutions enable a network access to monitors, such that CDSS may interact with a software Application Programming Interface (API); however, this often requires a significant integration effort and simpler means of collecting monitoring data such as connecting directly to individual bedside monitors may be preferable for a first iteration of a product. Analytical Once waveform data are reliably collected, if variability analysis is sought, parameters related to the analysis must be held constant across patients and studies. There are many variability metrics that characterize complex healthy biologic variation, with no clear optimal subset across clinical problems [27, 46, 54] and a broad characterization of the degree and complexity of variability is helpful. Predictive modelling is again replete with multiple techniques, where optimality is determined by the data and problem considered. Nonetheless, standardization of the analysis is required to ensure reproducibility. Psychological There remains a high rate of non-compliance with recommended best practices for many medical conditions [55]. Clinicians may disregard CDSS in medical practice, even in cases where the tool is well integrated, and the care is known to be sub-optimal [56–58]. This can result in cognitive errors including faulty heuristics [59] and a failure in metacognition/willingness to assess one’s thought process and assumptions [60]. The intuitive approach (i.e. “fast” thinking) is the most influential and tends to guide the analytical one (i.e. “slow” thinking), even though it is prone to systematic errors based on biases and heuristics; research has shown that experts routinely use intuition but revise their strategy when it does not work [61]. Most cognitive errors in diagnosis occur in cases where the problem seems routine and the clinician is certain [60]. Presenting probabilistic recommendations may help clinicians appreciate there is greater uncertainty in outcomes. Overall, CDSS has the potential to promote greater deliberative, slow, analytical thinking, circumvent clinical errors and improve patient outcomes by presenting the clinician, in a timely fashion, with additional information, such as predictive analytics, checklists, and other relevant data for diagnostic or prognostic consideration. Regulatory Regulatory approval is granted to a product that meets a minimum set of regulatory, technical and safety requirements, and is required for software being utilized as a medical device. Regulatory approval is required before a CDSS tool can be licensed for use in clinical settings in a particular country. The requirements for approval will vary among countries and may include the authorization of government and health authorities prior to manufacturing, use, storage, import, export, transport, or sale of the product. This regulatory approval process can be expedited through partnering with industry, specifically predictive software companies that have extensive experience in the regulatory approval process of predictive analytics in Europe, Canada and the US. Nevertheless, regulatory approval can be a lengthy and costly process that may pose to be a significant barrier for CDSS tools. Commercialization Industry partners are critical for the commercialization of waveform-based predictive analytics for clinical decision support. An additional step in commercialization is securing a platform to host the software analytics and a means to deliver the information to the product user is required. A critical step in the commercial deployment of a CDSS product is human factor testing where the performance of the product/device is evaluated in the clinical environment in which it would be deployed. Determination and evaluation of the product’s usability, safety, and effectiveness, along with ethics, privacy, and policy implications of using patient data for commercial purposes all must be considered [62]. While initially a barrier, if successful, commercialization may help expedite clinical adoption. Conclusions Complementary to the traditional monitoring of the present or trends of vital signs, variability-derived predictive models derived from intervals-in-time allow us to track the emergent properties of a complex system at a patient level. While measuring the absolute value of a clinical parameter, such as heart rate or respiratory rate, provides significant clinical information, the variation in these clinical parameters over time can prove to be additionally clinically valuable. Predictive modelling transforms variability into clinically relevant actionable predictive models, which may be then embedded into CDSS tools. It is possible that the use of waveform-based, variability-derived, machine learning-powered clinical decision support tools may help reduce diagnostic and prognostic uncertainty and ultimately reduce morbidity, mortality, and associated healthcare costs [63–65]. The presented roadmap for development and evaluation of waveform-based variability-derived predictive CDSS tools is hoped to assist the process of transforming monitoring and improving care. Acknowledgements The authors would like to gratefully acknowledge the multidisciplinary research personnel that make this research program possible, who assist with privacy approvals, research ethics approvals, investigational testing authorization from regulatory bodies, consenting and enrolling patients, collecting and entering clinical data, and so much more. Abbreviations Artificial intelligence Clinical decision support software Electrocardiogram Heart rate variability High frequency Low frequency Randomized controlled trial Respiratory rate variability Root mean square of successive differences between normal heartbeats Very low frequency Author contributions A.S. conceived the design and concept for this paper. A.S., K.N., R.R., C.H., N.S., N.H., J.B., D.J., T.R., D.B., S.F., J.P., S.D., K.B. contributed to the preparation, writing and review of this manuscript. All authors read and approved the final manuscript Funding The authors gratefully acknowledge the Canadian Institutes of Health Research (CIHR), Physicians Services Incorporated (PSI), The Ottawa Hospital Academic Medical Organization (TOHAMO), the University of Ottawa Faculty of Medicine, and the Ottawa Hospital Departments of Surgery and Critical Care, for funding support of this research program. Data availability No datasets were generated or analysed during the current study. Declarations Conflict interest Dr. Andrew Seely is the founder and CEO of Therapeutic Monitoring Systems (TMS) which holds licensing rights to several patents related to variability-derived clinical decision support, where both Andrew Seely and Christophe Herry are patent authors related to this work. Dr. A. Seely discloses the conflict of interest (CoI) in all grants, publications, presentations, and other public communication, and submits a CoI management agreement to Ottawa Hospital Research Institute annually. Footnotes Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement No datasets were generated or analysed during the current study.