Integrating decision trees with ANOVA for enhanced interaction visualisations Authors: Perseverence Savieri, Lara Stas, Kurt Barbé, Iftikhar Khan, Iftikhar Khan, Iftikhar Khan, Iftikhar Khan, Iftikhar Khan Published: 2026 ABSTRACT Interaction effects in ANOVA provide crucial insights into how the effect of one independent variable depends on the level of another. However, interpreting these interactions, especially three-way interactions, remains a challenge. Firstly, visualisation techniques typically rely on multiple separate two-dimensional plots conditioned on the levels of a third factor; as the number of levels of the third factor increases, so does the number of plots, complicating interpretation. Secondly, routinely applying uncorrected post-hoc pairwise comparisons oversimplifies interaction effects and, by ignoring the multiplicity of tests, inflates Type I error rates, potentially leading to misleading conclusions. Advanced methods, such as partial dependence plots and interaction decomposition models, have partially addressed these limitations; however, they typically require substantial statistical or computational expertise, which limits their accessibility. To address these challenges, this manuscript introduces EXCITE (Enhanced eXploration of Complex Interactions in ANOVA using Trees), an interactive web application developed using the Shiny package within the R statistical environment. EXCITE integrates traditional two- and three-way ANOVA models with decision tree-based visualisations. Decision trees detect interaction patterns via recursive data partitioning and present conditional subgroup relationships through an interpretable tree structure. The advantages of EXCITE include presenting two- and three-way interactions intuitively, enhancing accessibility, and facilitating accurate interpretation of complex statistical interactions. This integration provides researchers and educators across various scientific disciplines with a user-friendly tool for interpreting and visualising ANOVA interactions. We illustrate the capabilities of EXCITE using both simulated examples and a real dataset to demonstrate its practical applicability. The web application is freely available online, requiring no installation or coding expertise, at https://zq9mvv-vub0square.shinyapps.io/EXCITE-research-tool/. 1. Introduction 1.1. Background and problem statement Analysis of variance (ANOVA) is widely used to assess mean differences across groups defined by categorical factors [1–3]. Originally developed by Sir Ronald Fisher, ANOVA’s simplicity and interpretability have led to its widespread adoption across disciplines such as psychology, neuroscience, and medicine [4]. In two- and three-way ANOVA, researchers may test not only main effects but also interactions; whether the effect of one factor depends on the level of another [5,6]. Interactions, typically visualised as non-parallel means patterns across factor combinations [7,8], are critical for understanding heterogeneity among multiple factors [1,6]. Interpreting higher-order interactions, however, remains challenging due to the limitations of traditional visualisation methods. These effects are often shown using bivariate plots, which condition on the levels of a third factor, resulting in multiple two-dimensional plots across separate panels [5,7,9]. As the number of factors or the number of levels within the conditioned factor increases, this “split-plot” approach produces an expanding series of separate panels, each displaying only a fragment of the full interaction structure [5, 10]. For a three-way interaction, this already requires examining several conditioned plots; for higher-order interactions, the number of required panels increases rapidly, making it harder to see how the factors jointly influence the outcome. As the visualisation becomes more fragmented, important patterns may be less apparent. Researchers often turn to post hoc pairwise comparisons because omnibus ANOVA tests merely indicate whether an interaction exists, without specifying where the differences lie. These follow-up tests aim to identify which factor-level combinations differ from one another. However, as highlighted in recent reviews [4], the routine use of uncorrected post hoc comparisons can oversimplify inherently multivariate interaction patterns by reducing them to isolated mean contrasts [11,12]. This practice increases the risk of Type I error due to multiple testing and, crucially, does not resolve the core interpretability problem: it still does not provide an integrated view of how factors jointly shape the response [6,8]. Furthermore, standard ANOVA coding constraints (e.g., sum-to-zero) affect the interpretability of higher-order terms, as parameter estimates depend on the chosen contrast scheme and the parameterisation of the model [7]. 1.2. Existing methodology Several approaches have sought to improve the visualisation and interpretation of interaction effects [4,13]. Common methods, such as estimated marginal means plots and simple slope analyses, provide clearer insight than post hoc comparisons [5]. More advanced, machine-learning-inspired, including partial dependence plots and interaction decomposition models, capture complex relationships through graphical frameworks like ggplot2 and lattice [5,13]. However, these approaches often require substantial statistical or computational expertise, limiting accessibility for non-specialists. Interactive tools have recently emerged to address limitations in traditional ANOVA approaches. For instance, Jiang et al. [14] developed moreThanANOVA, to automate distribution and variance checks with balanced post hoc testing, while Sisso et al. [15] introduced an educational web application for teaching one-way ANOVA. Yet, none of these tools explicitly target the visualisation of complex two- or three-way interactions. They focus on assumption testing or instructional contexts rather than integrated visual exploration, leaving the challenge of multi-factor interaction interpretation largely unresolved. 1.3. Objectives Despite these advances in visualisation methods, significant gaps persist. Existing tools typically focus exclusively on ANOVA or machine learning visualisations [13–16], requiring users to interpret multiple-panel plots or demanding advanced statistical expertise, limiting their practical utility for non-specialists. Addressing these limitations requires a single-view visualisation that simplifies the interpretation of complex three-way interactions. To meet this need, we introduce EXCITE (Enhanced eXploration of Complex Interactions in ANOVA using Trees), an interactive Shiny application that integrates traditional ANOVA with decision-tree visualisation [17]. EXCITE demonstrates how decision trees complement classical ANOVA, improving interpretability and accessibility when exploring multifactor interactions. The novelty of the proposed approach lies precisely in integrating traditional two- and three-way ANOVA with the tree-based visual exploration. Through this innovative merging of ANOVA with decision tree visualisations, EXCITE aims to offer researchers an accessible, interactive tool that not only simplifies but also enhances the interpretability of intricate statistical interactions, thus providing clear value and broad applicability across diverse research domains. Importantly, EXCITE is positioned as a complement to factorial ANOVA, optimised for categorical predictors and designed to display two- and three-way interactions in a single tree-based visualisation of conditional relationships, rather than as a replacement for tools focused on assumption checking (e.g., moreThanANOVA) or model-agnostic machine-learning visualisations (e.g., partial dependence or interaction decomposition); consequently, the evaluation in this manuscript emphasises interpretability and workflow coherence with ANOVA rather than predictive benchmarking. The Methods section outlines the theoretical framework and app design; readers interested primarily in practical use may refer directly to Section 2.4 (Workflow and User Interface Overview), where the app’s operation is demonstrated step-by-step. The Results section presents applied examples, followed by a Discussion on implications and future extensions, and a Conclusion summarising key contributions. 2. Methods 2.1. Overview of two- and three-way ANOVA ANOVA can be conducted using either a single factor (one-way ANOVA) or multiple factors simultaneously (factorial ANOVA). When two factors are considered simultaneously, this is known as two-way ANOVA. Two-way ANOVA can assess the main effects of each factor and the potential interaction effects between these factors [3]. In a two-way factorial ANOVA design with interactions, the general linear model is expressed as: Yijkis the response for the kth observation within the combination of the ith level of factor A and the jth level of factor B. μ denotes the overall mean response across all factor combinations. Ai captures the main effect of the ith level of factor A, Bj captures the main effect of the jth level of factor B, (AB)ij represents the interaction effect between factors A and B, describing how the effect of factor A depends on the level of factor B, and vice versa. ϵijk represents the error term, assumed to be independently and identically normally distributed with mean zero and constant variance σ2 [1,3]. The model expands similarly for a three-way factorial ANOVA, capturing additional interactions: where each term incorporates an additional factor (factor C) and relevant interactions [2]. Specifically, interpreting a three-way interaction (ABC)ijk involves examining how the interaction between two factors (e.g., A and B) may vary depending on the level of a third factor (C). For instance, the interaction effect between factors A and B could change across different levels of factor C, meaning the relationship between A and B is not consistent but conditional on C. Similarly, the interaction between factors A and C may differ depending on the levels of factor B, and the interaction between factors B and C may shift according to the levels of factor A [3]. Thus, interpreting a three-way interaction involves understanding and visualising how these conditional relationships among factors differ across their combined levels. In practice, researchers frequently omit higher-order interactions (especially beyond three-way) from models to simplify interpretation and maintain sufficient statistical power, particularly when sample sizes are limited [18–20]. However, careful model specification should always be guided by theoretical considerations and study design rather than convenience alone. This is because in some contexts, such interactions may carry meaningful insights, for example, when a treatment’s effectiveness differs across multiple demographic and contextual variables simultaneously (e.g., Therapy*Sex*Age*Setting). To balance interpretability and practical significance, this manuscript specifically focuses on two- and three-way ANOVA to clearly illustrate the integration and interpretation advantages of combining ANOVA with decision-tree visualisations. 2.2. Decision tree modelling Decision trees are supervised machine learning models used for classification and regression due to their simplicity and interpretability [21,22]. They partition data hierarchically by applying recursive splits on predictor variables to form a tree-like structure composed of nodes and branches (see Fig 1). Each internal node represents a decision rule, splitting data into subsets based on a specific predictor, and each leaf node provides the final classification or prediction outcome [23,24]. Due to the transparent “if-then” logic, decision trees illustrate how different variables interact and influence outcomes. This visualisation makes complex dependencies immediately interpretable, especially for non-statistical audiences [22,25]. Example of a decision tree displaying hierarchical splits. Each path illustrates a unique interaction among predictor levels. In the context of EXCITE, decision trees complement ANOVA by visually representing interactions detected through omnibus tests [26]. Whereas ANOVA identifies interactions statistically, decision trees reveal how factor-level combinations shape the response through recursive binary splits. Importantly, this hierarchy reflects the structure of the visual representation rather than the statistical structure of the ANOVA interaction itself. EXCITE implements the Classification and Regression Trees (CART) algorithm [27] via the rpart R package [28], selecting splits at each node by maximising the reduction in within-node variance. The algorithm evaluates all possible predictors and split points, choosing the partition that minimises the sum of squared errors within the resulting subsets [26]. This recursive partitioning continues until predefined stopping criteria (e.g., minimum node size) are met, producing a tree that maps predictors to outcomes through a series of interpretable decision rules. Although the CART algorithm evaluates many candidate partitions during tree construction, the final tree displays only the subset of splits that produce the largest reductions in within-node variance. Consequently, the resulting diagram summarises the most influential conditional relationships rather than enumerating all possible factor-level comparisons. Balancing model flexibility with overfitting is a key aspect of tree modelling. The complexity parameter (CP) in rpart addresses this by penalising additional splits that do not sufficiently improve model fit [26,27]. Specifically, CP represents the minimum improvement in variance reduction required to justify a new split. Lower CP values permit deeper trees, while higher values prune the tree to a simpler structure. In EXCITE, the tree’s complexity parameter is directly linked to the statistical evidence from ANOVA. Specifically, the application automatically sets a CP threshold directly linked to the significance level of the ANOVA interaction terms. For example, when the ANOVA interaction is statistically significant at p < 0.01, EXCITE uses a lower CP threshold, permitting additional splits to reflect this interaction. In contrast, when interactions are nonsignificant, EXCITE applies a higher CP threshold to enforce stricter pruning. This approach ensures that the final tree prioritises splits aligned with statistically meaningful effects, reducing overfitting while preserving interpretability [13]. Additionally, EXCITE computes variable importance as the total variance reduction attributable to each predictor across all splits [27]. This measure highlights predictors that drive subgroup differentiation, even those not retained in the pruned tree, bridging machine-learning flexibility with traditional statistical rigour. For example, in a three-way ANOVA studying the effects of teaching method, school type, and gender on student performance, the variable ‘school type’ may be found to have high variable importance in the decision tree, even if it does not appear as a splitting variable in the final pruned tree. This could happen if ‘school type’ is strongly associated with ‘teaching method’, which is selected for splitting. As a result, ‘school type’ influences the structure of the interaction indirectly, highlighting its role despite its absence in the final displayed tree. By coupling this measure with interactive tree visualisations, EXCITE enables users to explore how key variables structure conditional subgroup patterns within the tree representation. 2.3. Mathematical specification of regression trees in EXCITE In this section, we formalise the tree-building procedure implemented in the rpart package [28] as it is used in the EXCITE web application. Throughout, we follow the terminology and notation of Breiman et al [27]. Let denote n independent observations on a continuous response y and categorical predictors xij, each with Lj unordered levels. Recursive binary partitioning. A regression tree partitions the predictor space into M disjoint terminal regions R1, ⋯, RM. Beginning with the root node R1= X, the algorithm considers all possible binary partitions of the levels for a given categorical predictor. For a predictor with Lj levels, there are 2Lj−1−1 possible ways to form two groups. The algorithm applies binary splits of the form: where xj is a categorical predictor and S⊂{1, ⋯,Lj} is a non-empty proper subset of its levels. For each candidate split (j, S), the CART cost function for continuous y is the total within–node sum of squares (i.e., the impurity of the region R) Choosing the split that maximises the reduction in impurity is equivalent to minimising the residual sum of squares in the two child nodes. This improvement can also be expressed as: where nL and nR are the number of observations and y―L and y―R are the means in the left and right child nodes, respectively. Because all predictors are factors, Δ(j, S) is evaluated over every binary partition of the Lj levels. For a predictor with Lj levels, an exhaustive enumeration requires 2Lj−1−1 subsets S. rpart first orders the levels by their means and tests only the Lj−1 splits defined by that ordering, reducing the search to O(LjlogLj), where O(·) is the big-O notation that gives an asymptotic upper bound on computational cost (up to constant factors). Cost–complexity pruning. The fully grown tree T0 over–fits, pruning produces a nested sequence {Tα} that balances bias and variance. For any subtree T, the cost–complexity functional [27] is: where nm=|Rm| is the number of observations in leaf m, Q(Rm) is the within-node mean-squared-error, so that nmQ(Rm)=SSE(Rm), and |T| is the number of terminal nodes. Increasing the complexity parameter α≥0 repeatedly removes the weakest link, the split with the smallest reduction in Δ/nm, and yields a nested sequence of sub-trees Tα1⊃Tα2⊃⋯. The subtree selected for prediction balances goodness-of-fit (R(T)) against model parsimony (|T|). Therefore, cost-complexity pruning does not aim to discard scientifically meaningful effects but to avoid overfitting that may arise when recursive partitioning continues to split small subsets of data. The pruning step, therefore, prioritises stability and interpretability in the visualisation while retaining the most informative interaction structure. Tree selection in EXCITE. The selection of the final regression tree is guided by both statistical evidence and interpretability. Let pint denote the omnibus p-value of the highest-order interaction term from the preceding ANOVA model, representing the strength of evidence for the most complex interaction specified in the factorial ANOVA model. Rather than allowing users to adjust the tree depth arbitrarily, EXCITE links this statistical evidence directly to the tree’s complexity parameter (CP), the tuning constant that controls pruning in the CART framework. Conceptually, EXCITE links the complexity of the decision tree directly to the statistical evidence provided by the factorial ANOVA. Stronger evidence for the highest-order interaction corresponds to a smaller complexity parameter (CP), allowing the tree to retain finer partitions of the data. Conversely, weaker evidence results in a larger CP value and consequently greater pruning, reducing the likelihood of displaying spurious interaction patterns. This adaptive mapping allows the visual complexity of the decision tree to reflect the strength of evidence for the highest-order interaction while promoting parsimonious and interpretable visualisations. Practically, EXCITE provides two tree display modes that differ in the degree of pruning applied after tree construction: (i) Maximum tree complexity, which displays the fully grown tree (CP = 0), and (ii) ANOVA-guided tree (recommended), which uses the omnibus p-value of the highest-order interaction (pint) as the complexity parameter (CP). A deep tree is first grown with CP = 0 using ten-fold cross-validation. Under the Maximum tree complexity option, this fully grown tree is displayed without additional pruning. Under the ANOVA-guided tree option, the tree is pruned using CP = pint , thereby aligning the degree of pruning with the statistical evidence for the highest-order interaction. If the highest-order interaction is not statistically significant (pint ≥ 0.05), EXCITE suppresses the decision tree and informs the user that the ANOVA-guided tree is not displayed because there is insufficient evidence to support visualisation of the highest-order interaction. These display modes do not alter the fitted ANOVA model or the interaction terms under consideration; rather, they control the visual complexity of the decision tree through the degree of pruning applied after tree construction. The relationship between the highest-order interaction p-value and the complexity parameter should therefore be interpreted as a heuristic mechanism for aligning visual model complexity with statistical evidence, rather than as a strict inferential rule. This evidence-based approach formalises the principle that tree complexity should not exceed statistical support, translating inferential strength into visual model depth and reducing arbitrary parameter tuning. It bridges the inferential rigour of ANOVA with the exploratory flexibility of decision trees, enabling consistent and interpretable visualisations of interaction effects [26,27]. Prediction. Each terminal node carries the fitted value y^m=y―Rm, so that a piecewise-constant regression surface whose paths translate directly into user-readable “if-then’‘ rules. That is, for a new observation x, we follow its decision path from the root to the appropriate terminal node and return the corresponding mean y^m as the prediction. Variable importance. For a predictor xj, its importance is the sum of the improvements over all nodes where xj is the splitting variable, weighted by the proportion of observations reaching each node. The calculation is as follows: where the sum is over all internal nodes t, where xj is the primary splitter, p(t) is the proportion of observations reaching node t, and Δ(s*,t) is the improvement from the split. This total contribution also includes the weighted improvement from surrogate splits that mimic the primary split. When predictors are strongly correlated, variable importance scores should be interpreted cautiously. Recursive partitioning algorithms may attribute variance to one predictor even when multiple correlated predictors contribute similarly to the observed pattern. Consequently, the variable importance measures in EXCITE should be interpreted as descriptive indicators of variance partitioning within the tree, rather than as providing a unique ranking of predictor importance. 2.4. Workflow and user interface overview The EXCITE app visualises complex ANOVA interactions in a single plot, using insights from decision tree methodology. It is hosted freely on shinyapps.io at https://zq9mvv-vub0square.shinyapps.io/EXCITE-research-tool/. The reproducible source code containing a complete list of utilised R packages can be found on GitHub at https://github.com/vub-square/EXCITE-Shiny-app. The user interface comprises eight tabs: Home, Dataset, Two-way ANOVA, Three-way ANOVA, Decision Tree, Reports, Manual, and Contact Us (see Fig 2). The tabs consist of an input (sidebar) panel for the user inputs and an output (main) panel, where the outputs are displayed. The workflow of the app is structured into five interactive stages. Dataset tab. The EXCITE app’s user interface displays a data preview. Data Import: On the Dataset tab (Fig 2), users begin by either uploading their datasets in various formats (CSV, Excel, SPSS, Stata, text) or selecting one of the example datasets. The application also allows users to simulate alternative versions of the built-in examples by manipulating two key design parameters: within-group variability and sample size, before any analysis is run. The latter can be done in the sidebar, under “Choose dataset to alter”, the user (i) chooses an example dataset (e.g., “Pain reduction (2-way)” or “Type 2 Diabetes (3-way)”), (ii) specifies a new standard deviation via a slider (thereby inflating or shrinking the residual variance), and (iii) sets the number of observations per factor combination with a numeric input (up to 200 per cell). Clicking “Run Simulation” generates a new dataset that honours the original factorial structure but reflects the requested variability and sample size. Adaptations in the dataset are immediately shown in the output of all downstream tabs, so users can explore how changes in confidence intervals’ width, sample size, or both influence ANOVA results and decision-tree visualisations. This feature turns the example datasets into interactive sandboxes for power and precision, rather than static demonstrations. A download button then allows the download of the simulated data in CSV, TSV, or Excel format for external use. Data Import: On the Dataset tab (Fig 2), users begin by either uploading their datasets in various formats (CSV, Excel, SPSS, Stata, text) or selecting one of the example datasets. The application also allows users to simulate alternative versions of the built-in examples by manipulating two key design parameters: within-group variability and sample size, before any analysis is run. The latter can be done in the sidebar, under “Choose dataset to alter”, the user (i) chooses an example dataset (e.g., “Pain reduction (2-way)” or “Type 2 Diabetes (3-way)”), (ii) specifies a new standard deviation via a slider (thereby inflating or shrinking the residual variance), and (iii) sets the number of observations per factor combination with a numeric input (up to 200 per cell). Clicking “Run Simulation” generates a new dataset that honours the original factorial structure but reflects the requested variability and sample size. Adaptations in the dataset are immediately shown in the output of all downstream tabs, so users can explore how changes in confidence intervals’ width, sample size, or both influence ANOVA results and decision-tree visualisations. This feature turns the example datasets into interactive sandboxes for power and precision, rather than static demonstrations. A download button then allows the download of the simulated data in CSV, TSV, or Excel format for external use. Data Exploration and Management: On the main panel of the Dataset tab, users can interactively preview the dataset in the “View Data” table, where they can sort, filter, and search rows using the table widgets as shown in Fig 2. The “Data Summary”, in the background generated with summarytools (v1.0.1), provides robust exploration tools that report summary statistics, detect variable types, and flag potential conversion issues. Under the “Data manipulation and preprocessing” heading on the sidebar panel, users can select variables to alter, for instance, change their data type (e.g., numeric to factor) or apply common transformations such as log, square root, standardisation, or centring. Clicking “Apply Changes” immediately writes the edited or newly transformed column back to the working dataset. This integrated workflow allows users to confirm that factor levels, numeric ranges, and missing-value patterns are sensible and, when needed, to fix them before proceeding to the ANOVA and Decision-Tree tabs. Data Exploration and Management: On the main panel of the Dataset tab, users can interactively preview the dataset in the “View Data” table, where they can sort, filter, and search rows using the table widgets as shown in Fig 2. The “Data Summary”, in the background generated with summarytools (v1.0.1), provides robust exploration tools that report summary statistics, detect variable types, and flag potential conversion issues. Under the “Data manipulation and preprocessing” heading on the sidebar panel, users can select variables to alter, for instance, change their data type (e.g., numeric to factor) or apply common transformations such as log, square root, standardisation, or centring. Clicking “Apply Changes” immediately writes the edited or newly transformed column back to the working dataset. This integrated workflow allows users to confirm that factor levels, numeric ranges, and missing-value patterns are sensible and, when needed, to fix them before proceeding to the ANOVA and Decision-Tree tabs. ANOVA Modelling: Under the Two-way ANOVA and Three-way ANOVA tabs, users can perform an ANOVA analysis by selecting one numeric outcome variable and corresponding categorical predictor variables. The application then presents detailed model summaries (Fig 3), visualisations (including boxplots and interaction plots), and the ANOVA model diagnostics (e.g., Shapiro-Wilk normality test). ANOVA Modelling: Under the Two-way ANOVA and Three-way ANOVA tabs, users can perform an ANOVA analysis by selecting one numeric outcome variable and corresponding categorical predictor variables. The application then presents detailed model summaries (Fig 3), visualisations (including boxplots and interaction plots), and the ANOVA model diagnostics (e.g., Shapiro-Wilk normality test). Decision Tree Visualisation: Building on the variables already specified for the ANOVA model, the main panel of the Decision Tree tab allows users to visualise the tree analysis. On the sidebar panel, users can select the tree display mode. The Maximum tree complexity option displays the fully grown tree, whereas the ANOVA-guided tree (recommended) prunes the tree using the omnibus p-value of the highest-order interaction as the complexity parameter (see Fig 4). The main panel displays two linked outputs: (i) an interactive “Tree Visualisation” and (ii) a bar plot under “Variable Importance”. The tree diagram shows, at each node, the mean outcome for that subgroup (top number) and its standard error (bottom number, preceded by “±”). To interpret a pathway in Fig 4, begin at the root node (top node), then move downward by selecting branches that match your factor levels. At each node, follow the left branch if the stated condition is met (“yes”) and the right branch if it is not (“no”). Continue this process until you reach a leaf node. The leaf node provides the mean outcome and its standard error (precision) for the specific combination of factor levels you’ve selected (e.g., “Placebo → Inactive → Obese BMI”). Smaller standard errors denote more precise estimates, while larger ones signal greater within-group variability. Every split visualises an ANOVA interaction, and the colour shading of the nodes reflects the predicted mean response for each terminal node. The green-blue gradient corresponds to the range of predicted values, with different colours indicating different levels of the outcome variable. This visual encoding allows for quick identification of subgroups with similar predicted outcomes, complementing the numerical predictions displayed at each node. The accompanying technical summary reports the CP table; EXCITE highlights the optimal CP that balances model fit and simplicity, so users can see why the displayed tree was pruned at a given depth. By embedding these explanations, accessible via checkbox “manual” buttons (e.g., ‘How to interpret the Decision Tree’), the interface gives step-by-step guidance on node values, standard errors, split logic, and pruning criteria, ensuring that researchers can interpret both the visual tree and the numerical summary with confidence. Decision Tree Visualisation: Building on the variables already specified for the ANOVA model, the main panel of the Decision Tree tab allows users to visualise the tree analysis. On the sidebar panel, users can select the tree display mode. The Maximum tree complexity option displays the fully grown tree, whereas the ANOVA-guided tree (recommended) prunes the tree using the omnibus p-value of the highest-order interaction as the complexity parameter (see Fig 4). The main panel displays two linked outputs: (i) an interactive “Tree Visualisation” and (ii) a bar plot under “Variable Importance”. The tree diagram shows, at each node, the mean outcome for that subgroup (top number) and its standard error (bottom number, preceded by “±”). To interpret a pathway in Fig 4, begin at the root node (top node), then move downward by selecting branches that match your factor levels. At each node, follow the left branch if the stated condition is met (“yes”) and the right branch if it is not (“no”). Continue this process until you reach a leaf node. The leaf node provides the mean outcome and its standard error (precision) for the specific combination of factor levels you’ve selected (e.g., “Placebo → Inactive → Obese BMI”). Smaller standard errors denote more precise estimates, while larger ones signal greater within-group variability. Every split visualises an ANOVA interaction, and the colour shading of the nodes reflects the predicted mean response for each terminal node. The green-blue gradient corresponds to the range of predicted values, with different colours indicating different levels of the outcome variable. This visual encoding allows for quick identification of subgroups with similar predicted outcomes, complementing the numerical predictions displayed at each node. The accompanying technical summary reports the CP table; EXCITE highlights the optimal CP that balances model fit and simplicity, so users can see why the displayed tree was pruned at a given depth. By embedding these explanations, accessible via checkbox “manual” buttons (e.g., ‘How to interpret the Decision Tree’), the interface gives step-by-step guidance on node values, standard errors, split logic, and pruning criteria, ensuring that researchers can interpret both the visual tree and the numerical summary with confidence. Reporting: Under the Reports tab, EXCITE allows users to export detailed analysis reports in HTML, PDF, or Word formats, capturing the entire analytic workflow, including data summaries, ANOVA results, and decision tree visualisations. Reporting: Under the Reports tab, EXCITE allows users to export detailed analysis reports in HTML, PDF, or Word formats, capturing the entire analytic workflow, including data summaries, ANOVA results, and decision tree visualisations. Two-way ANOVA tab. The main panel shows navigation to access ANOVA results, plots and diagnostics. Decision Tree tab. The sidebar shows options to fit the tree model, and the main panel where the decision tree plot and summary will be displayed. 3. Results 3.1. Example 1: Two-Way ANOVA This example examines the effects of Drug Type (placebo, low dose, high dose) and Gender (male, female) on pain reduction in 300 simulated observations. The two-way ANOVA results (Table 1 from the Two-way ANOVA tab in the app) show significant main effects of Drug Type (F(2, 294) = 818.71, p < .001) and Gender (F(1, 294) = 110.59, p < .001), and a significant Drug Type × Gender interaction (F(2, 294) = 8.48, p < .001). Effect sizes indicate a very large effect for Drug Type (η² = 0.85; 95% CI [0.83, 1.00]) and a moderate–large effect for Gender (η² = 0.27; [0.21, 1.00]); the interaction is small but significant (η² = 0.05; [0.02, 1.00]). 10.1371/journal.pone.0357663.t001Table 1Two-way ANOVA results for the effects of drug type and gender on pain reduction.Source Df SS MS F p Partial η² (95% CI)Drug Type22281.41140.7818.7<.0010.85 [0.83, 1.00]Gender1154.1154.1110.6<.0010.27 [0.21, 1.00]Drug Type × Gender223.611.88.5<.0010.05 [0.02, 1.00]Error (Residuals)294409.61.4Df = degrees of freedom; SS = sum of squares; MS = mean square; Partial η² = partial eta squared; CI = One-sided confidence intervals with upper bound fixed at 1. Two-way ANOVA results for the effects of drug type and gender on pain reduction. Df = degrees of freedom; SS = sum of squares; MS = mean square; Partial η² = partial eta squared; CI = One-sided confidence intervals with upper bound fixed at 1. Fig 5 (boxplots) shows that pain reduction increases with higher dosage levels and is consistently higher for females. Fig 6 (interaction plots) illustrates the Drug Type × Gender interaction more clearly: slopes differ between genders, with females showing greater pain reduction across all dosages. Boxplots from the Two-way ANOVA tab. (a) Pain reduction by drug type and (b) Pain reduction by gender. Interaction plots from the Two-way ANOVA tab. (a) Mean pain reduction by drug type and gender, and (b) Mean pain reduction by gender and drug type. The decision tree (Fig 7) highlights the conditional structure of this interaction. The root node sits at the top of the tree and contains every observation in the dataset; its mean of 9.8 is therefore the grand mean pain-reduction score across all treatment and gender combinations, and its accompanying standard error reflects the precision of this overall estimate. The first split is by Drug Type, separating high dose (node 3, mean = 13.2) from low/placebo (node 2, mean = 8.1). This is because a decision tree is restricted to binary splits; any factor with more than two levels is partitioned into two levels at a time; further downstream splits handle additional levels. Subsequent Gender splits show, for example, that within the high dose group, females (node 7, mean = 14.1) experience greater pain reduction than males (node 6, mean = 12.3). Similarly, in the placebo group, females (mean = 6.8) slightly outperform males (mean = 6.1), and in the low dose group, females (mean = 10.8) exceed males (mean = 8.9). Each node’s mean and standard error directly correspond to the subgroup averages shown earlier in boxplots and interaction plots, thus visually demonstrating how interactions influence subgroup differences within the tree representation. Two-way ANOVA decision tree visualisation of pain reduction. Illustrates recursive splitting based on drug type and gender. Compared with Table 1 and Figs 5,6 (interaction plots and boxplots), the tree offers a single tree-based view of subgroup means, simplifying interpretation by showing how the main and interaction effects are characterised by successive tree splits. 3.2. Example 2: Three-way ANOVA (Social Feedback × Inclusion Condition × Fear of Negative Evaluation) Whereas Example 1 illustrates how EXCITE visualises interactions between two categorical factors, this second example demonstrates the application of EXCITE to a three-way factorial design, where interpretation becomes substantially more challenging using conventional plots. Three-way ANOVA is commonly used when researchers seek to understand not only whether two factors interact, but whether that interaction itself depends on a third moderating variable. This setting provides a natural and practically relevant test case for evaluating whether a single, tree-based visualisation can clarify higher-order interaction structures. To this end, we analyse real data from an online social interaction experiment reported by Thériault et al. [29], which examined how social feedback (ambiguous vs. positive) and inclusion condition (Cyberball vs. Uberball) jointly influence participants’ fundamental needs, and whether this relationship depends on individuals’ fear of negative evaluation (FNE). Fear of negative evaluation was dichotomised into Low FNE and High FNE, yielding a balanced 2x2x2 factorial design. This dataset is well-suited for illustrating three-way interactions, as prior analyses showed that the effect of social inclusion and feedback differs systematically across levels of social anxiety, producing interaction patterns that are difficult to synthesise using standard multi-panel visualisations. Table 2 (from the Three-Way ANOVA tab) summarises the factorial structure underlying the patterns reported by Thériault et al. [29]. In the original study, the authors showed that individuals high in fear of negative evaluation are particularly sensitive to the combination of social feedback and inclusion context, with fundamental needs being lowest when positive feedback is followed by ordinary Cyberball inclusion. The present analysis reproduces this interaction structure in a standard three-way ANOVA framework to serve as input for visual exploration rather than as an independent test of new hypotheses. 10.1371/journal.pone.0357663.t002Table 2Three-way ANOVA results for the effects of feedback, inclusion condition, and fear of negative evaluation (FNE group) on fundamental needs.Source Df SS MS F p Partial η² (95% CI)Feedback10.930.931.920.1670.00 [0.00, 1.00]Inclusion110.2110.221<.0010.05 [0.02, 1.00]FNE118.1518.237.3<.0010.08 [0.04, 1.00]Feedback × Inclusion10.020.020.050.8290.00 [0.00, 1.00]Feedback × FNE10.080.080.170.6780.00 [0.00, 1.00]Inclusion × FNE10.70.71.430.2330.00 [0.00, 1.00]Feedback × Inclusion × FNE14.484.489.20.0030.02 [0.00, 1.00]Error (Residuals)430209.40.49Df = degrees of freedom; SS = sum of squares; MS = mean square; Partial η² = partial eta squared; CI = One-sided confidence intervals with upper bound fixed at 1 Three-way ANOVA results for the effects of feedback, inclusion condition, and fear of negative evaluation (FNE group) on fundamental needs. Df = degrees of freedom; SS = sum of squares; MS = mean square; Partial η² = partial eta squared; CI = One-sided confidence intervals with upper bound fixed at 1 Consistent with the original findings, significant main effects were observed for Inclusion (F(1, 430) = 20.96, p < .001) and FNE group (F(1, 430) = 37.27, p < .001), alongside a statistically significant three-way interaction between Feedback, Inclusion, and FNE group (F(1, 430) = 9.20, p = .0026). The magnitude of this interaction was modest (partial η² ≈ 0.02), but its interpretive relevance lies in how conditional effects differ across levels of social anxiety, a pattern that becomes clearer when examined graphically. However, the confidence intervals for the effect-size estimates are wide, indicating considerable uncertainty in the magnitude of these effects, which is typical for higher-order interactions and should be taken into account when interpreting these results. Figs 8 and 9 closely mirror the interaction patterns described in the original study, but reorganise them to highlight conditional differences across FNE groups and inclusion contexts. Interaction plots from the Three-way ANOVA tab. Plots showing feedback effects by inclusion condition and FNE group. Interaction plots from the Three-way ANOVA tab. Plots showing inclusion effects by the FNE group across ambiguous and positive feedback conditions. Fig 8 displays mean fundamental needs across feedback and inclusion conditions, separately for Low and High FNE. For Low FNE, needs scores were generally high across all conditions, with minor differences between feedback types. Uberball inclusion consistently produced higher means than Cyberball inclusion, but the magnitude of this difference was small. For High FNE, a clearer pattern emerged. Under Cyberball inclusion, participants who received positive feedback showed the lowest needs scores. In contrast, Uberball inclusion buffered this drop, yielding higher scores under both ambiguous and positive feedback. This demonstrates that individuals high in social anxiety are most sensitive to the combination of positive feedback followed by ordinary inclusion. Fig 9 reorganises the results using feedback as the facetting variable, with the FNE group on the x-axis and lines representing the two inclusion conditions. Under Ambiguous feedback, both the Low and High FNE groups report higher fundamental needs following Uberball inclusion than Cyberball inclusion, with only a modest decline from Low to High FNE. Under Positive feedback, this pattern becomes more pronounced: High-FNE participants in the Cyberball condition show the lowest needs scores in the entire figure, whereas High-FNE participants in the Uberball condition maintain comparatively high needs. This layout illustrates that although Uberball inclusion benefits participants across FNE levels, it is especially protective for individuals with High FNE, who show substantial reductions in fundamental needs only when exposed to Positive feedback followed by Cyberball inclusion. While Figs 8 and 9 illustrate the three-way structure through traditional interaction plots, each panel showing two factors at a time, this approach still requires readers to mentally integrate patterns across multiple subplots. The decision tree visualisation (Fig 10) condenses the same information into a single, unified tree structure. The root node splits first by FNE group, identifying social anxiety as the most influential predictor of fundamental needs. Participants with Low FNE form one branch with a comparatively high overall mean (node 3, M = 4.0 ± 0.7), whereas those with High FNE form a second branch with a lower mean (node 2, M = 3.6 ± 0.7). Three-way ANOVA decision tree visualisation of fundamental needs. Regression tree showing splits based on FNE group, inclusion condition, and feedback. Within the High FNE branch, the next split occurs on Inclusion condition, distinguishing the effects of Cyberball versus Uberball. High-FNE participants exposed to Cyberball (node 4) show reduced fundamental needs (M = 3.4 ± 0.8), particularly when the preceding feedback was positive (node 8, M = 3.3 ± 0.9), which represents the lowest outcome in the entire tree. In contrast, High-FNE individuals in the Uberball condition (node 5) report higher needs (M = 3.8 ± 0.5); among them, ambiguous feedback (node 10, M = 3.7 ± 0.5) maintains relatively stable scores, while positive feedback (node 11, M = 3.9 ± 0.5) yields the most favourable outcomes for this group. In the Low FNE branch, the pattern is reversed in magnitude but consistent in structure. The next split again occurs on the Inclusion condition, with the Cyberball subgroup (node 6) showing somewhat lower needs (M = 3.9 ± 0.8), followed by a final split on feedback. Ambiguous feedback (node 12, M = 3.7 ± 0.7) produces slightly lower means than positive feedback (node 13, M = 4.0 ± 0.7). In contrast, Low-FNE participants receiving Uberball inclusion (node 7) show the highest needs overall (M = 4.1 ± 0.6), regardless of whether they received ambiguous or positive feedback beforehand. Taken together, this tree-based representation (or single-plot strategy) mirrors and clarifies the patterns observed in Figs 8 and 9. By ordering predictors according to their relative contribution and displaying conditional effects along each decision path, the tree offers a single, integrated view of the three-way interaction, making complex subgroup differences more transparent and decision-oriented than a series of separate interaction plots. Fig 11 shows the variable importance fractions derived from the decision tree. FNE group contributed the largest proportion (51.9%), followed by Inclusion (33.5%), with Feedback contributing a smaller share (14.6%). This ranking matches the ANOVA: FNE and Inclusion exhibit the strongest main effects, while Feedback primarily plays a conditional role within the three-way interaction. The agreement between both approaches demonstrates the utility of EXCITE in revealing which variables matter most and how they interact. Variable importance in the Three-way ANOVA decision tree. 4. Discussion Decision trees are particularly well-suited for visualising ANOVA interactions because they provide an interpretable representation of conditional relationships between factors in a single diagram. Unlike traditional interaction plots that often require multiple separate visualisations, decision trees partition data recursively according to predictors that explain the largest reductions in within-node variance. The EXCITE Shiny application leverages this property by integrating decision-tree visualisations with factorial ANOVA analyses in an automated, user-guided workflow that takes researchers from data exploration to interpretation. This integration allows complex interactions, especially in three-way designs, to be displayed in one concise and interpretable graphic. Importantly, the hierarchy shown in the decision tree should not be interpreted as implying that ANOVA interaction effects themselves are hierarchical. In factorial ANOVA, interactions represent simultaneous conditional relationships among factors. The hierarchy instead reflects the visual structure imposed by recursive partitioning, which orders conditional mean differences according to their contribution to variance reduction. In this sense, the decision tree provides an interpretable representation of conditional subgroup patterns rather than redefining the statistical structure of the interaction. In this manuscript, we illustrate EXCITE using both simulated datasets and a real-world example drawn from a published social interaction study [29]. The empirical case demonstrates how the tool performs with naturally occurring variability and realistic interaction structures, while the simulated examples provide controlled settings in which the underlying interaction mechanisms are fully known. Together, these complementary data sources offer a balanced demonstration of EXCITE’s interpretive capabilities across both pedagogical and applied research contexts. Other user-friendly statistical tools exist, and they either focus on automating assumption checks and appropriate test selection with customizable plots [14] or assist non-statisticians with introductory analyses such as one-way ANOVA [30]. In contrast, EXCITE is designed specifically to visualise complex factorial interactions within a single diagram, using the recursive partitioning nature of decision trees to reveal how factors jointly influence outcomes. This unified view mitigates the limitations of multiple conditional plots by presenting conditional subgroup differences in a single visual structure. However, EXCITE should be viewed as a complementary interpretive tool rather than a replacement for formal statistical inference. In particular, EXCITE does not replace post-hoc testing when specific contrasts must be evaluated. Instead, the tree representation helps researchers identify influential factor combinations that may warrant further statistical examination. When formal pairwise comparisons are conducted based on these patterns, appropriate multiplicity adjustments (e.g., Tukey, Bonferroni, or similar procedures) should be applied to control Type I error. By visually highlighting these conditional relationships, the tool can reduce the exploratory reliance on large numbers of pairwise comparisons while still allowing formal inferential procedures to be applied when needed. At the same time, researchers must recognise the trade-off between Type I and Type II errors when interpreting interaction effects. Post-hoc testing without adjustment inflates false positives, whereas corrections reduce power. By highlighting influential factor combinations directly, EXCITE helps researchers identify meaningful conditional patterns before conducting confirmatory statistical tests. Regarding positioning relative to existing methods, EXCITE differs fundamentally from other visual machine-learning approaches like partial dependence plots and interaction decomposition models. Unlike these methods, which are designed for continuous predictors and predictive performance evaluation, EXCITE preserves the classical ANOVA context by mapping factor-level combinations through a tree-based representation. In this way, it serves as a complementary, ANOVA-coherent interpretive tool rather than a competing modelling framework. Although decision trees complement traditional ANOVA analyses effectively, they also present statistical and software constraints. Statistically, trees can be unstable when datasets are small or unrepresentative of the underlying population, producing inconsistent splits across samples [26,27,31]. A further issue concerns surrogate splits, backup rules CART uses when predictors contain missing values [27]. Because all examples in this study used complete-case data, the app’s behaviour with missing values has not yet been validated. Additionally, when predictors are strongly correlated, recursive partitioning algorithms may attribute variance to one predictor even when multiple correlated predictors contribute to the observed pattern. Consequently, variable importance measures derived from decision trees should be interpreted as descriptive indicators of variance partitioning rather than definitive evidence of predictor importance. Standard ANOVA assumptions of normality, independence, and homoscedasticity also remain in effect; significant violations can compromise both ANOVA and tree accuracy without appropriate data transformation. Therefore, these assumptions should be verified before using the tool. In addition, when predictors are highly correlated, CART may preferentially attribute variable importance to one predictor over another without indicating that alternative correlated predictors provide comparable explanatory information. Accordingly, variable-importance measures should be interpreted with caution in the presence of multicollinearity, and users are encouraged to assess multicollinearity before interpreting tree-based results. EXCITE is intended as an interaction interpretation and visualisation tool for an already-specified ANOVA model rather than a comprehensive model-diagnostic platform. From a software perspective, the current EXCITE version handles up to three-factor interactions and assumes basic ANOVA literacy from users. Moreover, the ANOVA-guided tree is conditioned on the statistical significance of the highest-order interaction and is therefore suppressed when this interaction is not significant, even if lower-order interactions remain significant. This design ensures consistency between the inferential evidence provided by the factorial ANOVA and the corresponding tree-based visualisation but limits the exploration of lower-order interactions in such situations. The current implementation also does not yet incorporate covariates as in ANCOVA. The restriction to two- and three-factor ANOVA is intentional and reflects a principled first step in validating the integration of decision trees with classical factorial designs. Two- and three-way interactions represent the most commonly used experimental structures in applied research, and they provide a tractable setting in which to establish the correctness and interpretability of tree-based visualisation. While the decision-tree visualisation organises subgroup means in a hierarchical structure, the underlying statistical evidence remains grounded in the ANOVA framework presented alongside the tree output. Users should therefore interpret the tree as a visual summary of conditional relationships supported by the factorial model rather than as a substitute modelling framework. Extending EXCITE to four-way and higher-order interactions would introduce substantial visual complexity, making it difficult to rigorously test whether the approach enhances rather than hinders understanding. By first demonstrating that tree-based representations reliably capture and clarify lower-dimensional interactions, this work lays the methodological foundation for incorporating more complex designs in future versions. A further limitation concerns the absence of direct empirical benchmarking against existing tools such as moreThanANOVA, partial dependence plots, or interaction decomposition models. Because EXCITE was developed as an interpretability companion to factorial ANOVA rather than a predictive or assumption-checking method, direct performance comparisons are not meaningful within the current framework. However, future evaluation will include a structured user study comparing EXCITE with multi-panel interaction plots and model-agnostic visualisations on interpretability outcomes, such as time-to-insight, accuracy in identifying the correct interaction pattern, and user confidence. Future development should focus on improving model robustness and extending analytical scope. Because CART relies on surrogate splits for missing data, systematic benchmarking under realistic missing-data conditions, combined with user options to rank or disable surrogates, would increase reliability. Integrating ensemble algorithms such as random forests or gradient-boosted trees could stabilise predictions and variable-importance measures while preserving the existing interface. Another enhancement would be to allow the complexity parameter to adapt when higher-order interactions are non-significant, but lower-order ones remain important, thereby broadening pedagogical and applied value. Future developments could also explore alternative criteria for guiding tree complexity, such as effect-size, based or hybrid approaches, to complement the current p-value-guided heuristic. Expanding EXCITE to include covariates (ANCOVA) and higher-order or repeated-measures designs, alongside planned interpretability benchmarks with existing approaches, would further extend its usability. Such extensions would position EXCITE as a comprehensive, interaction-centred analytics platform rather than a single-tree extension of ANOVA. 5. Conclusion EXCITE addresses a long-standing limitation in ANOVA-based research, the difficulty of interpreting complex interactions. By replacing multiple split-plot visualisations with a single, tree-based representation, the tool enables researchers to interpret multifactor interactions more intuitively and to recognise patterns that might otherwise remain obscured across separate plots. This integrated visualisation approach may help clarify the contribution of each factor combination and enhance understanding of higher-order interactions. Beyond visualisation, EXCITE extends traditional ANOVA by pinpointing which factor-level combinations drive significant interactions. While omnibus F-tests merely signal the presence of an interaction, EXCITE’s decision-tree framework reveals where and how those effects occur, providing actionable insights rather than abstract significance. This targeted identification offers a principled alternative to conventional post-hoc pairwise comparisons, which often oversimplify complex effects and inflate Type I error through multiple testing. By combining the inferential rigour of ANOVA with the interpretive precision of decision trees, EXCITE offers a practical and conceptually accessible framework for understanding multifactor relationships. The tool enhances statistical thinking by linking significance testing with visual exploration, empowering researchers across disciplines to interpret interactions more comprehensively and accurately. In doing so, EXCITE contributes to improving the quality and transparency of inference in experimental research and represents a step toward more intuitive, evidence-driven analytics in factorial designs. Taken together, these contributions position EXCITE as a complementary, ANOVA-coherent visualisation tool whose advantages will be further quantified through planned user-centred benchmarking focused on interpretability outcomes. 11 Sep 2025 Dear Dr. Savieri, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Oct 26 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Iftikhar Ahmed Khan Academic Editor PLOS ONE Journal Requirements: 1. When submitting your revision, we need you to address these additional requirements.-->--> -->-->Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at -->-->https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and -->-->https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf-->--> -->-->2. We note that your Data Availability Statement is currently as follows: All relevant data are within the manuscript and its supporting information files.-->--> -->-->Please confirm at this time whether or not your submission contains all raw data required to replicate the results of your study. Authors must share the “minimal data set” for their submission. PLOS defines the minimal data set to consist of the data required to replicate all study findings reported in the article, as well as related metadata and methods (https://journals.plos.org/plosone/s/data-availability#loc-minimal-data-set-definition).-->--> -->-->For example, authors should submit the following data:-->--> -->-->- The values behind the means, standard deviations and other measures reported;-->-->- The values used to build graphs;-->-->- The points extracted from images for analysis.-->--> -->-->Authors do not need to submit their entire data set if only a portion of the data was used in the reported study.-->--> -->-->If your submission does not contain these data, please either upload them as Supporting Information files or deposit them to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories.-->--> -->-->If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. If data are owned by a third party, please indicate how others may request data access.-->--> -->-->3. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise. ?> Additional Editor Comments: Please see the attached comment file. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? Reviewer #1: Partly Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? -->?> Reviewer #1: No Reviewer #2: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available??> The PLOS Data policy Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English??> Reviewer #1: Yes Reviewer #2: Yes ********** Reviewer #1: Please see my comments attached. Please see my comments attached. (Sorry for this) Reviewer #2: The manuscript is technically sound and the analysis is well designed. Creating visualization techniques for interpretation effects in ANOVA is highly challenging. Looking at the analysis and methodology presented in the article, it seems an excellent way to present visualization of interpretation effects in. a multi variable ANOVA. The authors have explained the methodology using examples very well. They also explained the mathematical reasoning quite well so that anyone with a little knowledge can understand it clearly. The examples help understanding the complex analysis presented in a simple manner making it easy for non mathematicians to understand as well. ********** what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy Reviewer #1: No Reviewer #2: Yes: Priyadarsini Dasari ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. Submitted filename: Review report of the manuscript.pdf 16 Dec 2025 Manuscript ID: PONE-D-25-38509 Manuscript title: Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations Reviewer #1: Weaknesses highlighted 1. The integration of decision trees with ANOVA is only superficially explained. For example, the rationale for linking ANOVA p-values with the complexity parameter (CP) remains unclear and insufficiently justified. We appreciate this insightful observation. We have expanded the explanation of how EXCITE links ANOVA p-values to the tree’s complexity parameter (CP) and clarified the conceptual rationale for this integration. Specifically, we now (i) provide a clearer justification that connects the strength of statistical evidence (p-value) to the permissible tree complexity, (ii) describe how this mapping maintains coherence between ANOVA inference and tree visualisation, and (iii) include supporting literature on cost-complexity pruning and model selection. These revisions appear in the Tree Selection in EXCITE subsection (page 12, lines 276-304). 2. The tool is not empirically compared with existing approaches (e.g., moreThanANOVA, partial dependence plots, interaction decomposition models). Without such benchmarks, it is difficult to assess the practical advantages of EXCITE. We appreciate this important suggestion. Our primary objective in this manuscript is to introduce EXCITE as a single-view, interaction-centred visualisation companion to factorial ANOVA, rather than to propose a new estimator or a replacement for existing methods. Tools such as moreThanANOVA focus on assumption checks and post-hoc testing automation, while partial dependence plots (PDPs) and interaction decomposition models typically arise in machine-learning workflows and target different model classes (often continuous/coded predictors and flexible nonparametric fits). Because EXCITE is explicitly tied to factorial ANOVA with categorical factors and emphasises a one-diagram display of interaction structure, a direct “accuracy” or “predictive performance” benchmark is not the most informative comparison. That said, we agree that positioning EXCITE against existing approaches will help readers understand its practical value. We have therefore (i) added a short positioning subsection that clarifies where EXCITE sits relative to moreThanANOVA, PDPs, and interaction decomposition, and (ii) expanded the limitations/future work to outline a planned user-study benchmarking interpretability outcomes (time-to-insight, accuracy of identifying the correct interaction pattern, and user confidence) across EXCITE vs. multi-panel interaction plots and PDP-style views. We believe these additions address the reviewer’s concern while keeping the paper focused on EXCITE’s intended role as an interpretability aid for factorial ANOVA. 3. The tool is restricted to two- and three-factor ANOVA designs. Although this limitation is acknowledged, the discussion of potential extensions is too superficial to compensate. We thank the reviewer for raising this important point. Our intention in the present manuscript is to establish a clear and rigorous foundation for integrating decision-tree methodologies with classical ANOVA, with an emphasis on interpretability and methodological transparency. Because this work represents the first systematic step in exploring the use of decision trees to visualise ANOVA interactions, we deliberately restricted the implementation to two- and three-factor designs. This decision was made for two main reasons: Conceptual clarity and methodological validation. Two- and three-way interactions represent the most commonly used factorial designs in applied research, and they provide a tractable setting in which to test, validate, and illustrate how hierarchical tree structures can meaningfully and accurately represent ANOVA interactions. Introducing higher-order interactions (four-way and above) without first establishing interpretability and correctness in lower-dimensional settings would risk confounding the methodological contribution. Interpretability considerations. One of EXCITE’s core aims is to reduce the burden associated with interpreting multi-factor interactions. While decision trees can technically accommodate higher-order splits, the resulting trees quickly become unwieldy and visually dense. Beginning with two- and three-factor designs allows us to demonstrate the interpretive advantages of the approach before extending it to settings where visual complexity naturally increases. We agree that outlining future extensions is valuable, and we have therefore expanded the Discussion to articulate a clearer development roadmap. This includes extending EXCITE to support (i) four-way and higher-order interactions, (ii) ANCOVA models with covariates, and (iii) the use of ensemble tree methods to improve robustness at higher dimensionality. These additions help to position EXCITE as an initial, principled foundation for a broader class of tree-based visualisation tools for factorial designs. 4. The examples rely solely on simulated datasets. No real-world case study is provided, which undermines the demonstration of practical applicability. We thank the reviewer for this helpful observation. In the revised manuscript, we have now added a real-world case study based on data from a study on an online social interaction experiment (Thériault et al., 2023). This newly added analysis, presented in Section 3.2, demonstrates how EXCITE performs on empirical data containing naturally occurring variability, non-ideal balance, and realistic interaction patterns. Including this example directly addresses the reviewer’s concern by showing EXCITE’s applicability to actual research scenarios. At the same time, we maintain that the simulation examples remain an important and deliberate component of the manuscript. Simulated datasets allow us to (i) clearly illustrate the mechanics of EXCITE in controlled settings where the underlying interaction structure is known, (ii) demonstrate how the built-in simulation module can alter noise levels, sample sizes, and effect patterns to mirror real-world conditions, and (iii) ensure reproducibility when explaining methodological features of the tool. This dual-example strategy, using both controlled simulations and a real dataset, enables readers to see EXCITE applied in transparent, pedagogical settings as well as in an authentic applied context. The combination of real and simulated examples, therefore, provides a balanced and comprehensive demonstration of the tool’s practical usability. We have also updated the Discussion to reflect this enhancement. 5. Figures (especially tree diagrams) are difficult to read and could be improved for clarity. We thank the reviewer for this valuable observation. In the revised manuscript, all figures, including the decision tree diagrams, have been updated to improve clarity, consistency, and visual accessibility. We corrected and reformatted the plots using the journal’s PACE-compliant figure guidelines, ensuring improved readability, higher resolution, and clearer labelling across the tree diagrams, interaction plots, and ANOVA visualisations. We believe these revisions substantially enhance the interpretability and overall presentation quality of the figures. 6. References contain inconsistencies and omissions. We thank the reviewer for noting the inconsistencies in the reference list. Although the manuscript was originally prepared using Mendeley, we have now conducted a full, manual audit of all references. We verified each entry against the source, corrected incomplete information (e.g., missing page ranges, inconsistent publisher details, and truncated DOIs), resolved duplicates, and ensured consistent formatting of book titles and journal names in accordance with the journal’s reference style. We are confident that the revised reference list is now complete and fully consistent. 7. The writing style is repetitive at times, which weakens the overall clarity of the manuscript. We thank the reviewer for this valuable feedback. We have carefully reviewed the entire manuscript to eliminate redundancy and improve clarity. Specifically, we revised sentences that repeated definitions or concepts across sections (e.g., reiterations of ANOVA fundamentals, decision tree principles, and descriptions of the app workflow) and consolidated overlapping explanations. We also streamlined transitional phrases and restructured several paragraphs for smoother logical flow between sections. These revisions have shortened the text, reduced duplication, and improved readability while preserving all essential technical details. Changes were implemented throughout the Methods (Sections 2.1-2.4), Results (Examples 1 and 2), and Discussion sections. Submitted filename: Response to Reviewers.docx 15 Feb 2026 Dear Dr. Savieri, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Apr 01 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Iftikhar Ahmed Khan Academic Editor PLOS One Journal Requirements: If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author Reviewer #3: All comments have been addressed Reviewer #4: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions??> Reviewer #3: Yes Reviewer #4: Partly ********** 3. Has the statistical analysis been performed appropriately and rigorously? -->?> Reviewer #3: Yes Reviewer #4: No ********** 4. Have the authors made all data underlying the findings in their manuscript fully available??> The PLOS Data policy Reviewer #3: Yes Reviewer #4: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English??> Reviewer #3: Yes Reviewer #4: Yes ********** Reviewer #3: Author has responded very well to the earlier comments given by reviewers and has addressed all the required changes. He also applied recommendations given, even though he justified why it wasn't not in the initial manuscript. He used real data to support his findings abs did a rigorous analysis. I suggest approval for this manuscript. Reviewer #4: Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations The authors present a regression tree based web-app to succinctly visualize interactions from ANOVA hierarchically. I appreciate the authors' goal to simplify understanding and communicating high-order interaction terms, but it is unclear to me if the described approach accomplishes that goal. I have some questions about the approach, as well as a couple points of disagreement. - You wrote repeatedly that your approach shows the "hierarchical structure" of the interaction. Interactions in ANOVAs of the sort you discuss have no hierarchy. They are simultaneous functions of all involved variables. I believe you know this and that you mean something else, like a ranking of sums of squares or ranking of binary-split mean differences. I believe you mentioned a few times that readers and researchers struggle with interactions. If you tell them several times that their interactions have hierarchical structure, will they not come away from your work thinking that? That won't be helpful. Helping people understand interactions is important. Trying to accomplish that by turning them into something they are not, like hierarchical trees, is unhelpful. In some sense, it parallels some methodologists' recommendations to look at simple slopes and now many applied researchers turn their interactions into a simple slopes plot and just interpret simple slopes. That is inadequate too. - You also wrote at least a couple times that your EXCITE approach removes the need for post-hoc means testing. At least that is how I understood you. I believe you asserted this, but I don't think you explained it. I also don't think it could be true, nor does it seem reasonable on logical grounds. For example, if I looked at my data in a way similar to your tree in Figure 10, meaning I started with some marginal difference that I expected (or knew) was biggest and then looked at mean differences by factor 2 within levels of factor 1, I'd have the middle of your tree. Then if I just looked at all eight cell means, I'd have the bottom of your figure. If I do this by hand in R or SPSS, write it up and not at least discuss p-value correction, I'd hope and expect reviewers to be highly critical. You seem to be saying that if a regression tree does all those comparisons for you and just shows you the ones it arrived at, that is different. Why? On lines 604 and 605, you write that EXCITE highlights the influential combinations directly, but the algorithm is checking all the combinations too, isn't it? - You also treat the p-value like a very concrete entity, linking your tuning parameter to it. Aren't p-values more heuristic and generally informative, rather than a hard and fast measure of support. Methods researchers have been encouraging people to rely less on p-values and more on effect sizes for decades. You appear to be putting a lot on the p-value. Relatedly, we often have quite low power for interactions. Rather than tuning low power and likely poorly estimated terms away, with more aggressive pruning, shouldn't we be encouraging researchers to think more clearly about high-order interactions, and ways to test and understand them better? - lines 208 to 217, I think you are saying that your method identifies important factors even in the presence of multicollinearity. My understanding of the issue is that we cannot usually identify the problem variable in a multicollinear regression problem, which of course includes your ANOVA problems. Taking your example even, "teaching method" and "school type" may be highly associated and therefore your tree algorithm attributes the variance to teaching method and omits school type from the final model. That presents the results much more concretely than is warranted. An algorithm picks teaching method, but if they are collinear, we don't actually know what's most important. If I understand this section, I think you are saying the researcher would be told one factor is the important one, when in reality, we don't really know. - I don't think you really explain, nor illustrate, why we want the "cost-complexity pruning" you introduce on line 254. If I have done a good experiment and been told I need some k-order interaction to understand the pattern of means, why isn't that the right place to start? Why do I want a tree algorithm to tell me to ignore smaller effects? I'm doing science after all, not trying to have simple results or small tables. - lines 530 and 531: is it bad that we ask scientists and readers to take time to think through and work to understand interactions? They can be complicated. Complicated things can take time. That's okay, isn't it? - Last, is your three-way ANOVA example good? All the partial eta^2 CIs are huge, like all (or nearly all) the possible numeric range. When I see something like that, I usually thinks there is something wrong with the data. Is such a weak data example good for this paper? If you look for a new example, maybe include one that illustrates more of the benefit of bringing these trees to your purpose. - In your response to Reviewer 1, you give a rationale for only discussing 2- and 3-way ANOVA interactions. But on line 146, you introduce n-way AVOVA. That seems unnecessary and adds rather little to the content. For example, you don't discuss higher-order terms. Notationally, it is also confusing. Earlier, you used i,j,k, but here it is i_1, i_2, ..., and then r. - on line 157, you point out that researchers frequently omit higher-order interactions... "to simplify interpretation". Can you add the words "lazily and sloppily", as in "...researchers frequently lazily and sloppily omit higher-order interactions..." That is only half a joke. Can you convey that while people frequently do omit higher-order terms to make their work easier, that that is not how one does good science? typos, corrections, etc. - line 135, where should be lower case and I assume a comma should follow the equation - line 186, you write "optimal", but optimal what? - line 221, rpart is implemented in a package or library, not in a manual. - figure 10, I think one of the Cyberballs should be Uberball ********** what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy Reviewer #3: Yes: Samuel Sarpong Fosu Reviewer #4: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] To ensure your figures meet our technical requirements, please review our figure guidelines: https://journals.plos.org/plosone/s/figures You may also use PLOS’s free figure tool, NAAS, to help you prepare publication quality figures: https://journals.plos.org/plosone/s/figures#loc-tools-for-figure-preparation. NAAS will assess whether your figures meet our technical requirements by comparing each figure against our figure specifications. 31 Mar 2026 Iftikhar Ahmed Khan, Academic Editor PLOS ONE March 25, 2026 Dear Editor and Reviewer(s), Thank you for taking the time to review our work (PONE-D-25-38509R1) and providing us with insightful feedback on ways to strengthen our paper. We have incorporated the suggested changes and addressed each point below. We believe that the revised version better communicates the novelty and significance of our work. We are confident that our web application stands as a valuable addition to the existing body of knowledge in this field. We hope that the revised version meets your expectations. Thank you for your time and consideration. On behalf of all the authors, Yours sincerely, Perseverence Savieri Corresponding Author Biostatistics and Medical Informatics Research Group (BISI) Vrije Universiteit Brussel (VUB), Brussels 1090, Belgium Perseverence.Savieri@vub.ac.be perseverencesavieri@gmail.com Manuscript ID: PONE-D-25-38509R1 Manuscript title: Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations Response to Reviewer #4: We thank the reviewer for the thoughtful and methodologically engaged comments. You raise several conceptual points regarding the interpretation of interactions and the positioning of the EXCITE framework. We have revised the manuscript to clarify these aspects. Importantly, EXCITE is designed as a visualisation and interpretation tool that complements classical ANOVA, rather than replacing its inferential framework. Below, we address each comment in detail. Major comments Comment 1: Interactions do not have a hierarchical structure We agree with the reviewer that ANOVA interaction terms are not hierarchical. Interactions represent simultaneous conditional relationships among factors, and our intention was not to imply otherwise. Instead, the hierarchy arises from the visual representation imposed by recursive partitioning, which orders conditional mean differences according to their contribution to variance reduction. In other words, the hierarchy in the trees reflects the structure of the tree visualisation, not the statistical structure of the interaction itself. You are right that we should make this very clear. Nonetheless, tree-based representations of interactions are widely used in statistical learning because they provide an interpretable representation of conditional relationships (Breiman et al., 1984; Hastie et al., 2009). In this context, our trees should be interpreted as a visual ordering of conditional effects, analogous to the way interaction plots organise effects across panels. To avoid possible misunderstanding, we clarified this explicitly in the manuscript (lines 174-176) and revised several occurrences of the term “hierarchical” so that it refers only to the structure of the tree representation rather than to the interaction itself. Comment 2: The method appears to remove the need for post-hoc testing. We appreciate the opportunity to clarify this point. EXCITE does not replace formal post-hoc inference, and this was not our intention. Rather, EXCITE addresses a common practical issue: researchers often conduct many exploratory pairwise comparisons to identify interaction patterns, and these comparisons can proliferate rapidly as the number of factor levels increases. The decision tree provides a data-driven visual decomposition of conditional means, allowing researchers to identify influential factor combinations before conducting confirmatory contrasts. However, this exploratory step does not eliminate the need for formal statistical inference. When specific contrasts are tested, appropriate adjustments for multiple comparisons (e.g., Bonferroni, Tukey, or similar procedures) remain necessary to control Type I error. To avoid ambiguity, we have revised the manuscript to state that EXCITE complements explicitly, but does not replace, post-hoc inference, and that multiplicity correction remains essential when evaluating statistical significance (lines 609–612). Comment 3: You write that EXCITE highlights the influential combinations directly, but the algorithm is checking all the combinations too, isn't it? We appreciate this clarification request. The reviewer is correct that the CART algorithm evaluates many possible candidate splits internally when constructing the regression tree. In that sense, the algorithm implicitly considers multiple possible partitions of the factor levels. However, the purpose of the decision-tree representation in EXCITE is not to enumerate all possible comparisons but to identify and display the partitions that produce the largest reductions in within-node variance, as defined by the CART splitting criterion (Breiman et al., 1984). In other words, although many candidate combinations are evaluated during tree construction, the final visualisation presents only the subset of conditional splits that most strongly structure the response variable. This behaviour differs from manually examining all cell means or conducting numerous pairwise comparisons, where every contrast must be inspected separately and, importantly, requires appropriate correction for multiple testing. The decision tree, therefore, provides a structured summary of the most influential conditional relationships, which can guide subsequent confirmatory analyses. When formal comparisons are conducted based on these patterns, standard multiplicity adjustments remain necessary to ensure valid statistical inference. To clarify this point, we added a short explanation in the manuscript (lines 183-187 and lines 618-631). Comment 4: The method relies heavily on p-values when tuning the complexity parameter. The reviewer raises an important point regarding the interpretation of p-values. In EXCITE, the p-value is used only as a heuristic to guide tree complexity, not as a strict inferential criterion. Specifically, the mapping between interaction significance and the complexity parameter aligns the depth of the tree visualisation with the strength of the statistical evidence. This approach prevents overly complex trees when interaction evidence is weak, while still allowing deeper partitions when interactions are strongly supported. Importantly, the ANOVA output displayed in the application includes effect sizes (partial η²) alongside p-values, and the interpretation of results should always consider both. We have clarified this point in the manuscript (lines 286-289). With respect to the reviewer’s concern about the typically low statistical power of interaction effects, it is important to note that the pruning rule affects only the visual tree representation and does not alter the underlying factorial ANOVA results. All estimated interaction terms remain available in the statistical output, allowing researchers to investigate higher-order effects directly, even when the tree representation is simplified for interpretability. The pruning rule therefore serves to stabilise the visualisation and reduce over-interpretation of small or unstable subgroup splits, rather than discouraging careful examination of interaction effects. We agree that alternative criteria, such as effect-size–based thresholds, could provide a valuable complement to p-value–guided pruning. The current implementation represents a first, pragmatic approach aligned with conventional ANOVA outputs, but future versions of EXCITE could incorporate effect-size–driven or hybrid criteria to guide tree complexity. We have added this as a limitation and direction for future development in the manuscript (lines 695-697). Comment 5: The multicollinearity example may imply that the algorithm identifies the most influential variable. We agree with the reviewer that when predictors are strongly correlated, recursive partitioning methods, like regression models, cannot reliably distinguish which variable is most influential. In such cases, the allocation of variance to a particular predictor depends on the structure of the data and the splitting criterion, and different but highly correlated predictors may lead to similar partitions. The purpose of the example was to illustrate that variable importance measures can reflect how predictors contribute to variance partitioning within the tree, even if they are not selected as splitting variables in the final pruned tree. However, we acknowledge that in the presence of multicollinearity, these measures should not be interpreted as uniquely identifying the most important predictor. We have revised the manuscript to clarify that variable importance reflects the role of predictors within the tree representation and should be interpreted with caution when predictors are correlated (lines 309–315). Comment 6: Why use cost-complexity pruning if higher-order interactions are scientifically meaningful? Cost-complexity pruning is a standard component of CART methodology (Breiman et al., 1984) and is used to prevent overfitting caused by recursive splitting of small subsets. The goal in EXCITE is not to suppress meaningful interactions but to ensure that the tree visualisation remains stable and interpretable. Without pruning, regression trees often grow deep structures that reflect sampling noise rather than meaningful patterns. Importantly, the ANOVA results remain fully available in the interface, meaning that all estimated interaction terms remain accessible even when the visual tree is pruned (lines 261-265). Comment 7: Is it problematic that interactions require time to interpret? We agree that complex interactions often require careful interpretation. The aim of EXCITE is not to oversimplify statistical reasoning but to provide a visual framework that helps researchers organise conditional relationships more efficiently. In many applied settings, interaction plots across multiple panels can be difficult to synthesise visually. The tree representation provides a complementary view that consolidates these conditional relationships in a single diagram. Comment 8: Concern about the three-way ANOVA example and wide effect-size intervals. We thank the reviewer for this important observation. We agree that the wide confidence intervals for the partial η² estimates indicate that the effect sizes are estimated with considerable uncertainty, making it difficult to distinguish precisely between small and larger effects based on these estimates alone. This imprecision likely reflects the limited information available for estimating higher-order interaction effects in the dataset, a common issue in factorial designs where interaction effects tend to have lower statistical power and greater variability than main effects. The purpose of this example is not to emphasise precise effect-size estimation, but to illustrate how EXCITE visualises conditional interaction structures in a realistic applied setting. Importantly, the interaction pattern itself is consistent with the findings reported in the original study (Thériault et al., 2023), and the decision tree representation focuses on the relative structure of subgroup differences rather than the magnitude of effect sizes. To address the reviewer’s concern, we have revised the manuscript to explicitly acknowledge the uncertainty in the effect-size estimates and clarify that the example is intended for visualisation purposes rather than precise effect-size interpretation (lines 507-510). Comment 9: n-way ANOVA notation is unnecessary and adds rather little to the content. We agree and have removed the general n-way notation to improve clarity. Comment 10: Suggestion regarding wording on higher-order interactions While we appreciate the reviewer’s humour, we felt that adopting this wording might introduce an unintended tone. However, we agree with the underlying point that higher-order interactions should not be omitted purely for convenience. To reflect this, we have revised the manuscript to emphasise that while higher-order interactions are often excluded to simplify interpretation or preserve statistical power, model specification should be guided by theoretical considerations and study design rather than convenience alone. This clarification has been added in the Methods section (lines 148–150). Minor corrections • All typographical issues identified by the reviewer were corrected. • Figure 10 clarification: The reviewer suggested a possible labelling error in Figure 10. We carefully checked the figure and confirmed that the labels are correct. No changes were required. We thank the reviewer again for the constructive comments, which helped improve the clarity and conceptual positioning of the manuscript Submitted filename: Response_to_Reviewers_auresp_2.docx 6 Jul 2026 Dear Dr. Savieri, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. The reviewer though recommended acceptance, yet raise few questions for calrification. You are requested to address the query and resubmit. Please submit your revised manuscript by Aug 20 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. As the corresponding author, your ORCID iD is verified in the submission system and will appear in the published article. PLOS supports the use of ORCID, and we encourage all coauthors to register for an ORCID iD and use it as well. Please encourage your coauthors to verify their ORCID iD within the submission system before final acceptance, as unverified ORCID iDs will not appear in the published article. Only the individual author can complete the verification step; PLOS staff cannot verify ORCID iDs on behalf of authors. We look forward to receiving your revised manuscript. Kind regards, Iftikhar Ahmed Khan Academic Editor PLOS One Journal Requirements: 1. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise. 2. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author Reviewer #4: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions??> Reviewer #4: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? -->?> Reviewer #4: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available??> The PLOS Data policy Reviewer #4: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English??> Reviewer #4: Yes ********** Reviewer #4: Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations PONE-D-25-38509R2 Dear authors, Thank you for answering my questions and explaining your perspective and thinking. I understand your goals and procedure better. Upon reading the second revision, I had a few more questions or points of clarification. - lines 269--271: You write that "... p_{int} denotes the smallest omnibus interaction p-value..., representing the strength of the evidence for the highest order interaction." While it may be common that the highest order interaction often has the largest p-value, that isn't a mathematical certainty, is it? Couldn't a mean square for a two-way interaction, in a three factor experiment have a smaller p-value than the three-way interaction? If this is correct, that it is possible, then shouldn't p_{int} always be the p-value for the highest order interaction, "representing the strength of the evidence for the highest order interaction"? - line 285, the difference between "all possible interactions" and "model interactions" isn't clear to me in the context of your paper. It sounds like EXCITE starts with the three-way interaction, which would include all possible interactions. Then what can "model interactions" be? It sounds like it would be for a user specified model that did not include all possible interactions, but you don't really discuss, nor does it sound like EXCITE does this. - line 466, I know you added language specifying that interactions are not hierarchical, which I appreciate. However, on line 466, you describe interaction effects "unfold"ing. Maybe I'm being too fussy, but this suggests a process in my mind, which again, I think is going to confuse matters for your potential users. Your readers are people who want help with two- and three-way interactions in ANOVA, which are among the easiest interactions to interpret. These readers do not need language that creates a false impression. ANOVA interactions don't unfold. As you write, they are simultaneous conditional associations. I think it would be preferrable to say something like ... showing how the main and interaction effects are characterized by successive splits. - Last, in a few places you describe how CART can misleadingly attribute variance, and therefore importance, to one of a set of collinear variables. This seems like a difficulty easily solved in software. Again, I presume your users are people who aren't very statistically savvy. They may not have a good understanding of multicollinearity. Despite your statements about possible collinearity and CART not flagging that, many users are likely to take the CART results as correct. Can you add something to EXCITE to check multicollinearity, perhaps separate from the cart procedure, and issue a noticeable warning or otherwise make it difficult for the user to ignore? ********** what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy Reviewer #4: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] To ensure your figures meet our technical requirements, please review our figure guidelines: https://journals.plos.org/plosone/s/figures You may also use PLOS’s free figure tool, NAAS, to help you prepare publication quality figures: https://journals.plos.org/plosone/s/figures#loc-tools-for-figure-preparation. NAAS will assess whether your figures meet our technical requirements by comparing each figure against our figure specifications. 12 Aug 2026 Manuscript ID: PONE-D-25-38509R2 Manuscript title: Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations Reviewer #4: Weaknesses highlighted Comment 1: Clarification of the definition of p_int We thank the reviewer for identifying this ambiguity. The implementation of EXCITE does not use the smallest interaction p-value across all interaction terms. Rather, it uses the p-value corresponding to the highest-order interaction term (i.e., the two-way interaction in a two-factor ANOVA or the three-way interaction in a three-factor ANOVA) to determine the complexity parameter. We have therefore revised the manuscript (lines 269-272) to remove the phrase "smallest omnibus interaction p-value" and instead explicitly state that p_int denotes the omnibus p-value for the highest-order interaction. This wording more accurately reflects both the methodology and software implementation. Comment 2: Clarification of tree complexity options Thank you for this helpful observation. We agree that the previous label "Model interactions" was ambiguous and did not accurately reflect the functionality of EXCITE. The application always fits the complete factorial ANOVA model; the available options control only how the resulting decision tree is displayed and pruned, rather than which interaction terms are included in the ANOVA model. To address this, we simplified the user interface by replacing the previous interaction labels with two tree display modes: Maximum tree complexity and ANOVA-guided tree (recommended). We also revised the corresponding manuscript text (lines 284-304 and 390-394) to clarify that these options control the degree of pruning applied to the decision tree while leaving the underlying ANOVA model unchanged. Comment 3: Wording describing interaction effects We thank the reviewer for this helpful suggestion. We agree that the term "unfold" could inadvertently imply a sequential process rather than the simultaneous conditional relationships represented by ANOVA interactions. We have revised the wording accordingly (lines 476-477) to clarify that the successive tree splits provide a hierarchical representation of the conditional structure of the fitted ANOVA model rather than suggesting that interaction effects develop sequentially. Comment 4: Handling multicollinearity in EXCITE We appreciate the reviewer's suggestion. We agree that multicollinearity can influence CART variable-importance measures and may not be readily recognised by inexperienced users. However, EXCITE was intentionally designed as an interaction interpretation and visualisation tool that operates after a factorial ANOVA model has been specified, rather than as a comprehensive model-building or diagnostic platform. Incorporating multicollinearity diagnostics would substantially broaden the scope of the application beyond its intended purpose and would logically require the inclusion of other model diagnostics, such as checks for normality, homoscedasticity, independence, and influential observations. Instead, we have strengthened the discussion of this limitation in the manuscript (lines 673-680) by explicitly advising users to assess multicollinearity and other model assumptions prior to interpreting the decision tree and its variable-importance measures. Submitted filename: Response_to_Reviewers_auresp_3.docx 20 Aug 2026 Integrating Decision Trees with ANOVA for Enhanced Interaction Visualisations PONE-D-25-38509R3 Dear Dr. Savieri, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Iftikhar Ahmed Khan Academic Editor PLOS One Additional Editor Comments (optional): Reviewers' comments: PONE-D-25-38509R3 PLOS One Dear Dr. Savieri, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS One. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. You will receive an invoice from PLOS for your publication fee after your manuscript has reached the completed accept phase. If you receive an email requesting payment before acceptance or for any other service, this may be a phishing scheme. Learn how to identify phishing emails and protect your accounts at https://explore.plos.org/phishing. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS One and supporting open access. Kind regards, PLOS One Editorial Office Staff on behalf of Dr. Iftikhar Ahmed Khan Academic Editor PLOS One