Automated residual plot assessment with the R package autovi and the Shiny application autovi.web

Weihao Li \addressnum 1, 2, Dianne Cook \addressnum 1, Emi Tanaka \addressnum 2, Susan VanderPlas \addressnum 3 and Klaus Ackermann \addressnum 1 · 2026

Read Automated residual plot assessment with the R package autovi and the Shiny application autovi.web by Weihao Li \addressnum 1, 2, Dianne Cook \addressnum 1, Emi Tanaka \addressnum 2, Susan VanderPlas \addressnum 3 and Klaus Ackermann \addressnum 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24236.txt.

Introduction

Li, Cook, Tanaka, VanderPlas, and Ackermann automate the squinting at residual plots that diagnoses regression models. Analysts judge model fit by eye from plots of leftover errors, a practice that scales badly and varies between people. The lineup protocol, hiding the real plot among decoy null plots, cuts subjectivity but demands even more human effort. Their 2026 package, autovi, hands the judgement to a computer vision model instead.

Handed a sample of residuals, the model returns a visual signal strength score and returns supporting information to guide the analyst's own assessment. A companion Shiny web application, autovi.web, makes the tool usable without programming. Li's team frames automation as consistency: the same plot earns the same evaluation regardless of who asks or how tired they are. The R package targets initial data analysis workflows where quick, repeatable diagnostic checks matter more than bespoke graphical craftsmanship.

Catalog record

CREATORWeihao Li \addressnum 1, 2, Dianne Cook \addressnum 1, Emi Tanaka \addressnum 2, Susan VanderPlas \addressnum 3 and Klaus Ackermann \addressnum 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24236

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automated residual plot assessment with the R package autovi and the Shiny application autovi.web Weihao Li \addressnum 1, 2, Dianne Cook \addressnum 1, Emi Tanaka \addressnum 2, Susan VanderPlas \addressnum 3 and Klaus Ackermann \addressnum 1 Abstract Visual assessment of residual plots is a common approach for diagnosing linear models, but it relies on manual evaluation, which does not scale well and can lead to inconsistent decisions across analysts. The lineup protocol, which embeds the observed plot among null plots, can reduce subjectivity but requires even more human effort. In today’s data-driven world, such tasks are well-suited for automation. We present a new R package that uses a computer vision model to automate the evaluation of residual plots. An accompanying Shiny application is provided for ease of use. Given a sample of residuals, the model predicts a visual signal strength (VSS) and offers supporting information to help analysts assess model fit. \runningheads Automated residual plot assessment W LI, D COOK, E TANAKA, S VANDERPLAS, AND K ACKERMANN keywords initial data analysis; statistical graphics; data visualisation; visual inference; computer vision; machine learning; hypothesis testing; regression analysis; model diagnostics 1 Introduction Regression analysis is a widely used statistical modelling technique for data in many fields. There is a vast array of software for conducting regression modelling and generating diagnostics. The package lmtest ( 44 ) provides a suite of conventional tests, while the stats package ( 32 ) offers standard diagnostic plots such as residuals vs. fitted values, quantile-quantile (Q-Q) plots, and residuals vs. leverage plots. Additional packages like jtools ( 23 ) , olsrr ( 16 ) , rockchalk ( 18 ) , and ggResidpanel ( 14 ) deliver similar graphical diagnostics, often with enhanced aesthetics or interactive features. These tools collectively produce the core diagnostic plots outlined in the classical text by 11 . The ecostats package ( 39 ) extends these diagnostics by incorporating simulation envelopes into residual plots. Meanwhile, DHARMa ( 15 ) compares empirical quantiles (0.25, 0.5, and 0.75) of scaled residuals to their theoretical counterparts, with a strong focus on identifying model violations such as heteroscedasticity, misspecified functional forms, and issues specific to generalised linear and mixed-effect models, like over/under-dispersion. It also provides conventional test annotations to reduce the risk of misinterpretation. However, relying solely on subjective assessments of these plots can lead to issues such as over-interpreting random patterns as model violations. 21 demonstrated that visual inference methods, particularly those using the lineup protocol ( 6 ) , offer more practical and reliable assessments of residual patterns than conventional tests, as they are less sensitive to minor departures. Packages such as nullabor ( 41 ) , HLMdiag ( 24 ) , and regressinator ( 34 ) support this approach by enabling users to compare observed residual plots with plots generated under null hypothesis, thereby helping to quantify the significance of any detected patterns. As noted in 20 , the lineup protocol has significant limitations in large-scale applications due to its reliance on human labour. To overcome this constraint, a computer vision model was developed alongside a corresponding statistical testing procedure to automate the assessment of residual plots. The model takes as input a residual plot and a set of auxiliary variables (such as the number of observations) and outputs a predicted visual signal strength (VSS). This VSS estimates the degree of deviation between the residual distribution of the fitted model and the reference distribution expected under correct model specification. To make the statistical testing procedure and trained computer vision model widely accessible, we developed the R package autovi along with a companion web interface, autovi.web , which allows users to automatically assess their residual plots using the trained computer vision model. The remainder of this paper is structured as follows: Section 2 introduces the definition and computation of visual signal strength. Section 3 expands on the computation of the null and bootstrapped residuals. Section 4 provides a detailed documentation of the autovi package, including its usage and infrastructure. Section 5 focuses on the autovi.web interface, describing its design and usage, along with illustrative examples. Finally, Section 6 presents the main conclusions of this work. 2 Definition and computation of visual signal strength To train a computer vision model, a measure of the visible pattern in a plot is needed. We call this the visual signal strength (VSS), which measures how prominently a specific set of visual patterns appears in an image. This can be computed for a training set of data, and plots, where the generating distributions are specified. In the context of classical normal linear regression model diagnostics, VSS describes the clarity of visual patterns on a diagnostic plot that may indicate model violations. Violations can be categorised as weak, moderate, or strong, but here we treat it as a continuous positive real variable. Importantly, its interpretation depends on how it is linked to a function of the data or the underlying data generating process. Consequently, the calculation of VSS can vary across different model classes or within the same model, depending on the generating function. VSS estimates the distance between the residual distribution of a fitted classical normal linear regression model and a reference distribution (see 20 , for details) . The distance measure is based on the Kullback-Leibler (KL) divergence: D=\log\left(1+D_{KL}\right), where D_{KL} is given by: D_{KL}=\int_{\mathbb{R}^{n}}\log\frac{p(\hat{\boldsymbol{e}})}{q(\hat{\boldsymbol{e}})}p(\hat{\boldsymbol{e}})d\hat{\boldsymbol{e}}, (1) here, \hat{\boldsymbol{e}} denotes the residual vector from the regression model, and p(\cdot) and q(\cdot) are the probability density functions of the reference residual distribution \mathrm{P} and the true residual distribution \mathrm{Q} , respectively. This distance measure depends on knowledge of the true residual distribution, which is unknown in practice. To compute D_{KL} for the training samples, Equation ( 1 ) takes different forms depending on the specific model violations. For instance, where necessary higher-order predictors, \boldsymbol{Z} , and their corresponding parameter, \boldsymbol{\beta}_{Z} , are omitted from the fitted linear model, the distance measure can be expanded as follows: D_{KL}=\frac{1}{2}\left(\boldsymbol{\mu}_{z}^{\top}(\text{diag}(\boldsymbol{R}\sigma^{2}))^{-1}\boldsymbol{\mu}_{z}\right), where \boldsymbol{\mu}_{z}=\boldsymbol{R}\boldsymbol{Z}\boldsymbol{\beta}_{z} , \boldsymbol{R}=\boldsymbol{I}_{n}-\boldsymbol{X}(\boldsymbol{X}^{\top}\boldsymbol{X})^{-1}\boldsymbol{X}^{\top} and \boldsymbol{X} is the design matrix of the regression model. The computer vision model approximates this mapping from a set of residuals to its corresponding distance measure. It is trained on a large number of synthetic regression models, each designed to simulate specific violations of classical linear regression assumptions. These models incorporate non-linearity through Hermite polynomial transformations of predictors, heteroscedasticity by making the error variance a predictor-dependent function, and non-normality by drawing residuals from distributions such as discrete, uniform, and lognormal. Both simple and multiple linear regression structures are used, with controlled parameters to generate diverse and complex residual patterns. Since the data-generating process is known, the distance measure D can be explicitly calculated, enabling supervised training. The computer vision model takes a residual plot as input and outputs the corresponding distance measure, learning to quantify model violations directly from visual patterns. Additional details are provided in 20 . 3 Definition and simulation of null and bootstrapped residuals In the subsequent sections, we will frequently refer to null residuals and bootstrapped residuals, so it is helpful to first define and explain how they are generated. Null residuals are used to generate null plots within the lineup protocol framework, serving as the foundation for the statistical testing in our automated residual plot

Read the complete text