A Compound Logistic Regression Model for Binary Responses Anthony Almudevar, PhD, Jacob Almudevar, MSc Abstract Logistic regression is the most commonly used method for constructing predictive models for binary responses. One significant drawback to this approach, however, is that the asymptotes of the logistic response function are fixed at 0 and 1, and there are many applications for which this constraint is inappropriate. More flexible models have been proposed for this application, most proceeding by supplementing the logistic response function with additional parameters. In this article we extend these models to allow correlated responses and the inclusion of covariates. This is achieved through the compound logistic regression model , for which the mean response is a function of several logistic regression functions. This permits a greater variety of models, while retaining the advantages of logistic regression. The methods are implemented in an R package CLmodel ( https://github.com/ALMUDEVAR163/CLmodel/ ). keywords Quasi-likelihood; Binary Models; Correlate of Protection 1 Introduction We are given paired observations (x_{i},y_{i}) , i=1,\ldots,n , where x_{i} is a p -dimensional vector of covariates and y_{i} is a binary response, interpreted as a Bernoulli random variable. The problem is to use the observations to estimate a model of the form P(y=1\mid x)=\pi(x),\,\,y\in\{0,1\},\,\,x\in{\mathbb{R}}^{p}. (1) To motivate the methodology proposed in this article we will initially assume that x=z\in{\mathbb{R}}^{1} is some biomarker able to predict the presence or absence of a condition, coded as y=1 or y=0 . We might reasonably expect response function \pi(z) to be monotone in z and bounded in [0,1] , so there must exist asymptotes: a_{L}=\lim_{z\rightarrow\,\text{-}\infty}\pi(z)\,\,\,\mbox{and}\,\,\,a_{U}=\lim_{z\rightarrow\,\infty}\pi(z). If \pi(z) is increasing, then a_{U} ( a_{L} ) is the prevalence of the condition among subjects with high (low) levels of the biomarker. One problem of importance is the identification of a threshold level z_{thr} , that is, a value near which \pi(z) transitions rapidly between the two asymptotes. If \pi(z) possesses odd symmetry, then z_{thr} can be defined as the point of symmetry, otherwise some canonical definition of z_{thr} can be used. We might also introduce a fourth quantity \alpha to represent the rate of increase of \pi(z) in the neighborhood of z_{thr} . To fix ideas, suppose z is some antibody concentration and y codes an infection event. A pure threshold model is proposed in 3 , given as \pi_{inf}(z)=a_{L}I\{z0 , but this is not required. Following 9 we have an estimate of \rho defined by {\hat{\rho}}=(N_{pair}-d)^{-1}\sum_{g=1}^{n_{g}}\sum_{\left\{i<{i^{\prime}}:g_{i}=g_{i^{\prime}}=g\right\}}{\hat{r}}_{i}{\hat{r}}_{i^{\prime}}, (32) where N_{pair} is the total number of correlated pairs contributing to the sum, and d is the number of estimated parameters. 2.4 Regularization We will show below that there will be some advantage to stabilizing the model fit by using regularization. The ridge regression solution is to minimize \Lambda=SSE+\lambda D, where SSE is the model error sum of squares, \lambda is a nonnegative penalty constant, D is a penalty term given by D=\theta^{T}B\theta, \theta is the p\times 1 parameter column vector, and B is a p\times p positive-definite matrix, which does not depend on \theta or \lambda ( 7 ) . This procedure renders the fit more stable while shrinking the parameter estimates towards zero. For logistic regression with independent responses, SSE is replaced with deviance. For dependent responses, the log-likelihood, and therefore the deviance is not formally defined. In this case, this form of regularization can be implemented directly into the estimating equations ( 20 ), giving G_{\lambda}(\theta)=G(\theta)-\lambda\frac{\partial D}{\partial\theta}=0. (33) For independent responses, the solution to these equations is equivalent to {\hat{\theta}}_{MLE}=\mbox{argmin}_{\theta}\,\text{-}2L(\theta)+\lambda D, but represents the equivalent penalized fitting criterion when the log-likelihood is not defined. Often, the choice B=I_{p} is made, so that D=\sum_{j=1}^{p}\theta_{j}^{2},\mbox{ and }\frac{\partial D}{\partial\theta}=2\theta. (34) The asymptotic distribution of {\hat{\theta}} can be deduced using the same method as that based on Equation ( 22 ), with the first and second order moments of Equation ( 23 ) replaced by \displaystyle E_{\theta^{*}}[G_{\lambda}(\theta^{*})] \displaystyle= \displaystyle\mbox{-}{\lambda}\theta^{*}, \displaystyle E_{\theta^{*}}[G_{\lambda}^{\prime}(\theta^{*})] \displaystyle= \displaystyle\mbox{-}\Omega^{W}_{\theta^{*}}-{\lambda}I_{p}. (35) This gives \displaystyle{\hat{\theta}} \displaystyle\approx \displaystyle\theta^{*}+\left[\Omega^{W}_{\theta^{*}}+{\lambda}I_{p}\right]^{-1}\left[G(\theta^{*})-{\lambda}\theta^{*}\right] (36) \displaystyle= \displaystyle\left[\Omega^{W}_{\theta^{*}}+{\lambda}I_{p}\right]^{-1}\left[\Omega^{W}_{\theta^{*}}\theta^{*}+G(\theta^{*})\right], leading to approximation {\hat{\theta}}\sim N(\theta^{\lambda}_{0},\Sigma^{W,{\lambda}}_{\theta^{*}}) based on the approximate moments \displaystyle\theta^{\lambda}_{0} \displaystyle= \displaystyle E_{\theta^{*}}[{\hat{\theta}}]\approx\left[\Omega^{W}_{\theta^{*}}+{\lambda}I_{p}\right]^{-1}\Omega^{W}_{\theta_{0}}\theta_{0} \displaystyle\Sigma^{W,{\lambda}}_{\theta_{0}} \displaystyle= \displaystyle\mbox{Var}_{\theta_{0}}[{\hat{\theta}}]\approx\left[\Omega^{W}_{\theta_{0}}+{\lambda}I_{p}\right]^{-1}\Omega^{WV}_{\theta_{0}}\left[\Omega^{W}_{\theta_{0}}+{\lambda}I_{p}\right]^{-1}, (37) (essentially this argument is given in 8 ). Inclusion of D induces bias, and usually smaller variance, therefore the choice of {\lambda} represents a bias-variance trade-off. Selection of {\lambda} by cross-validation is discussed in the next section. 2.5 Model selection by cross-validation Cross validation may be used for variable or model selection, and to select the penalty constant {\lambda} defined in ( 34 ). The strategy is to randomly partition the complete data set (of n observations) into K groups (as evenly as possible). Each group separately is taken as test data, then {\hat{\theta}}_{train} is estimated using the training data (the complete data set after deleting the test data). Then for each response y_{test} in the test data, the fitted value {\hat{y}}_{test} is calculated based on estimate {\hat{\theta}}_{train} and paired predictor {\tilde{x}}_{test} . After applying the procedure to the K test groups, each response y_{i} is paired with a cross-validated fitted value {\hat{y}}_{i}^{*} . The final step is to calculate a goodness-of-fit score GOF=GOF(y_{1},\ldots,y_{n};{\hat{y}}_{1}^{*},\ldots,{\hat{y}}_{n}^{*}), which measures collectively the distances between y_{i},{\hat{y}}^{*}_{i} . Possible goodness-of-fit scores include deviance: dev=\sum_{i=1}^{n}y_{i}\log({\hat{y}}^{*}_{i})+(1-y_{i})\log(1-{\hat{y}}^{*}_{i}), area under ROC curve: AUC=\frac{\sum_{i\in I_{0}}\sum_{i^{\prime}\in I_{1}}I\{{\hat{y}}^{*}_{i^{\prime}}>{\hat{y}}^{*}_{i}\}+0.5\times I\{{\hat{y}}^{*}_{i^{\prime}}={\hat{y}}^{*}_{i}\}}{|I_{0}|\times|I_{1}|}, where I_{j} are the indices for which y_{i}=j , j=0,1 , and the error sum of squares defined by the quadratic: SSE=\bar{y}^{T}V_{\theta}^{-1}\bar{y} where \bar{y} is an n\times 1 column vector with i th row equal to y_{i}-y^{*}_{i} , i=1,\ldots,n , and V_{\theta} is estimated using the complete data set. In our implementation, {\lambda} can be varied over a grid spanning {\lambda}=0 to some upper bound \lambda^{max} . For each proposed {\lambda} a model is fit, and evaluated using cross-validation. The choice \lambda^{GOF} is the one optimizing whatever GOF score is used. 3 Compound Logistic Model We next show how the elements \tilde{\pi}(\theta) , \tilde{\tau}(\theta) , \tilde{\pi}^{\prime}_{j}(\theta) and the derived components \Gamma(\tilde{\pi}(\theta)) , V_{\theta} , \theta\in{\mathbb{R}}^{p} , are specialized to the compound logistic model. Suppose each mean binary response element \pi_{i}(\theta) of \tilde{\pi}(\theta) is constructed from m components \pi_{ik}(\theta) , k=1,\ldots,m via compounding function H:{\mathbb{R}}^{m}\rightarrow(0,1) , giving \pi_{i}=H(\pi_{i1},\ldots,\pi_{im}),\,\,\,i=1,\ldots,n. Write the following partial derivatives as: H^{\prime}_{k}(u_{1},\ldots,u_{m})=\frac{\partial H(u_{1},\ldots,u_{m})}{\partial u_{k}},\,\,\,k=1,\ldots,m. Then for convenience denote H^{\prime}_{ik}=H^{\prime}_{ik}(\theta)=H^{\prime}_{k}(\pi_{i1}(\theta),\ldots,\pi_{im}(\theta)), and construct n -dimensional column vectors \tilde{H}^{\prime}_{k}=\tilde{H}^{\prime}_{k}(\theta)=[H^{\prime}_{1k}\ldots H^{\prime}_{nk}] , and \tilde{\pi}_{k}=\tilde{\pi}_{k}(\theta)=[\pi_{1k}\ldots\pi_{nk}] , k=1,\ldots,m . The derivative of response \pi_{i}(\theta) with respect to \theta_{j} is then decomposed into \frac{\partial\pi_{i}(\theta)}{\partial\theta_{j}}=\sum_{k=1}^{m}H^{\prime}_{ik}\frac{\partial\pi_{ik}(\theta)}{\partial\theta_{j}}, (38) from which it follows that \Gamma(\tilde{\pi})=\sum_{k=1}^{m}D_{\tilde{H}^{\prime}_{k}}\times\Gamma(\tilde{\pi}_{k}). (39) Interestingly, the gradient matrix of the estimating equation ( 20 ) for the model \tilde{\pi}(\theta) is simply a weighted sum of the gradient matrices for the models \tilde{\pi}_{k}(\theta) considered separately. 3.1 Application to Logistic Response Functions We can develop ( 39 ) further in the context of logistic regression. Suppose with each of the components we associate an n\times p design matrix X_{k} . Then \pi_{ik}=\phi(\eta_{ik}) , \eta_{ik}=\theta^{T}{\tilde{x}}_{ik} , where {\tilde{x}}_{ik} is the i th row of X_{k} . Then \frac{\partial\pi_{ik}(\theta)}{\partial\theta_{j}}=\frac{\partial\eta_{ik}}{\partial\theta_{j}}\phi(\eta_{ik})(1-\phi(\eta_{ik}))=x_{ijk}\tau_{ik} where \{X_{k}\}_{ij}=x_{ijk} and \tau_{ik}=\pi_{ik}(1-\pi_{ik}) . We may then write \Gamma(\tilde{\pi}_{k})=D_{\tilde{\tau}_{k}}X_{k},\,\,\,k=1,\ldots,m, (40) so that ( 39 ) simplifies to \Gamma(\tilde{\pi})=\sum_{k=1}^{m}D_{\tilde{H}^{\prime}_{k}}D_{\tilde{\tau}_{k}}X_{k}. (41) Note that we have assumed that each component \pi_{ik}(\theta) , k=1,\ldots,m , is a function of a common parameter \theta . In an application it may be more natural to assign distinct parameters to each component. In this case, it would simply be a matter of partitioning the elements of \theta , so that each element \theta_{j} appears in only one component. Otherwise, an element may appear in more than one component. The methodology remains the same in either case. This issue is discussed further below. 3.2 Example (Asymptote Model) We consider the asymptote model of ( 8 ) - ( 9 ), with independent responses. Regarding the estimating equation ( 20 ) and algorithm ( 27 ) we will set W_{\theta}=V_{\theta}=D_{\tilde{\tau}(\theta)} . Suppose the component \phi_{1}(\eta_{1}) incorporates a biomarker z_{i} , and \phi_{2}(\eta_{2}) , \phi_{3}(\eta_{3}) each incorporate a common covariate x_{i} . In addition, each component incorporates an independent intercept term, so we may write \displaystyle\eta_{i1} \displaystyle= \displaystyle\beta_{0,1}+\beta_{1,1}z_{i}, \displaystyle\eta_{i2} \displaystyle= \displaystyle\beta_{0,2}+\beta_{1,2}x_{i}, \displaystyle\eta_{i3} \displaystyle= \displaystyle\beta_{0,3}+\beta_{1,3}x_{i}. (42) The parameter is then \theta=(\beta_{0,1},\beta_{1,1},\beta_{0,2},\beta_{1,2},\beta_{0,3},\beta_{1,3})\in{\mathbb{R}}^{p} , p=6 . The design matrices are then X_{1}=\begin{bmatrix}1&z_{1}&0&0&0&0\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ 1&z_{n}&0&0&0&0\end{bmatrix},\,\,\,X_{2}=\begin{bmatrix}0&0&1&x_{1}&0&0\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ 0&0&1&x_{n}&0&0\end{bmatrix}, X_{3}=\begin{bmatrix}0&0&0&0&1&x_{1}\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ 0&0&0&0&1&x_{n}\end{bmatrix}. The six parameters of \theta are assumed linearly independent. However, by ( 41 ) the gradient matrix \Gamma(\tilde{\pi}) is a weighted sum of X_{0} , X_{1} and X_{2} , which have common column vectors. It is therefore important to verify that \Gamma(\tilde{\pi}) is a full rank matrix (so that \Omega^{W}_{\theta}=\Omega^{V}_{\theta} in algorithm ( 27 ) is invertible). Directly from ( 41 ) we have n\times p matrix \Gamma(\tilde{\pi})=\begin{bmatrix}H^{\prime}_{11}\tau_{11}&H^{\prime}_{11}\tau_{11}z_{1}&H^{\prime}_{12}\tau_{12}&H^{\prime}_{12}\tau_{12}x_{1}&H^{\prime}_{13}\tau_{13}&H^{\prime}_{13}\tau_{13}x_{1}\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ H^{\prime}_{i1}\tau_{i1}&H^{\prime}_{i1}\tau_{i1}z_{i}&H^{\prime}_{i2}\tau_{i2}&H^{\prime}_{i2}\tau_{i2}x_{i}&H^{\prime}_{i3}\tau_{i3}&H^{\prime}_{i3}\tau_{i3}x_{i}\\ \vdots&\vdots&\vdots&\vdots&\vdots&\vdots\\ H^{\prime}_{n1}\tau_{n1}&H^{\prime}_{n1}\tau_{n1}z_{n}&H^{\prime}_{n2}\tau_{n2}&H^{\prime}_{n2}\tau_{n2}x_{n}&H^{\prime}_{n3}\tau_{n3}&H^{\prime}_{n3}\tau_{n3}x_{n}\end{bmatrix}. From Equation ( 9 ) we have partial derivatives \frac{\partial H(u_{1},u_{2},u_{3})}{\partial u_{1}}=u_{3}-u_{2};\,\frac{\partial H(u_{1},u_{2},u_{3})}{\partial u_{2}}=1-u_{1};\,\frac{\partial H(u_{1},u_{2},u_{3})}{\partial u_{3}}=u_{1}. This gives H^{\prime}_{i1}=\pi_{i3}-\pi_{i2};\,H^{\prime}_{i2}=1-\pi_{i1};\,H^{\prime}_{i3}=\pi_{i1}. To simplify the demonstration, we will assume x_{i} is an indicator variable, with x_{i}=1 , i=1,\ldots,m and x_{i}=0 , i=m+1,\ldots,n . We can identify values \pi_{i2}=\pi_{2}^{+},\pi_{2}^{-} and \pi_{i3}=\pi_{3}^{+},\pi_{3}^{-} that hold for x_{i}=1,0 , respectively. Set d^{+}=\pi_{3}^{+}-\pi_{2}^{+} and d^{-}=\pi_{3}^{-}-\pi_{2}^{-} . Also set \tau_{k}^{+}=\pi_{k}^{+}(1-\pi_{1}^{+}) , \tau_{k}^{-}=\pi_{k}^{-}(1-\pi_{k}^{-}) , k=1,2,3 . The quantities \pi_{k}^{+} , \pi_{k}^{-} are in the interval (0,1) , so that \pi_{k}^{+} , \pi_{k}^{-} , \tau_{k}^{+} , \tau_{k}^{-} are all strictly positive. Let \tilde{0} , \tilde{1} be n -dimensional column vectors with elements zero and one, respectively. Finally, a partition of any n -dimensional column vector \tilde{u} into elements 1,\ldots,m and elements m+1,\ldots,n will be denoted (\tilde{u}^{+},\tilde{u}^{-}) . Extend this notational convention to any matrix with n rows. We may then write \Gamma(\tilde{\pi})= \left[\begin{array}[]{cccccc}d^{+}\tilde{\tau}_{1}^{+}&d^{+}(\tilde{\tau}_{1}\circ\tilde{z})^{+}&\tau_{2}^{+}(\tilde{1}\vbox{\hrule width=3.33333pt,height=6.77002pt}\tilde{\pi}_{1}^{+})&\tau_{2}^{+}(\tilde{1}\vbox{\hrule width=3.33333pt,height=6.77002pt}\tilde{\pi}_{1}^{+})&\tau_{3}^{+}\tilde{\pi}_{1}^{+}&\tau_{3}^{+}\tilde{\pi}_{1}^{+}\\ \rule[2.0pt]{25.0pt}{0.5pt}&\rule[2.0pt]{45.0pt}{0.5pt}&\rule[2.0pt]{45.0pt}{0.5pt}&\rule[2.0pt]{45.0pt}{0.5pt}&\rule[2.0pt]{25.0pt}{0.5pt}&\rule[2.0pt]{25.0pt}{0.5pt}\\ d^{-}\tilde{\tau}_{1}^{-}&d^{-}(\tilde{\tau}_{1}\circ\tilde{z})^{-}&\tau_{2}^{-}(\tilde{1}\vbox{\hrule width=3.33333pt,height=6.77002pt}\tilde{\pi}_{1}^{-})&\tilde{0}&\tau_{3}^{-}\tilde{\pi}_{1}^{-}&\tilde{0}\end{array}\right], where \circ is the Hadamard product. Assume m\geq 4 , n-m\geq 4 . Then suppose the vectors B^{+}=(\tilde{1}^{+},\tilde{z}^{+},\tilde{\pi}_{1}^{+}) are linearly independent vectors in {\mathbb{R}}^{m} . Then \tilde{\tau}_{1}^{+}=\tilde{\pi}_{1}^{+}\circ(\tilde{1}^{+}-\tilde{\pi}_{1}^{+}) , and so is a second order interaction with respect to vectors in B^{+} . Similarly, (\tilde{\tau}_{1}\circ\tilde{z})^{+}=\tilde{\pi}_{1}^{+}\circ(\tilde{1}^{+}-\tilde{\pi}_{1}^{+})\circ\tilde{z}^{+} , and so is a third order interaction with respect to vectors in B^{+} . It follows that V^{+}=(\tilde{1}^{+},\tilde{\pi}_{1}^{+},\tilde{\tau}_{1}^{+},\tilde{\tau}_{1}^{+}\circ\tilde{z}^{+}) are linearly independent vectors in {\mathbb{R}}^{m} . We then suppose B^{-}=(\tilde{1}^{-},\tilde{z}^{-},\tilde{\pi}_{1}^{-}) are linearly independent vectors in {\mathbb{R}}^{n-m} , and similarly conclude that V^{-}=(\tilde{1}^{-},\tilde{\pi}_{1}^{-},\tilde{\tau}_{1}^{-},\tilde{\tau}_{1}^{-}\circ\tilde{z}^{-}) are linearly independent vectors in {\mathbb{R}}^{n-m} . Next, let C_{i} , i=1,\ldots,6 , be the column vectors of \Gamma(\tilde{g}) , with partitions (C_{i}^{+},C_{i}^{-}) . Suppose there are real numbers a_{i} for which \sum_{i=1}^{6}a_{i}C_{i}=\tilde{0}. (43) If either d^{+}\neq 0 or d^{-}\neq 0 we must have a_{1}=a_{2}=0 in ( 43 ). Then since C_{4}^{-}=C_{6}^{-}=\tilde{0}^{-} , and C_{3}^{-} , C_{5}^{-} are linearly independent, we must also have a_{3}=a_{5}=0 . Finally, C_{4}^{+} , C_{6}^{+} are linearly independent, so we must have a_{4}=a_{6}=0 . It follows that the column vectors of C_{1},\ldots,C_{6} are linearly independent so that \Gamma(\tilde{\pi}) is full rank. 4 Fitting Methods The methods introduced here are supported in the R -package CLmodel . Fitting options are available as given in Table 1 below. Table 1: Fitting options available in CLmodel R -package. Option Values Description cop.flag TRUE/FALSE Force positive biomarker gradient exchangeable TRUE/FALSE Use exchangeable correlation model quasi.lik TRUE/FALSE Use quasi-likelihood estimation lambda \lambda\geq 0 Regularization parameter ( \lambda=0.0 for no regularization) The exchangeable option is set to FALSE or TRUE according to whether the responses are independent or possess exchangeable correlation, as described in Section 2.3 . In the latter case, the groups must be supplied. For the independent response case, the true variance is V_{\theta}=D_{\tilde{\tau}(\theta^{*})} so set W_{\theta}=D_{\tilde{\tau}(\theta)} in estimating equation ( 20 ), asymptotic distribution ( 24 ) and iterative algorithm ( 27 ). This is equivalent to the MLE. If the option exchangeable = TRUE is selected then the MLE (under the independence assumption) will be calculated (that is, using W_{\theta}=D_{\tilde{\tau}(\theta)} in iterative algorithm ( 27 )). The resulting fit is used to give an estimate of the correlation coefficient \hat{\rho} , as in Equation ( 32 ). This estimate in turn is used to construct the estimated correlation matrix \hat{R}_{\tilde{y}} , and approximate variance matrix V_{\theta}\approx\hat{V}_{\theta}=D_{\tilde{\tau}(\theta)^{1/2}}\hat{R}_{\tilde{y}}D_{\tilde{\tau}(\theta)^{1/2}} . At this point there are two further options. If options exchangeable = TRUE and quasi.lik = FALSE are selected, then the parameter estimates obtained by setting W_{\theta}=D_{\tilde{\tau}(\theta)} in iterative algorithm ( 27 ) are accepted, but the standard errors are obtained by setting V_{\theta}\approx\hat{V}_{\theta} in asymptotic distribution ( 24 ). If options exchangeable = TRUE and quasi.lik = TRUE are selected, then we set W_{\theta}=\hat{V}_{\theta} in iterative algorithm ( 27 ) and recalculate the parameter estimates. Then W_{\theta}=V_{\theta}=\hat{V}_{\theta} is used in asymptotic distribution ( 24 ) to derive standard errors. If the option lambda = 0.0 no regularization is used. Otherwise iterative algorithm ( 27 ) and asymptotic distribution ( 24 ) are replaced with ( 36 ) and ( 37 ), respectively, with \lambda equal to the value specified in the option. 4.1 Transformation of biomarker gradient Recall the discussion regarding identifiability of Section 3.2 . It was shown that a necessary condition for identifiability is that the gradient associated with the designated biomarker is not zero. Suppose in the asymptote model of Section 3.2 we have \eta_{1}=\beta_{0,1}+\beta_{1,1}x_{1} , and that x_{1} is the designated biomarker. In most applications the direction of the effect of x_{1} on the response (equivalently, the sign of the \beta_{1,1} ) would be known, or could be confidently inferred without resort to a formal estimate of \beta_{1,1} . In this case, we can assume \beta_{1,1}>0 , reversing the sign of x_{1} if needed. While it is possible to build such a constraint into an optimizer, we opt instead to reparametrize \eta_{1}=\beta_{0,1}+\beta_{1,1}x_{1} to \eta_{1}=e^{\xi}(x_{1}-\beta^{*}_{0,1}) , \xi\in(-\infty,\infty) . If additional covariates x_{2},\ldots,x_{q} are used in this component we can set \eta_{1}=e^{\xi}(x_{1}-\eta_{1}^{*}) , where \eta_{1}^{*}=\beta^{*}_{0,1}+\beta^{*}_{2,1}x_{2}+\ldots+\beta^{*}_{q,1}x_{q} . This would change the method of calculating \Gamma(\tilde{\pi}) in estimating equation ( 20 ), but the other components of the iterative algorithm and the asymptotic distribution would remain unchanged following that revision. The overall approach remains mathematically validated by Theorem 13.1 of 1 . 4.2 Initial solution The discussion regarding identifiability in Section 3.2 makes clear that using a good initial solution for iterative algorithm ( 27 ) or ( 36 ) is important. In particular, we cannot start with a null model defined by a zero biomarker gradient. The approach advocated here is to first fit the four parameter model based on ( 7 ), with single biomarker covariate x and binary response y , setting \eta=\alpha(x-x_{thr}) . This yields initial estimate \hat{\theta}^{*}_{0}=(\hat{\alpha},\hat{x}_{thr},\hat{a}_{L},\hat{a}_{U}) . These parameters can then be transformed to the correct model parametrization. 5 Simulation Study In this section we use a model simulation to assess the accuracy of the proposed methodology. Consider the model of Section 3.2 , based on compounding function ( 8 ) - ( 9 ). We take the prediction terms of Equation ( 42 ), where z_{i} is the biomarker, and x_{i} is some binary covariate. Following Section 4.1 , we set \eta_{i1}=\exp(\alpha)(z_{i}-\beta_{0,1}) , while \eta_{i2} , \eta_{i3} remain as they are in ( 42 ). We set the total number of responses n=200 . In addition, we will impose an exchangeable correlation model with \rho=0.1 , with observations randomly partitioned into groups of 5. The biomarker z_{i} is uniformly simulated from interval [-1,1] . Each x_{i} is randomly set to one with probability 0.5. Then set true parameter \displaystyle\theta_{true} \displaystyle= \displaystyle(\alpha,\beta_{0,1},\beta_{0,2},\beta_{1,2},\beta_{0,3},\beta_{1,3}) \displaystyle= \displaystyle(3,0,\log(0.25/0.75),0,\log(0.75/0.25),-2). The response curve is shown in Figure 2 . N_{sim}=50,000 response vectors were simulated (the same covariates z_{i} , x_{i} were used for each replication). Fit options were set to cop.flag = TRUE , exchangeable = TRUE and quasi.lik = TRUE (Table 1 ). Using common replications, N_{sim} fitted models were calculated with \lambda=0.0 (not regularized) and then with \lambda=0.1 (regularized). Figure 2: Response curve for simulated model of Section 5 . That ridge regression improves the stability of the numerical algorithm is clearly seen, in that 55.6% of fits failed to converge for {\lambda}=0 , while all converged for {\lambda}=0.1 . Table 2 gives numerical summaries for parameter estimates {\hat{\theta}}_{k} and reported standard errors S_{{\hat{\theta}}_{k}} for each parameter. We first note that the high convergence failure rate for the {\lambda}=0 case appears to have a number of effects on the distribution of the estimates and their standard errors. The highest quantile of the standard errors, Q_{0.99} , is uniformly larger for the {\lambda}=0 case, especially for \alpha and \beta_{0,1} , suggesting that regularization yields great stability and robustness. It should also be noted that the standard deviations of the estimates for the {\lambda}=0 case are uniformly smaller than for the {\lambda}=0.1 case. This suggests that more divergent fits are being truncated by convergence failure. Figures 3 - 4 give histograms of Z -scores Z=\frac{{\hat{\theta}}_{j}-\theta_{j}}{S_{{\hat{\theta}}_{j}}} for parameters j=1,\ldots,5 . Values of |Z|>5 were omitted from the histograms (frequencies are reported in Table 2 ). Skewness is evident in most distributions. Nontheless, the actual coverage of a 95% confidence interval based on the asymptotic normal distribution is close to 0.95 in all cases except for \beta_{0,1} for the {\lambda}=0 case. The estimates of \rho=0.1 were reasonably accurate, with \hat{\rho}=0.087\pm 0.063 for {\lambda}=0 and \hat{\rho}=0.090\pm 0.062 for {\lambda}=0.1 . Ridge regression introduces bias, so that the estimated model will depend on the degree of penalization. However, from Table 2 it can be seen that the parameter estimates for the two choices of {\lambda} are comparable. Table 2: Summary of simulation study of Section 5 . Penalty constants {\lambda}=0,0.1 were both used. Sample means and standard deviations of each parameter estimate {\hat{\theta}}_{j} are given. Various quantiles of the standard errors S_{{\hat{\theta}}_{j}} are also given. The entry “Cover.” gives the estimated actual coverage of a nominal 95% confidence interval based on the asymptotic normal distribution. The entry \%|Z|>5 gives the percentage of estimates which are further than 5 standard errors from the true parameter value. \alpha \beta_{0,1} \beta_{0,2} \beta_{1,2} \beta_{0,3} \beta_{1,3} {\lambda}=0 {\hat{\theta}}_{j} True value 3.000 0.000 -1.099 0.000 1.099 -2.000 Mean 3.107 0.001 -1.528 0.170 1.093 -2.212 SD 1.088 0.144 4.345 1.173 4.644 1.336 S_{{\hat{\theta}}_{j}} Q_{0.01} 0.581 0.013 0.295 0.414 0.293 0.377 Q_{0.25} 0.944 0.049 0.356 0.489 0.372 0.458 Q_{0.5} 1.239 0.074 0.393 0.529 0.419 0.499 Q_{0.75} 1.706 0.109 0.458 0.594 0.505 0.574 Q_{0.99} >10^{6} >10^{3} 3.806 3.957 4.834 4.805 Cover. 0.973 0.857 0.968 0.957 0.972 0.961 \%|Z|>5 0.000 2.159 0.000 0.000 0.000 0.000 {\lambda}=0.1 {\hat{\theta}}_{j} True value 3.000 0.000 -1.099 0.000 1.099 -2.000 Mean 2.478 -0.052 -1.271 0.101 0.976 -1.834 SD 5.439 3.445 8.060 8.408 8.902 7.916 S_{{\hat{\theta}}_{j}} Q_{0.01} 0.582 0.029 0.305 0.418 0.299 0.371 Q_{0.25} 0.860 0.077 0.362 0.490 0.367 0.444 Q_{0.5} 1.009 0.107 0.401 0.530 0.407 0.479 Q_{0.75} 1.174 0.150 0.478 0.599 0.476 0.538 Q_{0.99} 1.813 0.432 1.075 1.048 0.929 0.896 Cover. 0.950 0.951 0.974 0.965 0.971 0.961 \%|Z|>5 0.834 0.880 0.546 0.522 0.518 0.552 Figure 3: Histograms of parameter estimate Z -scores ({\hat{\theta}}_{j}-\theta^{*}_{j})/S_{{\hat{\theta}}_{j}} , from simulation study of Section 5 ( {\lambda}=0 ). Values |Z|>5 are omitted. Figure 4: Histograms of parameter estimate Z -scores ({\hat{\theta}}_{j}-\theta^{*}_{j})/S_{{\hat{\theta}}_{j}} , from simulation study of Section 5 ( {\lambda}=0.1 ). Values |Z|>5 are omitted. We finally note the difference in response between observations coded differently by indicator variable x_{i} evident in Figure 2 . The difference is attributable to the parameter \beta_{1,3}=-2 . The power for a two-sided alternative to null hypothesis H_{0}:\beta_{1,3}=0 at significance level \alpha=0.05 was estimated to be 0.97, 0.98 for \lambda=0.0,0.1 . 6 Example The data used in this example were collected as part of the Midlife in the United States (MIDUS) project. This project originated in 1995 from a national survey by the MacArthur Foundation Research Network on Successful Midlife Development, which was extended into a longitudinal study in 2002 at the University of Wisconsin-Madison through a grant from the National Institute on Aging. The data include interview and questionnaire responses, biomarker measurements, and cognitive assessments of middle-aged adults in the United States, with the goal of identifying the relationships between physical, psychological, and social factors of well-being in adulthood. The collection of survey data for MIDUS 3, the most recent phase of the project and the basis of this paper’s data, was conducted between 2013 and 2014, while the biomarker data were collected between 2017 and 2022. The results of the study have been made available through the Inter-University Consortium of Political and Social Research (ICPSR). Suppose we are given binary response y_{i} which codes a subject with diabetes. Then x_{i} is a binary covariate which codes a subject with asthma. We are then given a biomarker z_{i} representing hemoglobin A1c%, commonly used as a diagnostic biomarker for diabetes. We use the model of Section 3.2 , based on compounding function ( 8 ) - ( 9 ). Set \eta_{i1}=\beta_{0,1}+\beta_{1,1}z_{i} , \eta_{i2}=\beta_{0,2}+\beta_{1,2}x_{i} and \eta_{i3}=\beta_{0,3}+\beta_{1,3}x_{i} . Using the CLmodel R -package with exchangeable = FALSE and lambda = 0 (that is, the maximum likelihood estimate) we obtain the following output coefficient table: Est SE Z p-val (Z) b[1,1] -31.019268 5.9930032 -5.1759138 2.267982e-07 b[1,2] 4.893016 0.9797937 4.9939246 5.916451e-07 b[2,1] -4.689845 0.9747431 -4.8113650 1.499030e-06 b[2,2] 0.758998 1.3542669 0.5604493 5.751730e-01 b[3,1] 2.463809 0.5539285 4.4478823 8.672101e-06 b[3,2] -2.210098 0.9207106 -2.4004262 1.637599e-02 Figures 5 and 6 present two views of the resulting fitted model. Figure 5 shows the estimated response curve \hat{\pi}(z,x) . Separate curves \hat{\pi}(z,0) , \hat{\pi}(z,1) are plotted as a function of z (hemoglobin A1c%). In addition, kernel smoothers are superimposed, calculated separately for the x=0 , x=1 cases. Finally, separate rug plots show the location and value of the individual responses. There are n=632 observations, of which n_{1}=89 are coded as asthma positive. As expected, for small values of z (an after-fasting measure of blood glucose levels) the response function (the probability of diabetes) is near zero, while for larger values of z the response function approaches separate asymptotes for the x=0 , x=1 cases. The difference in upper asymptotes is entirely attributable to the parameter \beta_{1,3} , here estimated to be \hat{\beta}_{1,3}=-2.21 (note the differing coefficient indexing conventions). From the table, \beta_{1,3} differs significantly from zero with a p -value of 0.017. We conclude that the probability of diabetes at high levels of the biomarker is significantly lower for subjects with asthma (OR = 0.11; 95% CI = (0.02, 0.69)). For the x=0 (no asthma) group, the kernel smoother suggests that the shape of the response curve is appropriate for the data, beginning with a near zero lower asymptote, followed by a rapid increase around z=6\% , ending with an upper asymptote of approximately 0.92 (with a 95% upper confidence bound of 0.97). The rug plots indicates that data for the x=1 (asthma) group is very sparse above z=6\% . However, the kernel smoother similarly conforms to the estimated response curve for that group also. Figure 6 shows the same fitted curves, and includes 95\% confidence intervals (CI) for the upper asymptotes for the x=0 and x=1 groups. The two CIs overlap, but because the model allows a single parameter \beta_{1,3} to be responsible for any difference in the upper asymptotes it becomes straightforward to detect. We next note that hemoglobin A1c% is commonly used as a biomarker for diabetes mellitus, with A1c% \geq 6.5\% indicating diabetes and 5.7\%\geq A1c% <6.5\% indicating prediabetes according to guidelines of the American Diabetes Association (ADA) ( 10 ; 15 ) . These thresholds are superimposed in the response curve in Figure 6 . Interestingly, most of the rapid increase in the response curves occurs in the prediabetes range of 5.7\%-6.5\% , while the upper asymptote is essentially reached just above the 6.5\% threshold. Thus, the estimated model conforms very closely to the ADA guidelines. Finally, we note that the estimated model predicts a very high false positive rate for asthma positive subjects. That an association exists among hyperglycemia (and therefore elevated hemoglobin A1c%) and asthma has been consistently reported in the literature ( 14 ; 18 ; 12 ; 17 ) . This association is complex, and is described as “bidirectional” in 12 . That the use of inhaled corticosteroids to control asthma and chronic obstructive pulmonary disease (COPD) may be associated with the induction and worsening of diabetes has also been reported ( 14 ; 12 ) . It seems possible, therefore, that this association may be present in subjects not diagnosed with diabetes ( 14 ; 18 ) to a significant degree, leading to the higher false positive rate among asthma subjects reported above (note that diagnoses in the MIDUS 3 study are self-reported). Figure 5: Fitted response curves for selected COP model, Section 6 Figure 6: Fitted response curves for selected COP model, Section 6 7 Discussion In this article a collection of statistical methods extending the double-asymptotic binary response model was proposed. These models may have correlated responses, and can include additional covariates which potentially modify the components of the model, including the lower and upper asymptote. This is achieved through what we define as the compound logistic regression model , in which the mean response is a differential function of several logistic regression functions. This permits a greater variety of models, while retaining the advantages of standard logistic regression. The collection of methods is implemented in the R -package CLmodel ( https://github.com/ALMUDEVAR163/CLmodel/ ). Additional model features are demonstrated in the vignette.