Nonparametric Least squares estimators for interval censoring
Piet Groeneboom · 2025
Read Nonparametric Least squares estimators for interval censoring by Piet Groeneboom (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01103.txt.
Introduction
Piet Groeneboom studies estimation from interval censored data, where each subject's event time is known only to fall inside an interval between observation times, a common pattern in medical follow up studies. With several observation times per subject, the limit distribution of the nonparametric maximum likelihood estimator is still unknown in general.
Groeneboom proves consistency for a related isotonic least squares estimator and sketches its limit distribution. Simulations at sample sizes of one thousand and ten thousand show that the conjectured limit behavior is not yet clearly visible even there, a caution about relying on asymptotics. He also compares a simpler one step least squares estimator, which is easier to compute but uses less information and performs worse.
Catalog record
| CREATOR | Piet Groeneboom |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01103
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Nonparametric Least squares estimators for interval censoring Piet Groeneboom Abstract The limit distribution of the nonparametric maximum likelihood estimator for interval censored data with more than one observation time per unobservable observation, is still unknown in general. For the so-called separated case, where one has observation times which are at a distance larger than a fixed ϵ > 0 \epsilon>0 , the limit distribution was derived in [ 5 ] . For the non-separated case there is a conjectured limit distribution, given in [ 4 ] , Section 5.2 of Part 2. But the findings of the present paper suggest that this conjecture may not hold. We prove consistency of a closely related nonparametric isotonic least squares estimator and give a sketch of the proof for a result on its limit distribution. We also provide simulation results to show how the nonparametric MLE and least squares estimator behave in comparison. Moreover, we discuss a simpler least squares estimator that can be computed in one step, but is inferior to the other least squares estimator, since it does not use all information. For the simplest model of interval censoring, the current status model, the nonparametric maximum likelihood and least squares estimators are the same. This equivalence breaks down if there are more observation times per unobservable observation. The computations for the simulation of the more complicated interval censoring model were performed by using the iterative convex minorant algorithm. They are provided in the GitHub repository [ 8 ] . Delft Institute of Applied Mathematics, Mekelweg 4, 2628 CD Delft, The Netherlands. ?? 1 Introduction The simplest and most studied interval censoring model is the so-called interval censoring, case 1, or current status model. This model can be defined in the following way (see, e.g., Section 2.3 of [ 6 ] ). Consider a sample X_{1},X_{2},\ldots,X_{n} , drawn from a distribution with distribution function F_{0} . Instead of observing the X_{i} ’s, one only observes for each i whether or not X_{i}\leq T_{i} for some random T_{i} (independent of the other T_{j} ’s and all X_{j} ’s). More formally, instead of observing the X_{i} ’s, one observes (T_{i},\Delta_{i})=(T_{i},1_{\{X_{i}\leq T_{i}\}}). (1.1) One could say that the i -th observation represents the current status of item i at time T_{i} . We want to estimate the unknown distribution function F_{0} based on the data given in ( 1.1 ). For this problem the log likelihood function for distribution functions F (conditional on the T_{i} ’s) is given by \ell(F)=\sum_{i=1}^{n}\left(\Delta_{i}\log F(T_{i})+(1-\Delta_{i})\log(1-F(T_{i}))\right). (1.2) The (nonparametric) maximum likelihood estimator \hat{F}_{n} maximizes \ell over the class of all distribution functions. Since distribution functions are by definition nondecreasing, computing the maximum likelihood estimator poses a shape constrained optimization problem. As can be seen from ( 1.2 ), the value of \ell only depends on the values that F takes at the observed time points T_{i} . Hence one can choose to consider only distribution functions that are constant between successive observed time points T_{i} . The lemma below shows that this estimator can be characterized in terms of a greatest convex minorant of a certain diagram of points. The following result is Lemma 2.7 in [ 6 ] (and Proposition 1.2 in [ 4 ] ). Lemma 1 . [Characterization of the nonparametric ML estimator in the current status model] Consider the cumulative sum diagram consisting of the points P_{0}=(0,0) and P_{i}=\left(i,\sum_{j=1}^{i}\Delta_{j}\right),\,\,\,1\leq i\leq n, where the \Delta_{i} ’s correspond to the T_{i} ’s, which are supposed to be ordered 0<T_{1}\dots<T_{n} (one can also allow ties, but we disregard this further complication here). Then the nonparametric MLE \hat{F}_{n}(T_{i}) is given by the left derivative of the convex minorant of this diagram of points, evaluated at the point i . This maximizer is unique. Remark 1 . The left derivative of the convex minorant at P_{i} determines the value of \hat{F}_{n} at T_{i} and hence (by right continuity of the distribution function \hat{F}_{n} ) on [T_{i},T_{i+1}) , a region to the right of T_{i} . Lemma 1 shows that \hat{F}_{n} is in fact the isotonic regression on the indicators \Delta_{i}=1_{\{X_{i}T_{i}\}} , that is, it minimizes \displaystyle\sum_{i=1}^{n}\{F(T_{i})-\Delta_{i}\}^{2} (1.3) over all monotone nondecreasing (not necessarily bounded by 0 and 1 ) functions F . so we have the two characterizations of the nonparametric maximum likelihood estimator, the characterization as a maximizer of ( 1.2 ) and the characterization as a minimizer of ( 1.3 ) (see, e.g., [ 4 ] ). Alternatively, we can derive the equivalence of these formulations from Theorem 1.5.1 in [ 9 ] . If we turn to the (common) situation where there are more observation times per unobservable X_{i} , the situation is considerably more complicated, and the limit distribution of the nonparametric MLE is still unknown generally. We consider here the simplest extension, where one has two observation times per unobservable X_{i} ( interval censoring, case 2 ). Instead instead of observing the X_{i} ’s, one observes (U_{i},V_{i},\Delta_{i0},\Delta_{i1})\stackrel{{\scriptstyle\text{def}}}{{=}}(U_{i},V_{i},1_{\{X_{i}\leq U_{i}\}},1_{\{U_{i}<X_{i}\leq V_{i}\}}), (1.4) so instead of only observing that X_{i} is larger than or smaller than an observation T_{i} , we now have an observation interval (U_{i},V_{i}) , U_{i}<V_{i} , and we know whether our unobservable X_{i} is inside the interval or to the left or right of it. The log likelihood ( 1.2 ) changes into \displaystyle\ell(F)=\sum_{i=1}^{n}\left\{\Delta_{i0}\log F(U_{i})+\Delta_{i1}\log(F(V_{i})-F(U_{i}))+\Delta_{i2}\log(1-F(V_{i}))\right\}, (1.5) where \Delta_{i2}=1-\Delta_{i0}-\Delta_{i1} . It is not clear that there is an equivalent to the minimization of ( 1.3 ) in this situation. We cannot use Theorem 1.5.1 in [ 9 ] anymore because we have the difference of F(V_{i}) and F(U_{i}) in the terms \log(F(V_{i})-F(U_{i})) instead of just F(U_{i}) or F(V_{i}) by itself only. The perhaps most natural least squares approach is to consider minimization of \displaystyle\sum_{i=1}^{n}\left\{\left\{F(U_{i})-\Delta_{i0}\right\}^{2}+\left\{F(V_{i})-F(U_{i})-\Delta_{i1}\right\}^{2}+\left\{1-F(V_{i})-\Delta_{i2}\right\}^{2}\right\} (1.6) over all distribution functions F , so an isotonic regression on the indicators \Delta_{i0},\Delta_{i1},\Delta_{i2} , i=1,\dots,n . As an example, we analyze the behavior of the nonparametric MLE, maximizing ( 1.5 ) and the least squares estimator minimizing ( 1.6 ) in Example 1 . (a) (b) Fig 1: (a) Nonparametric MLE (blue) of F_{0} for a sample of size n=1000 , (b) Nonparametric least squares estimate (blue) minimizing ( 1.6 ) of F_{0} for the same sample. The solid black curve shows F_{0} . Example 1 . Suppose X_{1},\dots,X_{n} is either a sample from the truncated exponential distribution on [0,2] , with density \displaystyle f_{0}(x)=\frac{\exp{-x}}{1-\exp\{-2\}}1_{[0,2]}(x),\qquad x\in{\mathbb{R}}, or a sample from the Uniform distribution on [0,2] , and let the (U_{i},V_{i}),\,V_{i}>U_{i} , be the order statistics of the Uniform distribution on [0,2]^{2} for a sample of size 2. Note that this is a prototype of the non-separated case, where we can have arbitrarily small observation intervals [U_{i},V_{i}] (a) (b) Fig 2: (a) Simulated variances, times n^{2/3} , of the nonparametric MLE (black solid curve) and the least squares estimate (red) minimizing ( 1.6 ), for t_{i}=0.1,0.2,\dots,1.9 , linearly interpolated between values at the t_{i} for the model of Example 1 . The blue dashed curve is the theoretical limit curve one obtains from Theorem 1 in Section 2 . The simulated variances are based on 10,000 simulations of samples of size n=1000 for the truncated exponential distribution function F_{0} on [0,2] and the order statistics of the uniform distribution on [0,2]^{2}