Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks

Aoran Zhang, Tianyao Wei, Maria J. Guerrero, César A. Uribe · 2026

Read Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks by Aoran Zhang, Tianyao Wei, Maria J. Guerrero, César A. Uribe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18820.txt.

Introduction

Zhang and three colleagues propose a method for sparse network inference under imperfect detection. Old models focus on interaction recovery, while similarity graphs are less studied. The method uses low rank factorization with detection estimated from data. It keeps estimated values sparse with better relative scale, and an ADMM algorithm solves the problem. Tests on synthetic and real data show better recovery.

Catalog record

CREATORAoran Zhang, Tianyao Wei, Maria J. Guerrero, César A. Uribe
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76035
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18820

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks Aoran Zhang, Tianyao Wei, Maria J. Guerrero, César A. Uribe Abstract Recovering latent structure from count data has received considerable attention in network inference, particularly when one seeks both cross-group interactions and within-group similarity patterns in bipartite networks, which is widely used in ecology research. Such networks are often sparse and inherently imperfect in their detection. Existing models mainly focus on interaction recovery, while the induced similarity graphs are much less studied. Moreover, sparsity is often not controlled, and scale is unbalanced, leading to oversparse or poorly rescaled estimates with degrading structural recovery. To address these issues, we propose a framework for structured sparse nonnegative low-rank factorization with detection probability estimation. We impose nonconvex ℓ 1 / 2 \ell_{1/2} regularization on the latent similarity and connectivity structures to promote sparsity within-group similarity and cross-group connectivity with better relative scale. The resulting optimization problem is nonconvex and nonsmooth. To solve it, we develop an ADMM-based algorithm with adaptive penalization and scale-aware initialization and establish its asymptotic feasibility and KKT stationarity of cluster points under mild regularity conditions. Experiments on synthetic and real-world ecological datasets demonstrate improved recovery of latent factors and similarity/connectivity structure relative to existing baselines. I Introduction Latent recovery of connectivity structure from biased and partial observations is a recurring problem in signal processing and network inference [ 10 , 35 ] . In many scenarios, the object of interest is a bipartite interaction matrix with entries encoding frequencies, affinities, or event intensities between two classes of entities. When the data are observed through noisy counting processes, inference goes beyond standard matrix completion tasks as one must jointly account for latent low-rank structure, distorted measurements, and potential structural constraints of the recovered network. This setting is inherent in sensing and monitoring applications [ 41 , 36 ] , where observations, such as counts, are obtained via an imperfect sampling process. In this paper, we are interested in ecological interaction networks describing how species associate with locations and how environments shape biodiversity patterns [ 9 , 44 ] . These systems can naturally be represented as bipartite graphs where one node set corresponds to species and the other to geographic areas [ 19 , 11 ] . The intensity of interaction is recorded as the frequency of observations in field surveys. However, observations are limited by imperfect detection: even when a species is present at one location, its presence may go unrecorded due to factors such as limited sampling times, occlusions, or misidentification. As a result, the observed count matrix provides noisy information about the latent network [ 28 ] . The goal here is to denoise the measurements and, at the same time, recover interpretable structures: cross-group connectivity between locations and species, as well as within-group similarity patterns among locations and among species. This naturally motivates a structured latent-factor model under imperfect observations. Imperfect detection must be modeled explicitly, or otherwise, zeros and small counts may be incorrectly interpreted as weak or absent interactions rather than missing recordings. Prior work addressed this issue by modeling latent interaction intensity through nonnegative matrix factorization and estimating detection probabilities separately in an alternating scheme [ 16 ] . While this provides an important starting point, it does not fully exploit an additional structural prior that is central in many networked systems: sparsity in the induced within-group and cross-group relations. In many bipartite network recovery problems, the quantities of interest are not limited to the cross-group interaction matrix itself. The similarity structures within each group can be equally important for interpretation and downstream analysis. In ecological applications, for example, it is expected that only a subset of latent relationships will be active. This reflects selective co-occurrence, habitat specialization, or structured overlap in detectability. For instance, monitoring sites within the same habitat type may exhibit similar acoustic community compositions due to shared environmental conditions. However, sites separated by land-use barriers or fragmentation may display significantly different species assemblages, even when they share the same cover classification [ 19 ] . This motivates us to impose regularization on structured sparsity on both cross-group connectivity and within-group similarity matrices. The resulting estimation problem is difficult for several reasons: a) the parameterization of the latent intensity is bilinear with nonconvex likelihood; b) the structured sparsity penalties with non-negative constraints introduce non-smooth and nonconvex regularization; c) the regularizers consist of quadratic functions of factors, creating coupling across variables and ruling out direct applications of standard alternating minimization methods that are developed for separable matrix factorization models. As a result, the problem is a nonconvex, nonsmooth, and block-structured optimization problem with nonlinear, nonseparable constraints. In this work, we formulate latent connectivity recovery under imperfect detection as a structured, sparse, nonnegative, low-rank estimation problem. The contributions of this paper are as follows: • We propose a structured latent-factor model for bipartite count data under imperfect detection that jointly captures cross-group interaction intensity, within-group similarity, and feature-dependent detectability. • We introduce nonconvex \ell_{1/2} regularization on the induced similarity and connectivity matrices, leading to a nonlinear, nonsmooth, and nonseparable split formulation. • We develop a structure-exploiting ADMM-style augmented-Lagrangian scheme for this split problem. The specific variable splitting turns the nonsmooth regularizers into exact half-thresholding updates for the auxiliary blocks, while the factor blocks are handled by projected gradient steps under nonnegativity constraints, and the detectability block remains convex. • We provide a model-specific convergence analysis for the proposed split formulation. Under bounded iterates and vanishing first-order residuals of the inner factor updates, we establish asymptotic feasibility and stationarity of cluster points for the split problem. • We demonstrate the recovery performance on large-scale synthetic and real-world ecological datasets, showing that the proposed structured formulation can improve the recovery of latent connectivity and similarity patterns relative to baselines that do not impose a structured sparsity prior. The rest of this paper is organized as follows: Section II reviews related work. Section III introduces the proposed model and problem formulation. Section IV presents the optimization algorithm. Section V gives the convergence analysis. Section VI reports experimental results on synthetic and real datasets. II Related Work N-mixture Model Ecological interaction networks can be represented as bipartite networks in which interactions occur across two groups, e.g., plants and pollinators [ 27 , 2 ] , hosts and parasites [ 8 ] , or soundscape signals and locations [ 19 ] . In such systems, observed interaction counts arise from both underlying ecological processes and detection mechanisms, providing an indirect view of the latent interaction structure. This distinction is well established in ecological modeling, where hierarchical approaches such as N-mixture models explicitly separate latent ecological processes from the stochastic detection effects. [ 40 , 29 , 34 , 30 ] . A recent work combined the N-mixture framework with link prediction methods to extend imperfect-detection models to ecological interaction networks [ 16 ] . For link prediction problems, interaction intensity is often modeled via low-rank matrix factorization, where the interaction strength across groups is parameterized through latent embeddings [ 33 , 31 , 7 ] . In this representation, the entries of the interaction matrix are modeled as the inner product of the two latent

Read the complete text