TestNet: a method for inferring microbial networks with false discovery rate control for clustered and unclustered samples Authors: Chang Su, Yicong Mao, Mengyu He, Vanessa E. Van Doren, Colleen F. Kelley, Yi-Juan Hu Published: 2026 ABSTRACT Most existing methods for inferring microbial networks generate only point estimates of Pearson’s correlations without assessing their significance, and none accounts for clustering. We introduce TestNet, a novel method that delivers well-calibrated results by controlling the false discovery rate (FDR). TestNet uses a permutation-based procedure to generate valid null replicates that account for compositional effects, excess zeros in microbiome data, and clustering within samples when present. Our results demonstrate that TestNet is the only evaluated method that effectively controls the FDR while maintaining high power across a wide range of scenarios.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13059-026-04103-0. Background The importance of the human microbiome in human health and diseases has increasingly been recognized. Inference of microbial networks is an important problem in microbiome research, as it reveals interdependencies or interactions among microbial taxa within communities. These interactions include mutualism where taxa help or benefit from each other, competition where they compete for resources, commensalism where one benefits while the others are not affected, and parasitism where one benefits at the expense of another. The constructed microbial networks can subsequently be used to identify keystone taxa, which are highly connected with other taxa and significantly influence microbiome structures and functions. Keystone taxa can be manipulated to treat inflammation or restore community stability [1]. Additionally, microbial networks can be used to infer ecological ‘guilds’, which are groups of taxa with co-abundance behaviors and similar functions. Guilds can serve as units for microbiome data reduction or microbial association analysis with disease outcomes [2]. In this article, we use the term ‘dependence’ broadly to refer to any microbial relationship, including both ‘causality’ and ‘association’. While absolute abundances of individual taxa are of interest, these values are unobservable in sequencing-based experiments. Techniques such as 16S marker-gene sequencing or shotgun metagenomic sequencing produce taxon count data, where the total counts per sample are considered experimental artifacts. Consequently, these experiments provide information solely about taxon relative abundances, making the data compositional since relative abundances sum to one. Because a change in the relative abundance of one taxon necessarily implies a counterbalancing change in other taxa (i.e., compositional effects), standard correlation analysis of the relative abundances tends to yield negative correlations even if the underlying absolute abundances are independent. The analysis of compositional data typically involves some form of log-ratio transformation of the relative abundances, such as the centered-log-ratio (clr) transformation [3, 4]. Log-ratio transformation effectively removes the compositional effects (as the ratio of relative abundances of two taxa is independent of other taxa), and the transformed relative abundances have been used to make inference about absolute abundances in association studies [5, 6]. In addition, taxon count data exhibit several other complex features. These data are well known to be sparse, with 50–90% zeros. To enable log-ratio transformation, a common practice is to add a pseudo-count, typically 0.5 or 1, to the zeros or all entries of the taxa count table [3, 5]. However, the impact of this strategy is often left unevaluated. These data are high-dimensional, with 100–10000 taxa, and it is generally accepted that the dependence structures among the large number of microbial taxa within a community are sparse [7–10]. These data are highly overdispersed due to not only sample heterogeneity but also technical variation in the sequencing workflow, which includes steps such as DNA extraction, PCR amplification, amplicon sequencing, and taxonomy assignment. It is impossible to find a single parametric model that fits every taxon well. These data are also subject to experimental bias because each step in the aforementioned sequencing workflow preferentially measures some taxa over others [11]. Furthermore, microbial taxa within a community have complex interdependencies, and some relationships may be nonlinear [10]. It has become increasingly common to collect repeated (e.g., longitudinal) measures of the microbiome from the same subject [12, 13]. Not only do measures across subjects provide information about taxon interdependencies, but measures within subjects particularly facilitate this inference, as the large subject heterogeneity as well as all confounding factors between subjects are eliminated in within-subject comparisons. There are generally two approaches to inferring microbial networks. One approach is based on the marginal dependence between each pair of taxa, independent of all other taxa (e.g., SparCC [7]). The other approach is based on the conditional dependence between each pair of taxa while controlling for all other taxa (e.g., SpiecEasi [14]). While the first approach is unable to differentiate between direct and indirect dependencies, the second one requires more assumptions, such as the normality of data and the relationships among all taxa, and is more computationally complex. As pointed out in a recent review paper [15], the normality assumption of the log- or log-ratio transformed microbiome data does not always hold. Here, we focus on the initial scan of taxon-taxon relationships based on marginal dependence and aim to provide robust inference in the context of complex microbiome data. More precise microbial relationships can be delineated using more complex models after sub-networks are identified and the data dimension is reduced. Existing methods for inferring marginal dependencies primarily focus on estimating the basis correlations, specifically Pearson’s correlations of the underlying (log) absolute abundances. Then, they impose sparsity in dependencies through thresholding or penalization to identify only a few significant dependent pairs. SparCC [7] solves for the basis correlations using an iterative algorithm and then declares dependencies with a threshold of 0.3; the resulting correlations are claimed to be similar to the sample correlations of clr-transformed data when the number of taxa is large. CCLasso [8] estimates the basis covariances (and then the basis correlations) by minimizing the weighted least squares loss between the sample covariance matrix of clr data and the basis covariance matrix, combined with the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_1$$\end{document}L1 penalty to explicitly exploit the sparse dependence assumption. COAT [9] estimates the basis covariances by directly thresholding the sample covariances of clr data, while providing theoretical justification. These methods ignore the influence of sequencing variation (i.e., errors throughout the sequencing workflow) and the impact of their treatments of zeros, which may cause their estimators to be inconsistent. In fact, their sensitivity to detecting true correlations may decrease as the sample size increases. As these methods are designed for parameter estimation, they do not yield p-values or provide error rate control, and the resulting findings are therefore uncalibrated. The recently developed method SECOM [10] ‘normalizes’ the observed count data to recover the underlying absolute abundances. These estimates are then used to calculate Pearson’s correlations for measuring linear dependencies and distance correlations [16] for measuring nonlinear dependencies. When their weights are set to be equal, the normalized counts are essentially clr-transformed data. SECOM handles zero counts by restricting its analysis to complete cases when both taxa in the pair have non-zero counts. Unlike other methods, SECOM generates p-values, which are based on the t-test for Pearson’s correlations and permutation tests for distance correlations. However, there is no theoretical justification for either test. Additionally, while SECOM accounts for experimental bias and sequencing variation, it does not accommodate clustering structures in the samples. In this article, we address a significant gap in microbial network analysis by introducing a novel testing method called TestNet. We first demonstrate that the basis correlation coefficients are unidentifiable, which motivated us to focus on testing instead of estimation. We propose using the sample Pearson’s covariance and distance covariance of clr data as statistics for testing linear and nonlinear dependencies, respectively. We also propose a strategy for handling zero counts. Furthermore, we develop a permutation-based procedure for generating null replicates that accounts for compositional effects, extensive zero counts, and clustered samples. Finally, we construct an omnibus test that integrates results from both linear and nonlinear tests, enhancing power to detect general forms of dependencies. We evaluate TestNet against existing methods using extensive simulation studies and apply TestNet to two real microbiome studies: one consisting of independent samples and the other consisting of clustered samples. Results Simulation results Performance of TestNet and existing methods Using simulated data, we compared the performance of TestNet with SECOM, SparCC, CCLasso, and COAT. We applied TestNet and SECOM to the simulated data to infer both linear and nonlinear dependencies, and SparCC, CCLasso, and COAT to infer linear dependencies only. For SparCC and CCLasso, we applied a threshold of 0.3 to their estimated correlations. COAT has its own internal thresholding, so no additional thresholding was performed on its estimated correlations. SECOM produced p-values, which were used to detect dependencies by the BH procedure. All details regarding the simulation study design are provided in the Methods. We evaluated the methods using the following metrics: false positive rate (FPR, i.e., type I error) at the nominal level of 0.005 (as used in [10]), empirical FDR at the nominal level of 10%, sensitivity (i.e., recall) at the nominal FDR of 10%, and area under the precision-recall curve (AUC, sometimes referred to as PRAUC in the literature), where precision equals \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1-\text {FDR}$$\end{document}1-FDR. While the FPR offers insights into error rate control for individual tests, the FDR is more appropriate for managing errors in multiple testing scenarios. The AUC assesses the overall performance of a method across all nominal FDR levels. All results were based on 100 data replicates, except for the global null case (i.e., Identity covariance structure), where the FDR in each replicate was either 0 or 1 (i.e., exhibiting high variability) and thus the results were based on 1000 replicates to obtain more stable estimates. Note that for SparCC, CCLasso, and COAT, the FPR, FDR and sensitivity were calculated based on their detections, which are independent of the nominal levels set here; their AUC was calculated based on their estimated correlations without thresholding. We considered a wide range of sample sizes n, between 50 and 1000. We extended the upper limit to 1000 to account for the availability of large-scale microbiome studies [17, 18]. We began with cases where the data were simulated with independent samples, J=100J=100 taxa, Normal distributions, and no experimental bias. The results of inferring linear and nonlinear dependencies, based on data with the stringent and lenient filters, are displayed in Figs. 1, 2 and 3, and Additional file 1: Figure S1, respectively. In all these cases, TestNet yielded the most accurate empirical error rates, both FPR and FDR, which closely aligned with their respective nominal levels. Only when n became very large (e.g., 1000) did TestNet produce slightly inflated FPR and FDR (e.g., under AR4), as the bias term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 becomes more pronounced. Under the Identity covariance structure, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0, TestNet exhibited no inflation in FPR regardless of n. TestNet showed the highest sensitivity among all methods, with the exception of COAT, which suffered from severely inflated error rates. Additionally, TestNet achieved the highest AUC. The only exception was its linear test under the U-shaped structure, where any linear test is expected to have little power (Fig. 1).Fig. 1Results on inference of linear dependencies based on data simulated with sample sizes (x-axis) varying from 50 to 1000, independent samples, J=100J=100, Normal distributions, and no experimental bias, and filtered with the stringent criterion. ‘FPR’ is the false positive rate (i.e., type I error) at the nominal level of 0.005 (gray dashed line). ‘FDR’ is the empirical false discovery rate at the nominal level of 10% (gray dashed line). ‘Sensitivity’ is calculated based on the nominal FDR of 10%. ‘AUC’ is the area under the precision-recall curve Results on inference of linear dependencies based on data simulated with sample sizes (x-axis) varying from 50 to 1000, independent samples, J=100J=100, Normal distributions, and no experimental bias, and filtered with the stringent criterion. ‘FPR’ is the false positive rate (i.e., type I error) at the nominal level of 0.005 (gray dashed line). ‘FDR’ is the empirical false discovery rate at the nominal level of 10% (gray dashed line). ‘Sensitivity’ is calculated based on the nominal FDR of 10%. ‘AUC’ is the area under the precision-recall curve Fig. 2Results on inference of linear dependencies based on data filtered with the lenient criterion. Refer to the caption of Fig. 1 for additional information Results on inference of linear dependencies based on data filtered with the lenient criterion. Refer to the caption of Fig. 1 for additional information Fig. 3Results on inference of nonlinear dependencies based on data filtered with the stringent criterion. Refer to the caption of Fig. 1 for additional information Results on inference of nonlinear dependencies based on data filtered with the stringent criterion. Refer to the caption of Fig. 1 for additional information Among the existing methods, SECOM demonstrated performance most similar to TestNet, though it had lower sensitivity and AUC. However, SECOM failed to control error rates in many instances, especially when n was large or when testing linear dependencies at rare taxa due to the use of a lenient filter. SparCC exhibited low sensitivity and the lowest AUC, with high error rates when n was small. Its sensitivity even decreased as n increased, as its estimated correlations converged to values lower than the true correlations. CCLasso generally yielded low error rates and sensitivity, although its FDR was sometimes high (e.g., in Fig. 2). Conversely, COAT typically had high error rates and sensitivity, but its AUC was low. These conclusions generally hold when łog ZlogZ were sampled from Gamma distributions (Additional file 1: Figures S2 and S3) or when the number of taxa J was set to 50 (Additional file 1: Figures S4 and S5). As expected, TestNet’s inflation of error rates under large samples sizes slightly worsened when J became unrealistically small. Adding experimental bias The results for simulated data with experimental bias are displayed in Additional file 1: Figures S6 and S7. Compared to Figs. 1 and 2, experimental bias had minimal effects on the results of TestNet and other existing methods. This outcome is expected, as all methods are based on or closely related to Pearson’s covariance or distance covariance of clr-transformed data, both of which have been shown earlier to be invariant to multiplicative experimental bias. Clustered data With a fixed cluster size of four, we varied the number of clusters between 13 and 250 to achieve sample sizes between 52 and 1000. We applied TestNet with three different permutation schemes: shuffling data both between and within clusters (referred to as TestNet-b-w), shuffling data between clusters only (TestNet-b), and shuffling all samples irrespective of the clustering structure (TestNet-free). The results for data filtered with the stringent and lenient criteria are shown in Fig. 4 and Additional file 1: Figure S8, respectively. As expected, TestNet-b-w demonstrated the best performance among the three versions of TestNet. TestNet-free yielded inflated error rates, with a level of inflation similar to that of SECOM (when n was small), since SECOM also employed the free permutation scheme regardless of the clustering structure. TestNet-b, without the within-cluster shuffling, produced lower sensitivity than TestNet-b-w, particularly under the Sparse structure.Fig. 4Results on inference of linear dependencies based on clustered data simulated with the number of clusters (x-axis) varying from 13 to 250 and four samples in each cluster, and filtered with the stringent criterion. Refer to the caption of Fig. 1 for additional information Results on inference of linear dependencies based on clustered data simulated with the number of clusters (x-axis) varying from 13 to 250 and four samples in each cluster, and filtered with the stringent criterion. Refer to the caption of Fig. 1 for additional information Omnibus test The results of the TestNet omnibus test, along with its linear and nonlinear test results, for data filtered with the stringent and lenient criteria are presented in Additional file 1: Figures S9 and S10, respectively. The omnibus test controlled both error rates, FPR and FDR, and consistently tracked either the linear or nonlinear test, whichever demonstrated superior performance in sensitivity and AUC. Longitudinal rectal mucosal microbiome data The first real dataset we analyzed was generated from a longitudinal clinical study conducted by our research team [19] to investigate microbiome responses to experimentally induced rectal mucosal injury in men who have sex with men (MSM). At the baseline visit, 19 MSM underwent a rectal mucosal injury using biopsy forceps approximately 6 cm from the anal verge. Participants returned for follow-up visits on days 2, 5, and 8. Rectal mucosal secretions were collected at all four visits for microbiome composition profiling. These samples were sequenced by Illumina MiSeq for the 16S rRNA gene, which data were then summarized into a taxa count table at the genus level. Our goal was to identify inter-genus dependencies within the rectal mucosal microbiome community of these MSM. There were 372 genera with at least one read across the 76 samples. The library sizes were adequate, ranging from 12,601 to 353,656, with a median of 133,932 and a mean of 135,771. We applied the stringent filter to exclude genera with average relative abundances less than 0.1% across the 76 samples, resulting in 106 genera for network analysis. The ordination plot in Fig. 5 shows that the four samples from each subject tend to cluster together, indicating that microbiome profiles are more similar within subjects than across subjects. Therefore, it is important to account for this clustering structure when inferring the microbial network.Fig. 5Ordination of the longitudinal gut microbiome samples based on the Bray-Curtis distance. Samples from the same subject are marked with the same color and the same subject ID Ordination of the longitudinal gut microbiome samples based on the Bray-Curtis distance. Samples from the same subject are marked with the same color and the same subject ID We applied TestNet and other methods in the same manner as in the simulation studies. Specifically, we used TestNet with both between- and within-cluster permutations to preserve the clustering structure. The other methods were applied without considering the clustering structure. The numbers of dependent pairs of genera detected by these methods are displayed in Table 1. TestNet, using the omnibus test, detected 240 general dependencies at a nominal FDR level of 0.5%. With this nominal FDR level, we expect the detected dependencies to include approximately one false positive. As expected, the linear and nonlinear tests of TestNet detected less dependencies, 213 and 223 respectively, than the omnibus test (240). However, the three numbers of detections do not differ significantly, as the linear and nonlinear tests are inherently overlapping: their test statistics can capture both types of relationships. SECOM applied its linear and nonlinear tests separately to each pair of taxa. The linear test detected 210 dependencies, while the nonlinear test detected none–a discrepancy that is difficult to reconcile given the overlap of the two test statistics. In addition, applying the linear and nonlinear tests separately made it challenging to control the overall error rate. The other methods detected linear dependencies only. The number of detections varied dramatically between methods, from 5,010 to 768, highlighting the importance of selecting the appropriate method. In particular, some of these numbers, such as 1,146 and even 5,010, are unreasonably high, as there are only 5,565 tests in total. Note that none of the existing methods can control the FDR, especially in the presence of clustered samples. Table 1Number of dependent pairs of taxa detected in the real datasetsType of testDatasetSubjectsMethodLinearNonlinearOmnibusRectalMSMTestNet213223240mucosal(n=76n=76,SECOM2100microbiomeJ=106J=106)COAT5010SparCC1146CCLasso768GutLeanTestNet293946microbiome(n=63n=63,SECOM590J=55J=55)COAT909SparCC30CCLasso61J represents the number of genera in the network analysis after applying filters. The results from TestNet and SECOM are based on an nominal FDR level of 0.5% for the rectal mucosal microbiome data and 10% for the gut microbiome data Table 1Number of dependent pairs of taxa detected in the real datasetsType of testDatasetSubjectsMethodLinearNonlinearOmnibusRectalMSMTestNet213223240mucosal(n=76n=76,SECOM2100microbiomeJ=106J=106)COAT5010SparCC1146CCLasso768GutLeanTestNet293946microbiome(n=63n=63,SECOM590J=55J=55)COAT909SparCC30CCLasso61J represents the number of genera in the network analysis after applying filters. The results from TestNet and SECOM are based on an nominal FDR level of 0.5% for the rectal mucosal microbiome data and 10% for the gut microbiome data Number of dependent pairs of taxa detected in the real datasets J represents the number of genera in the network analysis after applying filters. The results from TestNet and SECOM are based on an nominal FDR level of 0.5% for the rectal mucosal microbiome data and 10% for the gut microbiome data Thus far, we have detected general dependencies using the TestNet omnibus test. To get further insights into whether each general dependence is roughly linear or nonlinear, we employed a heuristic strategy by comparing the linear and nonlinear test p-values. A total of 153 dependencies had linear p-values lower than or equal to their nonlinear counterparts, and we classified them as linear dependencies. The remaining 87 dependencies were classified as nonlinear dependencies. We further visualized the detected (general) dependencies by TestNet as network edges in Fig. 6. The most connected genera include Peptoniphilus (21 edges), Prevotella (19 edges), Eubacterium hallii group (19 edges), Anaerococcus (17 edges), and Peptostreptococcus (15 edges). These genera are all abundant and have previously been associated with receptive anal intercourse (RAI) in [20], suggesting that they are likely keystone taxa exerting a profound influence on the structure of the rectal mucosal microbiome community in MSM. In many subnetworks, their members tend to exhibit (linear) correlations in the same direction, either positive or negative, suggesting that they form ecological guilds that share co-abundance behaviors and similar functions.Fig. 6Microbial networks of general dependencies identified by TestNet (the omnibus test) based on the rectal mucosal microbiome data of MSM. Positive and negative linear correlation estimates are displayed as blue and red edges, respectively, with the thickness of the edges indicating the magnitude of the linear correlation estimates. Here, the linear correlation estimates were used for illustration purposes only Microbial networks of general dependencies identified by TestNet (the omnibus test) based on the rectal mucosal microbiome data of MSM. Positive and negative linear correlation estimates are displayed as blue and red edges, respectively, with the thickness of the edges indicating the magnitude of the linear correlation estimates. Here, the linear correlation estimates were used for illustration purposes only Cross-sectional gut microbiome data The second real dataset we analyzed was derived from a cross-sectional study [21] that sequenced the stool samples of 63 lean (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {BMI} < 25$$\end{document}BMI<25) individuals using 454/Roche pyro-sequencing for the 16S rRNA gene. The sequence data consisted of an average of 9,265 reads per sample (with a standard deviation of 3,864) across 87 genera detected in at least one sample. We applied the lenient filter to focus on the J = 55J=55 genera that appeared in at least five samples. Again, our goal was to identify inter-genus dependencies within the gut microbiome community of these lean subjects. Since the samples were independent, we applied TestNet with free shuffling across all samples. The numbers of detected pairs of taxa are summarized in Table 1. TestNet (omnibus) detected 46 general dependencies (at an FDR of 10%, implying 4–5 false positives), which were classified into 29 linear dependencies and 39 nonlinear ones. Although the other methods were designed for independent samples, their numbers of detections still varied dramatically from method to method. Our simple application of COAT detected 909 correlations. In contrast, the network analysis in the COAT paper [9] retained 254 correlations that were reproduced in at least 85% bootstrap replicates. Despite this, these results may still contain many false positives due to the high error rates of COAT. The microbial networks inferred by TestNet (omnibus) is visualized in Fig. 7. Bacteroides is the 2nd most connected taxon in the network. Within this genus, several species, such as Bacteroides fragilis and Bacteroides thetaiotaomicron, are well established as keystone taxa in the human gut microbiome [1]. The network includes a strong negative correlation between Bacteroides and Prevotella. The ratio of Bacteroides and Prevotella has long been used to define gut microbial enterotypes that reflect long-term dietary patterns [21].Fig. 7Microbial networks of general dependencies identified by TestNet (the omnibus test) based on the gut microbiome data of lean individuals. Refer to the caption of Fig. 6 for additional information Microbial networks of general dependencies identified by TestNet (the omnibus test) based on the gut microbiome data of lean individuals. Refer to the caption of Fig. 6 for additional information Discussion TestNet shares notable similarities with SECOM. TestNet’s use of the clr transformation aligns with SECOM’s normalization of observed count data. Both methods also measure dependencies using similar metrics: SECOM employs Pearson and distance correlations for linear and nonlinear dependencies, while TestNet uses Pearson and distance covariances. However, there are key differences between the two. SECOM incorporates Pearson and distance variances that are inflated by sequencing variation, which TestNet does not use. While SECOM addresses zero counts by restricting its analysis to complete cases, TestNet leverages the information in zeros by adding a pseudo-count of 0.5 for clr-transformation, followed by mean imputation. Additionally, SECOM overlooks the constraints in the ‘normalized’ data during its t-tests and permutation tests, whereas TestNet explicitly accounts for the constraints in the clr data within its permutation procedure. Strikingly, the FDR of SECOM exceeded 80% when constructing a network with 50–100 independent samples that included many rare taxa (e.g., Fig. 2), mirroring the scenario observed in the gut microbiome data we analyzed. We have demonstrated several biases in Pearson’s covariance and variance of the clr data, and have provided insights into the cause of each bias. The biases \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1$$\end{document}B1 through \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_4$$\end{document}B4 all contribute to the inconsistency of correlation estimates. However, only \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 has the potential to induce FDR inflation for TestNet. Therefore, even when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1$$\end{document}B1, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_3$$\end{document}B3, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_4$$\end{document}B4 are substantial, such as when the number of taxa J is small (e.g., J=20J=20 or even J=10J=10), TestNet is still able to control the FDR as long as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0, as occurs under the Identity structure (Additional file 1: Figures S11–S14). Furthermore, even when J is as small as 20 or even 10 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 becomes non-negligible under other dependence structures, TestNet continues to maintain satisfactory FDR control when the sample size is below 200 (Additional file 1: Figures S11–S14). TestNet is applicable to compositional data beyond microbiome data, such as gene expression data, brain cell subtypes, and immune cell subsets. However, verifying the sparse dependence assumption in real data is challenging because the underlying absolute abundances are unobservable. Simulation therefore provides the only practical way to assess this assumption. In our study, we considered two dependence structures, namely the “Hub” and “Block” scenarios, in which dependent pairs account for approximately one quarter of all possible pairs and may appear relatively dense at first glance. However, the sparse dependence assumption concerns the aggregation of dependencies rather than merely the number of dependent pairs. In these scenarios, positive and negative dependencies coexist and partially offset each other when aggregated across the network, so the sparse dependence assumption still holds (Additional file 1: Figure S15). On the other hand, the relatively small set of dependent pairs detected by TestNet (Linear) provides empirical support for the plausibility of the sparse dependence assumption in the analyzed real datasets (Table 1). If the sparse dependence assumption were violated, the number of detected pairs would increase dramatically. The inherent assumption underlying our permutation procedure is that, under the null hypothesis of no taxon-taxon dependence, the absolute abundances of one taxon are exchangeable across individuals relative to those of the other taxon. In case-control studies, differences in the marginal mean or variance of taxon abundances across groups violate this assumption and can inflate type I error rates [22]; therefore, TestNet should be applied separately within each group. The same issue arises in clustered pre-/post-treatment data, where treatment-induced shifts break exchangeability, and TestNet should likewise be conducted separately for pre- and post-treatment samples. Overall, our method is intended as an initial screening tool to detect the presence of any taxon-taxon dependence, after which more targeted analyses can be used to characterize the specific dependence structure. We have implemented our method in the R package TestNet, which is available on GitHub at https://github.com/yijuanhu/TestNet in formats appropriate for Macintosh or Windows. Despite the permutation procedure, TestNet is computationally efficient for data with small sample sizes and very sparse dependencies. For example, using one core of a MacBook Pro laptop (Apple M3 Pro, 18GB memory), it took 27 seconds to analyze a simulated dataset with 100 samples, 100 taxa, and the AR1 structure, 15 seconds to analyze the gut microbiome data, and 10 minutes to analyze the rectal mucosal microbiome data. The extended time for the last analysis is due to the large number of dependencies, requiring over 60,000 permutation replicates. TestNet could be improved by splitting the sequential permutation procedure into blocks of a fixed number of permutations and parallelizing the permutations within each block. In practice, Pearson and Spearman correlations remain widely used for constructing microbial networks. As noted in the Introduction, standard correlation analysis applied to relative abundance data tends to induce negative correlations even when the underlying absolute abundances are independent. In contrast, correlation analysis of raw read counts tends to produce spurious positive correlations, as all taxa are strongly correlated with library size, which can vary substantially across samples. Among the three data types, clr-transformed data therefore provide the most reasonable basis for correlation analysis; indeed, this consideration lies at the core of SECOM. Additional file 1: Figure S16 shows that, under the stringent filtering scheme, Pearson and Spearman correlations computed on clr-transformed data achieved the best performance among the three data types and exhibited performance comparable to SECOM, whereas correlations based on relative abundances or read counts resulted in severely inflated FDR. Under the lenient filtering scheme, which retains many rare taxa (Additional file 1: Figure S17), correlation analysis based on clr-transformed data also deteriorated, due to the spurious correlations induced by rare taxa, as discussed earlier. Conclusions We have introduced TestNet, the first testing method for inferring microbial networks that provides calibrated results by controlling the FDR. TestNet is highly robust and performs well even with a large number of zeros in the data. TestNet is also highly versatile: it accommodates both independent and clustered samples, offers separate linear and nonlinear tests, and includes an omnibus test that eliminates the need to pre-specify the type of relationship. Methods We consider a microbial community with J taxa. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z_{ij}$$\end{document}Zij denote the underlying absolute abundance of taxon j in sample i. In what follows, we suppress the sample index i in the notation except where it is required for clarity. The corresponding underlying relative abundance is defined as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j=Z_j/\sum _{j=1}^JZ_j$$\end{document}Yj=Zj/∑j=1JZj. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xj be the observed read count, whose distribution is governed by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj and library size l (i.e., sequencing depth). We calculate the observed relative abundance as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j=(X_j+0.5)/\sum _{j'=1}^J(X_{j'}+0.5)$$\end{document}Pj=(Xj+0.5)/∑j′=1J(Xj′+0.5), where a pseudo-count of 0.5 is added to enable the clr transformation defined below. For a general variable \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_j$$\end{document}Wj, which may represent \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z_j$$\end{document}Zj, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj or \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj, the clr transformation is defined as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}W_j\equiv \log W_j - J^{-1}\sum _{j'=1}^J \log W_{j'}$$\end{document}clrWj≡logWj-J-1∑j′=1JlogWj′. We assume that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z_j> 0$$\end{document}Zj>0 for all taxa across all samples. Under this assumption, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj are well defined without the need for a pseudo-count. This assumption is reasonable, as the observed read count data lack the resolution to distinguish true zero abundance from extremely low but nonzero abundance. Our method is based on two key facts: (1) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j=\textrm{clr}Y_j$$\end{document}clrZj=clrYj, which establishes a link between absolute and relative abundances, and (2) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj is a consistent estimator for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj (assuming no experimental bias; this assumption will be relaxed in “Impact of experimental bias” section). Additionally, our method relies on the following two assumptions:Sparse Dependence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \left| \sum \limits _{j'\ne j}\textrm{Cov}\left( \log Z_j, \log Z_{j'} \right) \right| \le \textrm{Var} \left( \textrm{clr}P_j\right) , ~~~~~j=1,\ldots ,J, \end{aligned}$$\end{document}∑j′≠jCovlogZj,logZj′≤VarclrPj,j=1,…,J, where all \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} \left( \textrm{clr}P_j\right)$$\end{document}VarclrPj’s are bounded.Independence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xjs across taxa are independent given \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(Y_1,Y_2,\ldots ,Y_J)$$\end{document}(Y1,Y2,…,YJ).The Sparse Dependence Assumption is more easily satisfied in the presence of large overdispersion arising from sequencing variations. In this case, large overdispersion becomes a bless rather than a curse. The Independence Assumption ensures that sequencing variations across taxa are independent. Sparse Dependence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \left| \sum \limits _{j'\ne j}\textrm{Cov}\left( \log Z_j, \log Z_{j'} \right) \right| \le \textrm{Var} \left( \textrm{clr}P_j\right) , ~~~~~j=1,\ldots ,J, \end{aligned}$$\end{document}∑j′≠jCovlogZj,logZj′≤VarclrPj,j=1,…,J, where all \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} \left( \textrm{clr}P_j\right)$$\end{document}VarclrPj’s are bounded. Sparse Dependence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \left| \sum \limits _{j'\ne j}\textrm{Cov}\left( \log Z_j, \log Z_{j'} \right) \right| \le \textrm{Var} \left( \textrm{clr}P_j\right) , ~~~~~j=1,\ldots ,J, \end{aligned}$$\end{document}∑j′≠jCovlogZj,logZj′≤VarclrPj,j=1,…,J, where all \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} \left( \textrm{clr}P_j\right)$$\end{document}VarclrPj’s are bounded. Independence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xjs across taxa are independent given \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(Y_1,Y_2,\ldots ,Y_J)$$\end{document}(Y1,Y2,…,YJ). Independence Assumption: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xjs across taxa are independent given \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(Y_1,Y_2,\ldots ,Y_J)$$\end{document}(Y1,Y2,…,YJ). Test statistic for linear dependencies As in all the aforementioned existing methods, the linear dependence of interest is Pearson’s covariance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\log Z_j, \log Z_k)$$\end{document}Cov(logZj,logZk) for j≠ kj≠k. It is related to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}Z_j, \textrm{clr}Z_k)$$\end{document}Cov(clrZj,clrZk) through the equation\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \textrm{Cov}\left( \textrm{clr}Z_j, \textrm{clr}Z_k\right) = \textrm{Cov}\left( \log Z_j, \log Z_k\right) + \mathcal {B}_1^{(jk)} + \mathcal {B}_2^{(jk)}, \end{aligned}$$\end{document}CovclrZj,clrZk=CovlogZj,logZk+B1(jk)+B2(jk),where\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathcal {B}_1^{(jk)} = -J^{-1}\textrm{Var} \left( \log Z_j\right) -J^{-1}\textrm{Var} \left( \log Z_k\right) +J^{-2}\sum \limits _{j'}\textrm{Var} \left( \log Z_{j'}\right) \end{aligned}$$\end{document}B1(jk)=-J-1VarlogZj-J-1VarlogZk+J-2∑j′VarlogZj′and\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathcal {B}_2^{(jk)} = -J^{-1}\sum \limits _{j'\ne j}\textrm{Cov}\left( \log Z_j, \log Z_{j'} \right) -J^{-1}\sum \limits _{j'\ne k}\textrm{Cov}\left( \log Z_k, \log Z_{j'} \right) + J^{-2}\sum \limits _{j'\ne j''}\textrm{Cov}\left( \log Z_{j'}, \log Z_{j''}\right) . \end{aligned}$$\end{document}B2(jk)=-J-1∑j′≠jCovlogZj,logZj′-J-1∑j′≠kCovlogZk,logZj′+J-2∑j′≠j′′CovlogZj′,logZj′′. We omit the superscript (jk) in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1^{(jk)}$$\end{document}B1(jk) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2^{(jk)}$$\end{document}B2(jk) in general unless otherwise specified. We refer to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1$$\end{document}B1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 as bias terms, which are induced by using \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_i$$\end{document}clrZi in place of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj. In particular, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 exclusively accounts for the portion attributable to taxon-taxon dependencies. Since \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\textrm{clr}P_j)$$\end{document}Var(clrPj) is bounded, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\log Z_j)$$\end{document}Var(logZj) is also bounded due to their relationship in Eq. (2) below. Consequently, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1=O(J^{-1})$$\end{document}B1=O(J-1) is negligible as long as J is sufficiently large. Similarly, under the Sparse Dependence Assumption, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=O(J^{-1})$$\end{document}B2=O(J-1) is also negligible. Next, we substitute the observed relative abundances \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_i$$\end{document}Yi in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}Cov(clrYj,clrYk) to obtain \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov(clrPj,clrPk) and calculate the sample-based statistic, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk), as our test statistic. In fact, all existing methods are closely related to this statistic. The replacement of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_i$$\end{document}Yi with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_i$$\end{document}Pi introduces another source of bias that is often overlooked by existing methods. To demonstrate this, we use conditional expectation to establish the relationship between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov(clrPj,clrPk) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}Cov(clrYj,clrYk) for j≠ kj≠k, conditioning on \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbb {Y}=(Y_1,Y_2,\ldots ,Y_J)$$\end{document}Y=(Y1,Y2,…,YJ):\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)= & \textrm{E}\left[ \textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k | \mathbb {Y})\right] + \textrm{Cov}\left[ \textrm{E}(\textrm{clr}P_j | \mathbb {Y}), \textrm{E}(\textrm{clr}P_k | \mathbb {Y})\right] \\\approx & \textrm{E}\left[ \textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k | \mathbb {Y})\right] + \textrm{Cov}\left( \textrm{clr}Y_j, \textrm{clr}Y_k\right) . \end{aligned}$$\end{document}Cov(clrPj,clrPk)=ECov(clrPj,clrPk|Y)+CovE(clrPj|Y),E(clrPk|Y)≈ECov(clrPj,clrPk|Y)+CovclrYj,clrYk. The last step follows from the delta method, which provides an approximation accurate to the order of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(l^{-1})$$\end{document}O(l-1). Recall that l denotes the library size, which is at least in the range of thousands. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_3\equiv \textrm{E}\left[ \textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k | \mathbb {Y})\right]$$\end{document}B3≡ECov(clrPj,clrPk|Y) represents the bias caused by sequencing variation (when the underlying relative abundances are fixed). We derived in Additional file:Text S1 that, under the Independence Assumption,1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \mathcal {B}_3\approx -J^{-1}\textrm{E}\left[ \textrm{Var} \left( \textrm{clr}P_j| \mathbb {Y}\right) \right] -J^{-1}\textrm{E}\left[ \textrm{Var} \left( \textrm{clr}P_k| \mathbb {Y}\right) \right] + J^{-2}\sum \limits _{j'} \textrm{E}\left[ \textrm{Var} \left( \textrm{clr}P_{j'}| \mathbb {Y}\right) \right] , \end{aligned}$$\end{document}B3≈-J-1EVarclrPj|Y-J-1EVarclrPk|Y+J-2∑j′EVarclrPj′|Y,which indicates that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_3$$\end{document}B3 is an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(J^{-1})$$\end{document}O(J-1) term. In summary, the overall bias in \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov(clrPj,clrPk) relative to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\log Z_j, \log Z_k)$$\end{document}Cov(logZj,logZk) is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1+\mathcal {B}_2+\mathcal {B}_3$$\end{document}B1+B2+B3. Thus, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk) is not a consistent estimator for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\log Z_j, \log Z_k)$$\end{document}Cov(logZj,logZk) as the sample size n goes to infinity. Nevertheless, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk) makes a reasonable statistic for testing linear dependencies, since the overall bias is an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(J^{-1})$$\end{document}O(J-1) small term. Correlation coefficients cannot be identifiable due to sequencing variation Although sequencing variation has only a small impact on the covariance, it tends to have a much larger effect on the variance and hence the correlation. We use the conditional expectation again to establish the relationship between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\textrm{clr}P_j)$$\end{document}Var(clrPj) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\textrm{clr}Y_j)$$\end{document}Var(clrYj):2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \textrm{Var} (\textrm{clr}P_j)= & \textrm{E}\left[ \textrm{Var} (\textrm{clr}P_j | \mathbb {Y})\right] + \textrm{Var} \left[ \textrm{E}(\textrm{clr}P_j | \mathbb {Y})\right] \nonumber \\\approx & \textrm{E}\left[ \textrm{Var} (\textrm{clr}P_j | \mathbb {Y})\right] + \textrm{Var} \left( \textrm{clr}Y_j\right) \nonumber \\= & \textrm{E}\left[ \textrm{Var} (\textrm{clr}P_j | \mathbb {Y})\right] + \textrm{Var} \left( \log Z_j\right) +\mathcal {B}_1^{(jj)}+\mathcal {B}_2^{(jj)}\nonumber \\\approx & \textrm{E}\left[ \textrm{Var} (\textrm{clr}P_j | \mathbb {Y})\right] + \textrm{Var} \left( \log Z_j\right) . \end{aligned}$$\end{document}Var(clrPj)=EVar(clrPj|Y)+VarE(clrPj|Y)≈EVar(clrPj|Y)+VarclrYj=EVar(clrPj|Y)+VarlogZj+B1(jj)+B2(jj)≈EVar(clrPj|Y)+VarlogZj. The last approximation follows from the fact that both \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1^{(jj)}=O(J^{-1})$$\end{document}B1(jj)=O(J-1) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2^{(jj)}=O(J^{-1})$$\end{document}B2(jj)=O(J-1) are negligible. If the variance in read count \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xj due to sequencing, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (X_j|\mathbb {Y})$$\end{document}Var(Xj|Y), is at least \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(l^2)$$\end{document}O(l2) (i.e., it does not approach zero when divided by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l^2$$\end{document}l2 as l approaches infinity), the bias term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_4\equiv \textrm{E}\left[ \textrm{Var} (\textrm{clr}P_j | \mathbb {Y})\right] =\textrm{E}\left[ \textrm{Var} \{\textrm{clr}(X_j+0.5) | \mathbb {Y}\}\right]$$\end{document}B4≡EVar(clrPj|Y)=EVar{clr(Xj+0.5)|Y} remains significant regardless of the value of l. More details can be found in Additional file:Text S2. This condition holds for read count data with some level of overdispersion, such as the Negative-Binomial data, though it represents a very moderate level of overdispersion. This bias causes the variance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\textrm{clr}P_j)$$\end{document}Var(clrPj) to substantially overestimate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Var} (\log Z_j)$$\end{document}Var(logZj) and, consequently, the correlation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cor}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cor(clrPj,clrPk) to markedly underestimate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cor}(\log Z_j, \log Z_k)$$\end{document}Cor(logZj,logZk). This result has important implications for methods that rely on thresholding \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cor}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cor^(clrPj,clrPk), as they tend to miss true correlations. Since the correlation coefficient is unidentifiable, we use \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk) as a test statistic and focus on testing its deviation from the null hypothesis. Test statistic for nonlinear dependencies Following SECOM, we adopt distance covariance [16] to measure nonlinear dependencies and denote the distance covariance of two variables, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_1$$\end{document}W1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_2$$\end{document}W2, by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{dCov}(W_1, W_2)$$\end{document}dCov(W1,W2). By definition, distance covariance measures the difference in the joint characteristic function of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_1$$\end{document}W1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_2$$\end{document}W2 with and without the assumption of independence between \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_1$$\end{document}W1 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_2$$\end{document}W2. The sample distance covariance, denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}(W_1, W_2)$$\end{document}dCov^(W1,W2), is calculated as the sample Pearson’s covariance between the (centered) pairwise distances among the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_1$$\end{document}W1 data and the distances among the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_2$$\end{document}W2 data. Specifically, denote the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_1$$\end{document}W1 data on n samples by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{W_{1i}, i=1,\ldots ,n\}$$\end{document}{W1i,i=1,…,n}, and compute the n× nn×n Euclidean distance matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{d_{1,ii'}\}=\{|W_{1i}-W_{1i'}|\}$$\end{document}{d1,ii′}={|W1i-W1i′|} and then the centered distance matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{D_{1,ii'}\}=\{d_{1,ii'}-\overline{d}_{1,i.}-\overline{d}_{1,.i'}+\overline{d}_{1,..}\}$$\end{document}{D1,ii′}={d1,ii′-d¯1,i.-d¯1,.i′+d¯1,..}, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\overline{d}_{1,i.}=n^{-1}\sum _{l=1}^nd_{1,il}$$\end{document}d¯1,i.=n-1∑l=1nd1,il, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\overline{d}_{1,.i'}=n^{-1}\sum _{l=1}^nd_{1,li'}$$\end{document}d¯1,.i′=n-1∑l=1nd1,li′, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\overline{d}_{1,..}=n^{-2}\sum _{l,m=1}^nd_{1,lm}$$\end{document}d¯1,..=n-2∑l,m=1nd1,lm. Similarly, compute the distance matrix and centered distance matrix based on the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$W_2$$\end{document}W2 data as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{d_{2,ii'}\}$$\end{document}{d2,ii′} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{D_{2,ii'}\}$$\end{document}{D2,ii′}. Finally, calculate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}(W_1, W_2)=n^{-2}\sum _{i,i'=1}^n D_{1,ii'} D_{2, ii'}$$\end{document}dCov^(W1,W2)=n-2∑i,i′=1nD1,ii′D2,ii′. Unlike Pearson’s covariance, distance covariance is always non-negative. A zero distance covariance indicates independence of two random variables, whereas a zero Pearson’s covariance does not does not necessarily imply independence unless the variables are Gaussian-distributed. Although distance covariance can capture more general patterns of dependencies including linear ones, it is less efficient than Pearson’s for capturing linear dependencies. More details about distance covariance can be found in [16, 23]. Our distance covariance of interest is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{dCov}(\log Z_j, \log Z_k)$$\end{document}dCov(logZj,logZk). Following the procedure in the linear case, we replace \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj, equate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj, and substitute \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj. We obtain the test statistic for nonlinear dependencies as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}dCov^(clrPj,clrPk). Like its linear counterpart, distance covariance and variance are subject to bias due to data compositionality and sequencing variation. However, it remains unclear how to derive these bias terms in closed forms, so we resort to simulation studies for their evaluation. Handling zeros The use of pseudo-counts (e.g., 0.5) to address zeros, while common in microbiome data analysis, introduces a problem that, fortunately, has a straightforward solution. For a zero entry of the taxa count table corresponding to sample i and taxon j, i.e., \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_{ij}=0$$\end{document}Xij=0, the clr-transformed value is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_{ij} = \log 0.5 - J^{-1} \sum _{j'}\log (X_{ij'}+0.5)$$\end{document}clrPij=log0.5-J-1∑j′log(Xij′+0.5). This value remains constant across taxa within the same sample but varies across samples, which tends to create spurious associations between pairs of taxa, especially when both taxa in a pair have extensive zeros. To eliminate these spurious associations, we propose replacing such \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_{ij}$$\end{document}clrPij values with their taxon-specific means before calculating the test statistics, where the taxon-specific means are based on the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_{ij}$$\end{document}clrPij values at zero entries only. This modification effectively breaks the spurious associations at zero entries while maximally preserving the orders of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_{ij}$$\end{document}clrPij values between non-zero and zero entries. We denote the modified \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_{ij}$$\end{document}clrPij value by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mclr}P_{ij}$$\end{document}mclrPij and redefine the test statistics as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}\left( \textrm{mclr}P_j, \textrm{mclr}P_k\right)$$\end{document}Cov^mclrPj,mclrPk and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}\left( \textrm{mclr}P_j, \textrm{mclr}P_k\right)$$\end{document}dCov^mclrPj,mclrPk. Impact of experimental bias In the presence of experimental bias, the underlying relative abundance that we aim to ‘measure’, denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y^*_j$$\end{document}Yj∗, differs from the true relative abundance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj. We assume that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y^*_j$$\end{document}Yj∗ is related to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj and the taxon-specific bias factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _j$$\end{document}γj through the WMC model [11]:\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} Y^*_j=\frac{\exp (\gamma _j)Y_j}{\sum _{j'=1}^J\exp (\gamma _{j'})Y_{j'}}. \end{aligned}$$\end{document}Yj∗=exp(γj)Yj∑j′=1Jexp(γj′)Yj′. It follows that their clr-transformed values differ only by a taxon-specific constant:\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \textrm{clr}Y^*_j = \left( \gamma _j- J^{-1}\sum \limits _{j'=1}^J\gamma _{j'}\right) + \textrm{clr}Y_j, \end{aligned}$$\end{document}clrYj∗=γj-J-1∑j′=1Jγj′+clrYj,which leads to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}Y^*_j, \textrm{clr}Y^*_k)=\textrm{Cov}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}Cov(clrYj∗,clrYk∗)=Cov(clrYj,clrYk) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{dCov}(\textrm{clr}Y^*_j, \textrm{clr}Y^*_k)=\textrm{dCov}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}dCov(clrYj∗,clrYk∗)=dCov(clrYj,clrYk). Since \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj is now a consistent estimator for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y^*_j$$\end{document}Yj∗ (not \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj), our test statistics still target at the desired dependencies of interest. Permutation-based inference Asymptotic inference using \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}\left( \textrm{mclr}P_j, \textrm{mclr}P_k\right)$$\end{document}Cov^mclrPj,mclrPk and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}\left( \textrm{mclr}P_j, \textrm{mclr}P_k\right)$$\end{document}dCov^mclrPj,mclrPk as test statistics may not work well for several reasons. Firstly, most microbiome studies have small sample sizes ranging from 50 to 200. Secondly, the distribution of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mclr}P_j$$\end{document}mclrPj values still tends to be skewed, although less skewed than the original relative abundances \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj. Thirdly, the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mclr}P_j$$\end{document}mclrPj distribution has a point mass resulting from the modification at zeros. Therefore, we rely on permutation for inference. The permutation procedure is based on shuffling the clr\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj values for each taxon, as these values most align with absolute abundances and are therefore independent across taxa under the null hypothesis. In contrast, shuffling raw read count data is inappropriate because read counts are positively correlated through shared and varying library sizes, even in the absence of true taxon-taxon dependence. The rationale for not shuffling the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mclr}P_j$$\end{document}mclrPj values will be explained later. Recall that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov(clrPj,clrPk) has a non-zero value \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1 + \mathcal {B}_2 + \mathcal {B}_3$$\end{document}B1+B2+B3 under the null hypothesis that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\log Z_j, \log Z_k)=0$$\end{document}Cov(logZj,logZk)=0. Therefore, the covariance calculated from the permutation replicates should have the same non-zero mean to ensure their validity. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ be the shuffled \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj across samples, where shuffling is done independently for different taxa to generate a permutation replicate. Unfortunately, the naive covariance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\widetilde{\textrm{clr}P_j}, \widetilde{\textrm{clr}P_k})=0$$\end{document}Cov(clrPj~,clrPk~)=0 due to the independence of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_k}$$\end{document}clrPk~ for any j≠ kj≠k. Note that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ across taxa no longer sums to zero as the original \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj does, so we re-center \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ to obtain a modified covariance3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} & \textrm{Cov}\left( \widetilde{\textrm{clr}P_j} - J^{-1}\sum \limits _{j'} \widetilde{\textrm{clr}P_{j'}}, ~~\widetilde{\textrm{clr}P_k} -J^{-1}\sum \limits _{j'} \widetilde{\textrm{clr}P_{j'}}\right) \nonumber \\= & -J^{-1}\textrm{Var} \left( \widetilde{\textrm{clr}P_j}\right) -J^{-1}\textrm{Var} \left( \widetilde{\textrm{clr}P_k}\right) + J^{-2}\sum \limits _{j'} \textrm{Var} \left( \widetilde{\textrm{clr}P_{j'}}\right) \nonumber \\= & -J^{-1}\textrm{Var} \left( \textrm{clr}P_j\right) -J^{-1}\textrm{Var} \left( \textrm{clr}P_k\right) + J^{-2}\sum \limits _{j'} \textrm{Var} \left( \textrm{clr}P_{j'}\right) , \end{aligned}$$\end{document}CovclrPj~-J-1∑j′clrPj′~,clrPk~-J-1∑j′clrPj′~=-J-1VarclrPj~-J-1VarclrPk~+J-2∑j′VarclrPj′~=-J-1VarclrPj-J-1VarclrPk+J-2∑j′VarclrPj′,where the last equation follows from the fact that shuffling preserves variance. Using (2), we can express (3) as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1+\mathcal {B}_3$$\end{document}B1+B3. This value nearly matches \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}(\textrm{clr}P_j, \textrm{clr}P_k)=\mathcal {B}_1+\mathcal {B}_2+\mathcal {B}_3$$\end{document}Cov(clrPj,clrPk)=B1+B2+B3 under the null hypothesis, except for the loss of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 term. Recall that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0 under the Identity covariance structure and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=O(J^{-1})$$\end{document}B2=O(J-1) in general cases satisfying the Sparse Dependence Assumption. Meanwhile, the sample covariance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk), which is closely associated with the final test statistic, has a standard error of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(n^{-0.5})$$\end{document}O(n-0.5). When n is small relative to J (e.g., n=200n=200 and J=100J=100), \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2$$\end{document}B2 becomes negligible compared to the variability of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrPj,clrPk). In other words, under the Sparse Dependence Assumption, the permutation replicates generated under the global null hypothesis, while preserving the zero-sum structure, serve as valid replicates for the observed data. Note that in each permutation, all \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj must be shuffled independently. If only \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj at taxon j is shuffled while all other taxa remain intact, the covariance after recentering becomes \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{Cov}\left\{ \widetilde{\textrm{clr}P_j} - J^{-1}(\widetilde{\textrm{clr}P_j}-\textrm{clr}P_j), ~~\textrm{clr}P_k -J^{-1}(\widetilde{\textrm{clr}P_j}-\textrm{clr}P_j)\right\} =-J^{-1}\textrm{Var} (\textrm{clr}P_j) +O(J^{-2})$$\end{document}CovclrPj~-J-1(clrPj~-clrPj),clrPk-J-1(clrPj~-clrPj)=-J-1Var(clrPj)+O(J-2), which does not align with the mean of the observed statistic, even under the Identity covariance structure where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0. Moveover, if the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{mclr}P_j$$\end{document}mclrPj values are shuffled, it becomes difficult to preserve the zero-sum structure at the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj level, making it challenging to match the means of the statistics. Finally, we expect the means to align similarly for the nonlinear test statistics, although deriving a closed-form expression for their means is unattainable. As done for the observed data, we modify the centered \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ data at zero entries by replacing them by their taxon-specific means, where the zero statuses are tracked during the shuffling of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj values. The centered and modified \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ data are denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{mclr}P_j}$$\end{document}mclrPj~, based on which we calculate the permutation statistics \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}\left( \widetilde{\textrm{mclr}P_j}, \widetilde{\textrm{mclr}P_k}\right)$$\end{document}Cov^mclrPj~,mclrPk~ and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}\left( \widetilde{\textrm{mclr}P_j}, \widetilde{\textrm{mclr}P_k}\right)$$\end{document}dCov^mclrPj~,mclrPk~. Finally, we obtain two-sided p-values for the linear tests and one-sided p-values for the nonlinear tests by comparing the permutation statistics with their observed counterparts. The shuffling scheme can be readily extended to accommodate clustered data. To preserve the similarity of samples within the same subjects (or clusters), the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj data from these samples should be shuffled as a whole across subjects and simultaneously shuffled within subjects. When subjects have unequal numbers of samples (i.e., cluster sizes), the shuffling should be stratified by subjects with the same cluster size. We expect the linear test based on Pearson’s covariance to perform better when the dependence between a pair of taxa is linear, and the nonlinear test based on distance covariance to perform better when the dependence is nonlinear. Since we do not know the dependence type a priori for every pair of taxa, we construct an omnibus test for testing a general dependence that combines the results from both tests for each pair. This omnibus test utilizes the existing permutation replicates. We use the minimum of the p-values obtained from the linear and nonlinear tests as the final test statistic and use the corresponding minima from the permutation replicates to simulate the null distribution [24], thereby obtaining the omnibus p-value. Given the large number of p-values for all pairs of taxa from each type of test, linear, nonlinear or omnibus, we aim to declare correlated pairs by controlling the false discovery rate (FDR). Evidently, these p-values are not independent due to not only the inherently pairwise nature of network analysis but also taxon-taxon correlations. Fortunately, the Benjamini-Hochberg (BH) procedure [25], commonly used to control the FDR, allows for positive dependence among test statistics corresponding to null hypotheses. Positive dependence means that obtaining a significant result in one test increases the likelihood of obtaining another significant test. Our test statistics exhibit this dependence structure because obtaining a significant result for a pair of uncorrelated taxa, such as taxa A and B, increases the likelihood of obtaining a significant result for another pair of uncorrelated taxa, such as taxa A and C, if taxa B and C are correlated, either positively or negatively. Therefore, we apply the BH procedure to detect correlated pairs of taxa. In fact, we use Sandve’s procedure [26], a variation of BH that incorporates a sequential stopping rule for permutation-based multiple testing. This rule substantially improves computational efficiency by stopping the addition of replicates for tests once a pre-specified minimum number of extreme replicates has been reached or if the estimated FDR falls below a given threshold. The complete workflow of TestNet is depicted in Fig. 8.Fig. 8Workflow of TestNet, illustrated with the linear test. Each box represents a matrix. The superscript ‘(1)’ denotes the permutation replicate number Workflow of TestNet, illustrated with the linear test. Each box represents a matrix. The superscript ‘(1)’ denotes the permutation replicate number Simulation studies To ensure a fair evaluation of all methods, we adopted the settings described in [9, 10] to generate the underlying absolute abundance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Z_j$$\end{document}Zj with various dependence structures. We considered the following seven covariance structures, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=\{\sigma _{jk}\}=\{\textrm{Cov}(\log Z_j, \log Z_k)\}$$\end{document}Σ={σjk}={Cov(logZj,logZk)}, for simulating linear dependencies, along with an additional dependence structure called U-shape for simulating nonlinear dependencies.(Identity) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=I_J$$\end{document}Σ=IJ. It represents the global null and is an ideal case as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0.(AR1+) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=0.5$$\end{document}σjk=0.5 if |j-k|=1|j-k|=1, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}=1$$\end{document}σjj=1.(AR1) Similar to AR1+, but set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.5$$\end{document}σjk=±0.5 if |j-k|=1|j-k|=1. The ± symbol indicates taking the positive and negative values with equal probabilities.(AR4) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.4$$\end{document}σjk=±0.4 if |j-k|=4|j-k|=4, ± 0.2±0.2 if |j-k|=3|j-k|=3 or 2, ± 0.1±0.1 if |j-k|=1|j-k|=1, and otherwise 0. Set the diagonal elements (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjj) large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite.(Sparse) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=\text {diag}(A_1, A_2)$$\end{document}Σ=diag(A1,A2), where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1=B+\epsilon I_{J_1}$$\end{document}A1=B+ϵIJ1, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_2=4I_{J-J_1}$$\end{document}A2=4IJ-J1, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J1=\lfloor \sqrt{J} \rfloor$$\end{document}J1=⌊J⌋. Here, B is a symmetric matrix with lower triangular entries sampled from the uniform distribution on [-1, -0.5] ∪ [0.5, 1][-1,-0.5]∪[0.5,1] with probability 0.3, and otherwise 0. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon =\max (-\lambda _{\min }(B), 0) + 0.01$$\end{document}ϵ=max(-λmin(B),0)+0.01, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _{\min }(B)$$\end{document}λmin(B) denotes the smallest eigenvalue of B, ensures that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1$$\end{document}A1 is positive definite.(Hub) Sample three hub taxa from the top 20 most abundant taxa. For a hub taxon j and any other taxon k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.7, and otherwise 0. For two non-hub taxa j and k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite.(Block) Divide the J taxa equally into 10 blocks. For each pair (j, k) within a block, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.5, and otherwise 0. For each pair (j, k) between blocks, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite.(U-shape) Group every two taxa together. Set a nonlinear, U-shaped correlation for the two taxa within each group, while keeping the taxa in different groups independent. (Identity) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=I_J$$\end{document}Σ=IJ. It represents the global null and is an ideal case as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0. (Identity) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=I_J$$\end{document}Σ=IJ. It represents the global null and is an ideal case as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_2=0$$\end{document}B2=0. (AR1+) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=0.5$$\end{document}σjk=0.5 if |j-k|=1|j-k|=1, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}=1$$\end{document}σjj=1. (AR1+) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=0.5$$\end{document}σjk=0.5 if |j-k|=1|j-k|=1, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}=1$$\end{document}σjj=1. (AR1) Similar to AR1+, but set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.5$$\end{document}σjk=±0.5 if |j-k|=1|j-k|=1. The ± symbol indicates taking the positive and negative values with equal probabilities. (AR1) Similar to AR1+, but set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.5$$\end{document}σjk=±0.5 if |j-k|=1|j-k|=1. The ± symbol indicates taking the positive and negative values with equal probabilities. (AR4) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.4$$\end{document}σjk=±0.4 if |j-k|=4|j-k|=4, ± 0.2±0.2 if |j-k|=3|j-k|=3 or 2, ± 0.1±0.1 if |j-k|=1|j-k|=1, and otherwise 0. Set the diagonal elements (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjj) large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (AR4) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.4$$\end{document}σjk=±0.4 if |j-k|=4|j-k|=4, ± 0.2±0.2 if |j-k|=3|j-k|=3 or 2, ± 0.1±0.1 if |j-k|=1|j-k|=1, and otherwise 0. Set the diagonal elements (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjj) large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (Sparse) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=\text {diag}(A_1, A_2)$$\end{document}Σ=diag(A1,A2), where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1=B+\epsilon I_{J_1}$$\end{document}A1=B+ϵIJ1, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_2=4I_{J-J_1}$$\end{document}A2=4IJ-J1, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J1=\lfloor \sqrt{J} \rfloor$$\end{document}J1=⌊J⌋. Here, B is a symmetric matrix with lower triangular entries sampled from the uniform distribution on [-1, -0.5] ∪ [0.5, 1][-1,-0.5]∪[0.5,1] with probability 0.3, and otherwise 0. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon =\max (-\lambda _{\min }(B), 0) + 0.01$$\end{document}ϵ=max(-λmin(B),0)+0.01, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _{\min }(B)$$\end{document}λmin(B) denotes the smallest eigenvalue of B, ensures that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1$$\end{document}A1 is positive definite. (Sparse) Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }=\text {diag}(A_1, A_2)$$\end{document}Σ=diag(A1,A2), where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1=B+\epsilon I_{J_1}$$\end{document}A1=B+ϵIJ1, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_2=4I_{J-J_1}$$\end{document}A2=4IJ-J1, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J1=\lfloor \sqrt{J} \rfloor$$\end{document}J1=⌊J⌋. Here, B is a symmetric matrix with lower triangular entries sampled from the uniform distribution on [-1, -0.5] ∪ [0.5, 1][-1,-0.5]∪[0.5,1] with probability 0.3, and otherwise 0. The term \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon =\max (-\lambda _{\min }(B), 0) + 0.01$$\end{document}ϵ=max(-λmin(B),0)+0.01, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _{\min }(B)$$\end{document}λmin(B) denotes the smallest eigenvalue of B, ensures that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_1$$\end{document}A1 is positive definite. (Hub) Sample three hub taxa from the top 20 most abundant taxa. For a hub taxon j and any other taxon k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.7, and otherwise 0. For two non-hub taxa j and k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (Hub) Sample three hub taxa from the top 20 most abundant taxa. For a hub taxon j and any other taxon k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.7, and otherwise 0. For two non-hub taxa j and k, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (Block) Divide the J taxa equally into 10 blocks. For each pair (j, k) within a block, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.5, and otherwise 0. For each pair (j, k) between blocks, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (Block) Divide the J taxa equally into 10 blocks. For each pair (j, k) within a block, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.5, and otherwise 0. For each pair (j, k) between blocks, set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jk}=\pm 0.3$$\end{document}σjk=±0.3 with probability 0.2, and otherwise 0. Set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _{jj}$$\end{document}σjjs large enough so that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ is positive definite. (U-shape) Group every two taxa together. Set a nonlinear, U-shaped correlation for the two taxa within each group, while keeping the taxa in different groups independent. (U-shape) Group every two taxa together. Set a nonlinear, U-shaped correlation for the two taxa within each group, while keeping the taxa in different groups independent. In the linear dependence settings, we generated the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj data with a given variance-covariance matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\boldsymbol{\Sigma }$$\end{document}Σ and a mean vector μμ (each element of μμ sampled from U[0, 10]) from either the Normal or Gamma distribution. For the Gamma distribution, we set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j=\mu + FU_j/\sqrt{20}$$\end{document}logZj=μ+FUj/20, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$FF^{\textrm{T}}=\boldsymbol{\Sigma }$$\end{document}FFT=Σ and the components of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$U_j$$\end{document}Uj are independent gamma variables with a shape parameter of 10 and a scale parameter of 1. In the U-shape dependence setting, we first generated the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj data independently for one taxon in each group, and then generated the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj data for the other taxon in the same group as the orthogonal polynomial of degree 2 from the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj data of the first taxon, plus an error term drawn from N(0, 0.05). The underlying relative abundance were calculated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j=Z_j/\sum _{j'=1}^J Z_{j'}$$\end{document}Yj=Zj/∑j′=1JZj′. In some cases, we simulated clustered data with four samples in each cluster. We generated each cluster-specific mean as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu _i=\mu + \epsilon _i$$\end{document}μi=μ+ϵi, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon _i\sim N(0,1)$$\end{document}ϵi∼N(0,1), and assigned this mean to all samples within the cluster. In other cases, we introduced experimental bias by calculating \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j=\exp (\gamma _j)Z_j/\sum _{j'=1}^J \exp (\gamma _{j'})Z_{j'}$$\end{document}Yj=exp(γj)Zj/∑j′=1Jexp(γj′)Zj′, where the bias factors \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _j$$\end{document}γjs were sampled from N(0, 0.5). Finally, we generated the read count data \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xj independently for each taxon, given \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj and the library size l, from the Negative-Binomial distribution with a mean of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$lY_j$$\end{document}lYj and an overdispersion parameter of 1. Here, l was sampled from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N(10000, (10000/3)^2)$$\end{document}N(10000,(10000/3)2) and left-truncated at 2000. This overdispersion value produced realistic proportions of zeros, similar to those observed in real microbiome data. The Dirichlet-Multinomial distribution was not used because it leads to negative correlations across taxa even when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj is fixed. It is important to note that this step of generating \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j$$\end{document}Xj was often missed in simulation studies conducted for existing methods such as COAT and CCLasso [8, 9], where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y_j$$\end{document}Yj was directly used as the observed relative abundance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_j$$\end{document}Pj to evaluate those methods. Consequently, those evaluations ignored the effect of sequencing variation. Throughout this work, the number of taxa J was set to 100 unless otherwise specified as 50. We considered these relatively small J values because our method performs better with larger J values. Prior to analysis, it is customary to filter out very rare taxa that are likely contaminants and increase the burden of multiple testing. We considered two filtering criteria. The stringent criterion filters out taxa with mean relative abundances less than 0.1%, which tends to reduce the number of taxa by 50%. The lenient criterion filters out taxa that are present in less than 5% of all samples, which often led to no exclusions in our simulated data. The stringent criterion was used as the default unless otherwise specified. Verifying the sparse dependence assumption Additional file 1: Figure S15 compares the two sides of the inequality in the Sparse Dependence Assumption, demonstrating that the assumption holds across all scenarios considered. The x-axis represents the sample variance of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj, calculated from read count data based on n=10000n=10000 samples, a large sample size to ensure accurate variance estimation. Examining the bias terms \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1$$\end{document}B1–\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_4$$\end{document}B4 Figure 9 compares the sample variances and covariances of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj (equivalently, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj), and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj, based on n=10000n=10000 samples. The upper panel shows that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\log Z_j, \log Z_k)\approx \widehat{\textrm{Cov}}(\textrm{clr}Z_j, \textrm{clr}Z_k)$$\end{document}Cov^(logZj,logZk)≈Cov^(clrZj,clrZk) for j≠ kj≠k and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Var}}(\log Z_j)\approx \widehat{\textrm{Var}}(\textrm{clr}Z_j)$$\end{document}Var^(logZj)≈Var^(clrZj), indicating that the bias \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_1+\mathcal {B}_2$$\end{document}B1+B2 is small, though noticeable in some cases such as AR1+. The middle panel shows that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cov}}(\textrm{clr}Y_j, \textrm{clr}Y_k)\approx \widehat{\textrm{Cov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cov^(clrYj,clrYk)≈Cov^(clrPj,clrPk), suggesting that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_3$$\end{document}B3 is small. However, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Var}}(\textrm{clr}P_j)\gg \widehat{\textrm{Var}}(\textrm{clr}Y_j)$$\end{document}Var^(clrPj)≫Var^(clrYj) in most cases, indicating that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {B}_4$$\end{document}B4 is large. The lower panel shows that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cor}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}Cor^(clrPj,clrPk) is significantly attenuated compared to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{Cor}}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}Cor^(clrYj,clrYk). Distance variances and covariances for measuring nonlinear dependencies are displayed in Additional file 1: Figure S18, showing similar comparison results, with the exception that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\textrm{dCov}}(\textrm{clr}P_j, \textrm{clr}P_k)$$\end{document}dCov^(clrPj,clrPk) tends to be smaller than \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{\mathrm{dCov}}(\textrm{clr}Y_j, \textrm{clr}Y_k)$$\end{document}dCov^(clrYj,clrYk).Fig. 9Upper panel: Sample Pearson’s covariances (in black) and variances (in blue) of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj. Middle panel: Sample Pearson’s covariances (in black) and variances (in blue) of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj. Lower panel: Sample Pearson’s correlations of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj. The red line represents the 45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘ reference line Upper panel: Sample Pearson’s covariances (in black) and variances (in blue) of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log Z_j$$\end{document}logZj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Z_j$$\end{document}clrZj. Middle panel: Sample Pearson’s covariances (in black) and variances (in blue) of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj. Lower panel: Sample Pearson’s correlations of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}Y_j$$\end{document}clrYj and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj. The red line represents the 45\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^{\circ }$$\end{document}∘ reference line Evaluating the modification of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj at zero entries Additional file 1: Figure S19 displays the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj values before and after the modification of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj at zero entries for two pairs of taxa, based on n=1000n=1000 samples. The upper panel demonstrates that for a pair of truly uncorrelated taxa, both of which have many zeros, the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\textrm{clr}P_j$$\end{document}clrPj values show a spurious Pearson’s correlation of 0.45, which reduces to merely −0.013 after the modification. The lower panel indicates that for a pair of truly correlated taxa, the modification has a minimal effect on the correlation measure, which is 0.14 before and 0.15 after modification. Assessing the validity of permutation replicates Additional file 1: Figure S20 compares the distributions of observed and permutation test statistics across all null taxon pairs based on one replicate of observed data and one permutation replicate. For inferring linear dependencies (upper panel), the observed statistics (i.e., sample Pearson’s covariances) show a slight shift to the left of zero with n=100n=100 samples, which becomes more evident with n=1000n=1000 samples. However, the corresponding permutation statistics without centering \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~ are centered at zero, creating a discrepancy from the observed statistics. After centering \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~, the distribution of the permutation statistics matches well with that of the observed statistics. For inferring nonlinear dependencies (lower panel), the match between the observed statistics (i.e., sample distance covariances) and the corresponding permutation statistics also occurs after centering \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widetilde{\textrm{clr}P_j}$$\end{document}clrPj~. This match was verified for all dependence structures and with both the stringent and lenient filters, as shown in Additional file 1: Figures S21–S24. Supplementary Information Additional file 1: Texts S1 and S2, Figures S1–S24 Additional file 1: Texts S1 and S2, Figures S1–S24