More Permutations Do Not Always Increase Power: Non-monotonicity in Monte Carlo Permutation Tests

Suman Cha, Seongchan Lee, Antonin Schrab, Ilmun Kim, [0.6em] Department of Statistics, Data Science, Yonsei University, Seoul, South Korea, [0.4em] These authors contributed equally to this work. · 2026

Read More Permutations Do Not Always Increase Power: Non-monotonicity in Monte Carlo Permutation Tests by Suman Cha, Seongchan Lee, Antonin Schrab, Ilmun Kim, [0.6em] Department of Statistics, Data Science, Yonsei University, Seoul, South Korea, [0.4em] These authors contributed equally to this work. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03886.txt.

Description

Suman Cha, Seongchan Lee, Antonin Schrab, and Ilmun Kim challenge a common intuition about Monte Carlo permutation tests, a cornerstone of model-free statistical inference. Practitioners widely assume that sampling more permutations improves a test, in particular that statistical power tends to grow with the Monte Carlo budget. The authors show that intuition is false in general.

Leveraging the saw-toothed structure that power inherits from the discreteness of permutation distributions, they give a simple structural explanation for why power can decrease as more permutations are sampled. They then prove that such decreases occur infinitely often as the Monte Carlo budget grows. The result is a caution for practice: enlarging the simulation budget does not guarantee a uniformly better permutation test.

Catalog record

CREATORSuman Cha, Seongchan Lee, Antonin Schrab, Ilmun Kim, [0.6em] Department of Statistics, Data Science, Yonsei University, Seoul, South Korea, [0.4em] These authors contributed equally to this work.
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03886

Browse the full library · About the Tower of Babel