Prior-Free Sample Size Design for Test-and-Roll Experiments

Kentaro Kawato, Shosei Sakaguchi · 2026

Read Prior-Free Sample Size Design for Test-and-Roll Experiments by Kentaro Kawato, Shosei Sakaguchi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02414.txt.

Description

Kentaro Kawato and Shosei Sakaguchi study how large a trial should be when the winner will be rolled out to everyone else. In a test-and-roll design, a decision maker tests on some units, then gives the rest the treatment that did better. A bigger trial improves the final choice, but it also leaves more trial units stuck with the weaker option for a time.

The pair argue that a standard worst-case regret rule can shrink trials too far. They propose a rule that weighs the worst-case gain from adding one more matched pair against its cost. For both simple yes-no outcomes and measured outcomes, the analysis points to a clear benchmark: the trial should cover about a third of the group. No prior belief about effects is needed.

Catalog record

CREATORKentaro Kawato, Shosei Sakaguchi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02414

Browse the full library · About the Tower of Babel