Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments

Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang · 2025

Read Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments by Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10331.txt.

Description

Zhenkang Peng and co-authors introduce a data-pooling framework for treatment selection in online experiments in a 2025 paper. Randomized experiments are the gold standard for causal claims, but business settings bring thin traffic, varied treatment effects, and overlapping experiments. These problems make treatment effect estimates noisy and policy roll-out risky.

The framework, called DPTR, pools data across experiments instead of treating each one alone. It handles overlapping and non-overlapping traffic, with linear or nonlinear models. Theory shows it beats standard difference-in-means and least squares methods when many experiments run without overlap. Simulations and two real platform datasets confirm its value, giving product teams a steadier way to pick which treatment to roll out.

Catalog record

CREATORZhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE131 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10331

Browse the full library · About the Tower of Babel