Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments
Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang · 2025
Read Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments by Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10331.txt.
Description
Zhenkang Peng and co-authors introduce a data-pooling framework for treatment selection in online experiments in a 2025 paper. Randomized experiments are the gold standard for causal claims, but business settings bring thin traffic, varied treatment effects, and overlapping experiments. These problems make treatment effect estimates noisy and policy roll-out risky.
The framework, called DPTR, pools data across experiments instead of treating each one alone. It handles overlapping and non-overlapping traffic, with linear or nonlinear models. Theory shows it beats standard difference-in-means and least squares methods when many experiments run without overlap. Simulations and two real platform datasets confirm its value, giving product teams a steadier way to pick which treatment to roll out.
Catalog record
| CREATOR | Zhenkang Peng, Chengzhang Li, Ying Rong, Renyu (Philip) Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 131 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10331