Exploration, Confirmation, and Replication in the Same Observational Study: A Two Team Cross-Screening Approach to Studying the Effect of Unwanted Pregnancy on Mothers’ Later Life Outcomes
Samrat Roy, Department of Statistics, Data Science, University of Pennsylvania · 2025
Read Exploration, Confirmation, and Replication in the Same Observational Study: A Two Team Cross-Screening Approach to Studying the Effect of Unwanted Pregnancy on Mothers’ Later Life Outcomes by Samrat Roy, Department of Statistics, Data Science, University of Pennsylvania (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14480.txt.
Introduction
Studying cause and effect in human lives often means working with a single precious dataset, where peeking at the data can bias the verdict. Samrat Roy and co-authors propose a discipline for that bind in this 2025 paper, called two team cross-screening. They apply it to the long-term effects of unwanted pregnancies carried to term, using a Wisconsin study that followed mothers for decades.
The researchers split into two teams, and the data split along a meaningful trait. Each team explored its own half freely, then wrote the analysis plan for the other half. Only ideas that looked promising got forwarded, which trims false leads. Outcomes covered mental and physical health, money, and life satisfaction in later years.
Catalog record
| CREATOR | Samrat Roy, Department of Statistics, Data Science, University of Pennsylvania |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14480
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploration, Confirmation, and Replication in the Same Observational Study: A Two Team Cross-Screening Approach to Studying the Effect of Unwanted Pregnancy on Mothers’ Later Life Outcomes Thanks: This work was supported by Israeli Science Foundation under Grant 406/24. Samrat Roy, Department of Statistics, Data Science, University of Pennsylvania Abstract The long-term consequences of unwanted pregnancies carried to term on mothers have not been explored much. We use data from the Wisconsin Longitudinal Study (WLS) and propose a novel approach, namely two team cross-screening, to study the possible effects of unwanted pregnancies carried to term on various aspects of mothers’ later life mental health, physical health, economic well-being, and life satisfaction. Our approach, unlike existing approaches to observational studies, enables investigators to perform exploratory data analysis, confirmatory data analysis, and replication in the same study. This is a valuable property when there is only one data set available with unique strengths. In two team cross-screening, the investigators split themselves into two teams and the data is split as well according to a meaningful covariate. Each team then performs an exploratory data analysis on its part of the data to design an analysis plan for the other part of the data. The complete freedom of the teams in designing the analysis has the potential to generate new unanticipated hypotheses in addition to a prefixed set of hypotheses. Moreover, only the hypotheses that looked promising in the data each team explored are forwarded for analysis (thus alleviating the multiple testing problem). These advantages are demonstrated in our study of the effects of unwanted pregnancies on mothers’ later life outcomes. Keywords: Causal Inference, Multiple Testing, Sample Splitting, Replicability. 1 Introduction 1.1 An Observational Study Assessing the Effect of Unwanted Pregnancies on Mothers’ Later Life Outcomes Unwanted pregnancies are prevalent worldwide ( Bearak et al., 2018 ) . For unwanted pregnancies carried to term, it is crucial to assess their effect on the child who is born and the mother to design policies that best support the child and the mother. For the mother, there is evidence of short-term effects such as less emotional attachment to her baby during pregnancy, greater parental stress at 6 months and one year postpartum, and less effective parenting strategies ( Pakseresht et al., 2018 ) . However, the long term effects have not been studied much. The only study we are aware of is Herd et al. (2016) that used linear regression to examine one aspect of mental health, depression. In this work, our aim is to study the possible effects of an unwanted pregnancy carried to term on various aspects of a mother’s later life mental health, physical health, economic well-being, and life satisfaction. A data set that has unique strengths to carry out such an observational study is the Wisconsin Longitudinal Study (WLS) ( Herd et al., 2014 ) . The WLS randomly sampled one-third of people graduating from high school in Wisconsin in 1957 and has followed them longitudinally every decade since. Strengths of the WLS include: – The WLS asked if the pregnancy was wanted somewhat contemporaneously at about 36 years of age. Women who had experienced pregnancies by the age of 36 were asked for each pregnancy: ‘Before you became pregnant, did you want to become pregnant at that time?’ and if the woman answered no, ‘Did you want to have another baby sometime?”. If the woman answered no to both questions, it was coded as an unwanted pregnancy. This way of identifying an unwanted pregnancy when the woman is of age 36 is better than asking a woman when a child is completely grown up, but not as good as asking a woman about her pregnancy intention prior to her pregnancy, as her recall could be colored by the events of the intervening years between the pregnancy and the time the woman turned 36. However, we proceed with this way of identification due to the lack of any other reliable data sources. – Prospectively before the birth of the participants’ children, the WLS measured a wide range of variables that could confound the relationship between unwanted pregnancies and later life outcomes. These variables include family history, adolescent characteristics, educational and occupational achievements, and aspirations. – The WLS has long-term longitudinal follow-up of its respondents when they were around age 53, 65 and 72, allowing the study of later life outcomes. Note that women in the WLS experienced most of their pregnancies before the 1973 Roe v. Wade Supreme Court decision that made abortion legal throughout the United States. Before 1973, abortion was illegal in Wisconsin. Thus, this dataset is most relevant to the question of what is the effect of unwanted pregnancy in places where abortion is illegal. This is an important current question because abortion is currently completely illegal in 24 countries, legal only when the women’s health is at risk in 37 other countries, and illegal in some U.S. states following the 2022 Dobbs v. Jackson decision. 1.2 Exploratory Data Analysis, Confirmatory Data Analysis and Replication in Observational Studies Exploratory data analysis (EDA), confirmatory data analysis, and replication are three important aspects of building strong evidence from observational studies. EDA helps form hypotheses that might not have been anticipated and remove outliers that do not represent natural variation in the data ( Tukey, 1977 ) . It can also help identify a variable that does not measure what one thought it did and may suggest a different variable that better measures the construct that one is actually interested in. John Tukey coined the term EDA and developed many foundational techniques for it, but he also stressed the importance of confirmatory data analysis. Likening EDA to detective work about a crime and confirmatory data analysis to a scientific trial of the hypotheses suggested by EDA ( Tukey, 1977 ) , Tukey said ( Tukey, 1980 ) : “Important questions can demand the most careful planning for confirmatory analysis.” Replication of results is also important for science. Susser (1973) emphasized the importance of replication in observational studies, defining replication as consistency with respect to different “times, circumstances and people.” Replicating an observational study is particularly helpful when the concerns about hidden bias in the original and replicated studies are different ( Rosenbaum, 2001 ) . One way this can happen is when the reasons for receiving treatment in the two studies differ ( Rosenbaum, 2001 ; Rosenbaum, 2015 ) . For example, if a study in New York City found that fish eaters live longer than non-fish eaters, then repeating the study in Philadelphia would provide limited additional information as both studies could be biased because fish is a comparatively expensive food in cities, favored by those seeking to maintain a healthy diet ( Rosenbaum, 2015 ) ; a more helpful replication would be the study where Lund and Bønaa (1993) compared wives of fishermen in Norway to wives of other workers of similar income. The reasons why some subjects in the New York City study eat more fish than others (e.g., the fish eaters have higher income and are more health conscious) differ from the reasons why some subjects in the Norway study eat more fish than others (e.g., the wives of