Surrogate-Based Prevalence Measurement for Large-Scale A/B Testing DOI: XXXXXXX.XXXXXXX Conference: The 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 9–13, 2026; Jeju, Korea Zehao Xu, Tony Paek, Kevin O’Sullivan, Attila Dobi Abstract . Online media platforms often need to measure how frequently users are exposed to specific content attributes in order to evaluate trade-offs in A/B experiments. A direct approach is to sample content, label it using a high-quality rubric (e.g., an expert-reviewed LLM prompt), and estimate impression-weighted prevalence. However, repeatedly running such labeling for every experiment arm and segment is too costly and slow to serve as a default measurement at scale. We present a scalable surrogate-based prevalence measurement framework that decouples expensive labeling from per-experiment evaluation. The framework calibrates a surrogate signal to reference labels offline and then uses only impression logs to estimate prevalence for arbitrary experiment arms and segments. We instantiate this framework using score bucketing as the surrogate: we discretize a model score into buckets, estimate bucket-level prevalences from an offline labeled sample, and combine these calibrated bucket level prevalences with the bucket distribution of impressions in each arm to obtain fast, log-based estimates. Across multiple large-scale A/B tests, we validate that the surrogate estimates closely match the reference estimates for both arm-level prevalence and treatment–control deltas. This enables scalable, low-latency prevalence measurement in experimentation without requiring per-experiment labeling jobs. 1. Introduction Modern recommender systems and media platforms must balance user engagement against the need to manage exposure to certain content attributes. Teams often summarize such exposure objectives in terms of prevalence : the fraction of impressions associated with a given target category. At the same time, product decisions are largely driven by large-scale A/B experiments ( 8 ) , where variants adjust ranking or filtering and are evaluated on both engagement and attribute-specific exposure metrics. One way to measure prevalence is to sample contents from traffic, label it with LLMs using expert-validated prompts, and apply an estimation. In our setting, we use PPSWOR (probability-proportional-to-size without replacement) sampling ( 7 ) and the Hansen–Hurwitz estimator ( 6 ) to obtain high-quality and unbiased measurements. However, directly using LLM-based prevalence as a default metric for every experiment is impractical. Running a separate LLM job per arm and per segment is expensive, and quickly becomes infeasible on a platform with hundreds of concurrent experiments. Moreover, experimenters often care about relatively small but meaningful changes in prevalence. A single LLM measurement per arm tends to emphasize the absolute level, so many small treatment–control deltas appear statistically non-significant. Running LLM labeling per experiment, per segment, and per day to address this would further amplify cost and latency. We address these challenges with a ML-score surrogate method that reuses a single LLM-based calibration of model scores across many experiments. We discretize model scores into buckets, estimate bucket-level prevalences from an offline calibration sample, and then combine these with the observed distribution of impressions over buckets in each experiment arm to obtain fast, log-based prevalence estimates. We integrate this method into a large-scale experiment platform, and augment it with a day-level aggregation that focuses on treatment and control delta, improving sensitivity to small shifts. Our approach is closely related to surrogate-outcome methods, where the quantity of interest is not directly observed but inferred using an intermediate signal ( 1 ) . Such methods are commonly used when long-run outcomes are costly or slow to measure in experiments, and only short-run outcomes are readily observed. In our setting, direct daily LLM-based prevalence measurement is accurate but operationally impractical; the ML-score therefore serves as a surrogate for prevalence. A key advantage of our setting is that we can directly validate surrogate quality by comparing surrogate-based estimates to LLM-based prevalence estimates on the same experiments. 2. Prevalence Estimation We consider the problem of estimating the prevalence of a category k on a large-scale media platform. Let \mathcal{K}=\{\text{Food-Recipe},\text{Lawn and Garden},\text{Gen-AI Generated},...\} denote the set of content-attributes considered in this study, and fix a particular k\in\mathcal{K} . Our target quantity is the prevalence of category k within S – which can denote the full population or a specific subset such as an experiment arm (control, treatment), a user demographic (country, age), an app surface, or intersections of these. The engineering details of our production prevalence pipeline are described in Farooq et al. ( 5 ) . In this section, we briefly recap the core estimator and introduce notation that we will use throughout the rest of the paper, in particular when we describe the ML-score surrogate method and its calibration. 2.1. Notation We consider a large population of content items, indexed by i=1,\dots,N . For each item i , we observe: • I_{i} : the total number of impressions of item i over a given time window. • Z_{i,k}\in\{0,1\} : a label indicating whether item i is truly in category k ( 1 ) or not ( 0 ). For a given segment S , let \mathcal{D}(S)\subseteq\{1,\dots,N\} denote the set of items that receive impressions from S , and let I_{i}(S) be the number of impressions of item i from S over the time window of interest. The total impressions of item i satisfy I_{i}=\sum_{S}I_{i}(S) . The prevalence of category k in segment S can be written as: (1) \displaystyle\mathcal{P}_{k}(S) \displaystyle=\mathbb{P}\!\left(Z_{k}=1\mid\text{impressions}\in S\right) \displaystyle=\frac{\sum_{i\in\mathcal{D}(S)}Z_{i,k}I_{i}(S)}{\sum_{i\in\mathcal{D}(S)}I_{i}(S)}, When S is the entire population, \mathcal{D}(S)=\{1,\dots,N\} and I_{i}(S)=I_{i} , recovering the global prevalence definition. 2.2. Sampling Design and Hansen–Hurwitz Estimator Directly labeling all items in \mathcal{D}(S) (often billions of items across many segments) is infeasible, so we estimate these quantities from a sample. We adopt a PPSWOR design, in which items are sampled with probabilities proportional to a chosen size measure. Compared with simple random sampling, PPSWOR allows us to oversample items that are more informative (e.g., high-impression items), achieving similar statistical power with a much smaller sample size. For category k and segment S , each item i\in\mathcal{D}(S) is assigned a sampling weight (2) w_{i,k}(S)\propto f\bigl(I_{i}(S)\bigr), and is selected into the sample with probability (3) p_{\text{sample},i,k}(S)\;=\;\frac{w_{i,k}(S)}{\sum_{j\in\mathcal{D}(S)}w_{j,k}(S)}. Here f is a function of impressions; it can be I_{i}(S) itself, or a combination such as I_{i}(S) multiplied by other factors (e.g., model score). We discuss concrete choices of f in more detail in Section 5.1 . Suppose we draw a sample of size n from \mathcal{D}(S) and obtained sampled indices \{i_{1},\dots,i_{n}\} . The Hansen–Hurwitz estimator for the total target-category impressions in S is (4) \widehat{T}_{k}(S)\;=\;\frac{1}{n}\sum_{t=1}^{n}\frac{Z_{i_{t},k}I_{i_{t}}(S)}{p_{\text{sample},i_{t},k}(S)}. The prevalence in segment S is then estimated as (5) \widehat{\mathcal{P}}_{k}(S)\;=\;\frac{\widehat{T}_{k}(S)}{\sum_{j\in\mathcal{D}(S)}I_{j}(S)}. In practice, both \sum_{j\in\mathcal{D}(S)}I_{j}(S) and \sum_{j\in\mathcal{D}(S)}w_{j,k}(S) are computed directly from the underlying logs for segment S . 2.3. Weighted Reservoir Sampling To efficiently draw a PPSWOR sample at scale, we use weighted random sampling with a reservoir ( 4 ) . Each item i is assigned an index (6) U_{i}^{1/w_{i,k}(S)}, where U_{i}\sim\mathrm{Uniform}(0,1) and w_{i,k}(S) is the sampling weight. The n items with the largest index are selected into the sample. This procedure implements PPSWOR in a single streaming pass, without the need to know N in advance. 2.4. LLM Labeling A central design choice in our prevalence pipeline is the source of “ground-truth” labels for whether an item belongs to a category k\in\mathcal{K} . In principle, there are three broad options: • Heuristic or rule-based labels (including model-score thresholds). For example, one might classify an item as positive if its category-specific score exceeds a fixed threshold. This approach is easy to deploy, but generally the model’s precision and recall are often not sufficient to serve as a reference standard , especially when policies evolve. More fundamentally, any measurement that reuses the same signals or thresholds that are already used for enforcement risks “grading our own homework” and systematically underestimating true prevalence ( 9 ) . • Human expert annotations (specialists or trained raters), which provide very high quality labels but do not scale: obtaining tens of thousands of fresh labels per category would be prohibitively slow and costly ( 13 ) . • LLM-based labeling (prompted large language models, optionally with expert-reviewed prompts and calibration), which offers a pragmatic middle ground: substantially higher fidelity than simple heuristics or raw score thresholds, at a fraction of the cost and latency of full human annotation. As a result, prompted LLMs are increasingly used as scalable labelers/judges in practice ( 12 ; 15 ; 10 ) . For each sampled item i , the LLM produces a binary label Z_{i,k}\in\{0,1\}, where Z_{i,k}=1 indicates that the item is judged to belong to category k , and Z_{i,k}=0 otherwise. Substituting these labels into the Hansen–Hurwitz estimator ( 4 ) and the prevalence formula ( 5 ) yields an unbiased estimate of category prevalence for the platform or for any base population over which the sampling is defined. In practice, we use this LLM-based estimator as the reference standard against which we calibrate our ML-score surrogate method in Section 4 . 3. Methodology: ML Score Surrogate Prevalence The LLM-based estimator provides high-quality measurements of category prevalence, but it requires running an LLM labeling job whenever we want a new estimate. This is acceptable for a small number of global or on-demand studies, but becomes problematic when we attempt to use prevalence as a default experiment metric at scale—for example, to (i) measure prevalence deltas for many experiments per day, or (ii) estimate prevalence differences across multiple user or traffic segments within a single experiment. There are two main challenges. Cost. Even a one-time measurement with a sample of roughly tens of thousands items can incur non-trivial labeling cost. Scaling this approach to millions of items across many experiments and segments quickly becomes prohibitively expensive. Latency. We support both real-time and batch (bulk) LLM labeling. Real-time labeling is significantly more expensive per item than bulk labeling, while bulk jobs typically take on the order of 10–24 hours from submission to completion. With many concurrent experiments and segments, end-to-end latency can be even longer, and heavy usage can also stress LLM API capacity and stability. These limitations motivate the need for an alternative approach that can: • avoid per-experiment LLM labeling, • provide low-latency estimates suitable for always-on monitoring, and • scale to many experiments and segments at near-log-query cost. To address this, we adopt a surrogate-based measurement approach that decouples expensive labeling from per-experiment evaluation. In this paper we instantiate the surrogate using model-score bucketing, yielding a score-bucket proxy prevalence estimator that can be computed from impression logs for arbitrary cohorts at query time. 3.1. Bucket-Level Prevalence Let m_{i,k}\in[0,1] be the model score for category k on item i ; higher values indicate that item i is more likely to be true in category k . We discretize the model scores for into B buckets: \mathcal{B}=\{b_{1},\dots,b_{B}\}, where each bucket b_{j} corresponds to an interval b_{1}=[u_{0},u_{1}),\;b_{2}=[u_{1},u_{2}),\;\dots,\;b_{B}=[u_{B-1},u_{B}], with 0=u_{0}0.3 ) each contain only about 2% of the mass. With a sample of 10{,}000 items, fewer than 200 would lie in each higher bucket, limiting statistical power for estimating \mathcal{P}_{k,b} at the upper end of the score range. Using model score as auxiliary information ( 14 ) , i.e., w_{i,k}\propto I_{i}m_{i,k} , mitigates this issue by boosting the inclusion probability of higher-score items. On the same underlying dataset, as illustrated in Figure 2 (b), this design yields a much more balanced bucket distribution: every bucket has at least \sim 7% of the sample, and the (0.9,1] buckets account for roughly 16% each. This improves the statistical power of higher bucket-level prevalence estimates with minimal operational overhead. (a) Impression-weighted sampling ( w_{i,k}\propto I_{i} ). (b) Impression \times score sampling ( w_{i,k}\propto I_{i}m_{i,k} ). Figure 2 . Comparison of score-bucket distributions under two sampling schemes. Left: sampling with weights proportional to impressions alone produces a very low-score-heavy sample, leaving few examples in high-score buckets. Right: sampling with weights proportional to impression \times model score yields a more balanced distribution across buckets, improving the precision of bucket-level prevalence estimates. 5.2. Online Integration 5.2.1. Deterministic SQL Metric For integration with the experiment platform, we first implement a deterministic end-to-end SQL metric. The key idea is that the platform fills in a small set of parameters for each experiment and runs a single SQL query that: (1) reads the latest calibration table \{\widehat{\mathcal{P}}_{k,b}\}_{b\in\mathcal{B}} , (2) pulls model scores of category k and daily impression logs for the experiment arms, (3) aggregates impressions to obtain bucket-level shares c_{k,b}(S) per arm per day, and (4) computes arm-level prevalences and confidence intervals analytically. For a given experiment, the metric is parameterized by: • Inputs: – exp_name , exp_version , – day d , – experiment groups (e.g., control , treatment ). • Implementation: the experiment platform populates these variables into a templated SQL query, then executes the query once per experiment per segment per day. • Outputs: – daily \widehat{\mathcal{P}}_{k}(S;d) per arm, – confidence intervals per arm (derived from \mathrm{Var}(\widehat{\mathcal{P}}_{k}(S;d)) ), – arm-level differences \Delta(d)=\widehat{\mathcal{P}}_{k}(\mathrm{treatment};d)-\widehat{\mathcal{P}}_{k}(\mathrm{control};d),\quad d=1,\dots,D, and associated z -scores and p -values. 5.2.2. Simulation-Based Extensions If a purely deterministic SQL implementation is infeasible, or if we wish to express prevalence as a user-level metric, we can add a simulation layer on top of the score-bucket calibration. Algorithm 1 shows how; for a given day d , (1) we draw one calibrated prevalence \mathcal{P}_{k,b}^{\star} per score bucket from the logit-normal calibration, (2) simulate which impressions are flagged at the impression level, and (3) aggregate to obtain daily prevalence estimates for the control and treatment arms. Algorithm 1 ML-score surrogate Prevalence (Single Calibration Draw) 1: input: day d ; category k ; user u ; item i and its model score m_{i,k} ; calibration table \{\widehat{\mathcal{P}}_{k,b},\text{Var}(\widehat{\mathcal{P}}_{k,b})\}_{b\in\mathcal{B}} ; impression logs \mathrm{imp}_{d,u,i} ; experiment groups g(u)\in\{\mathrm{control},\mathrm{treatment}\} 2: output: daily prevalences \widehat{\mathcal{P}}_{k}(\mathrm{control};d) , \widehat{\mathcal{P}}_{k}(\mathrm{treatment};d) , and daily effect \Delta(d) 3: 4: /* Step 1: draw one calibrated prevalence per bucket */ 5: for each bucket b\in\mathcal{B} do 6: \mu_{k,b}\leftarrow\ln\!\left(\widehat{\mathcal{P}}_{k,b}/(1-\widehat{\mathcal{P}}_{k,b})\right) 7: \sigma_{k,b}\leftarrow\sqrt{\mathrm{Var}(\widehat{\mathcal{P}}_{k,b})}\big/\bigl(\widehat{\mathcal{P}}_{k,b}(1-\widehat{\mathcal{P}}_{k,b})\bigr) \triangleright Delta-Method ( 11 ) variance on logit scale 8: draw z_{k,b}\sim\mathcal{N}(0,1) \triangleright e.g., Box–Muller ( 2 ) 9: \theta_{k,b}\leftarrow\mu_{k,b}+\sigma_{k,b}z_{k,b} 10: \mathcal{P}_{k,b}^{\star}\leftarrow\exp(\theta_{k,b})/\bigl(1+\exp(\theta_{k,b})\bigr) 11: 12: /* Step 2: attach buckets and \mathcal{P}_{k,b}^{\star} to impressions */ 13: for each log entry (u,i) on day d do 14: b\leftarrow\mathrm{bucket}(m_{i,k}) \triangleright map score to bucket 15: I_{d,u,i}\leftarrow\sum\mathrm{imp}_{d,u,i} \triangleright aggregate impressions of item i for user u 16: attach \mathcal{P}_{k,b}^{\star} to (d,u,i) 17: 18: /* Step 3: simulate target-category impressions per user */ 19: for each user u do 20: I_{d,u}\leftarrow\sum_{i}I_{d,u,i} 21: \mathrm{V}_{d,u}\leftarrow 0 \triangleright target-category impressions of item for user u 22: for each item i shown to user u on day d do 23: for t=1 to I_{d,u,i} do \triangleright loop over impressions of i 24: draw r\sim\mathrm{Uniform}(0,1) 25: if r<\mathcal{P}_{k,\mathrm{bucket}(i)}^{\star} then 26: V_{d,u}\leftarrow V_{d,u}+1 27: 28: /* Step 4: aggregate to group-level prevalences */ 29: for each group g\in\{\mathrm{control},\mathrm{treatment}\} do 30: I_{d,g}\leftarrow\sum_{u:\,g(u)=g}I_{d,u} 31: V_{d,g}\leftarrow\sum_{u:\,g(u)=g}V_{d,u} 32: \widehat{\mathcal{P}}_{k}(g;d)\leftarrow V_{d,g}/I_{d,g} 33: 34: /* Step 5: daily effect */ 35: \Delta(d)\leftarrow\widehat{\mathcal{P}}_{k}(\mathrm{treatment};d)-\widehat{\mathcal{P}}_{k}(\mathrm{control};d) In Step 1, we convert the bucket-level estimate and variation into a logit-normal distribution and draw a single \mathcal{P}_{k,b}^{\star} per bucket. In Steps 2–4, we attach these \mathcal{P}_{k,b}^{\star} values to the impression logs, run a Monte Carlo simulation at the impression level, and aggregate by user and group to obtain day- d prevalences \widehat{\mathcal{P}}_{k}(\mathrm{control};d) and \widehat{\mathcal{P}}_{k}(\mathrm{treatment};d) . In practice, this yields good point estimates for each arm per category per day 6. Discussion We conclude by discussing two practical questions that arose in deploying the ML-score surrogate method: (i) how to interpret and claim wins for small prevalence shifts in experiments, and (ii) how to choose an appropriate bucketization of model scores. 6.1. Detecting Small Shifts with Statistical Power 6.1.1. Calibration-Dominated Uncertainty in Arm-Level Estimates The variability of the ML-score surrogate estimates has two conceptually distinct components: (1) Global calibration variation at the bucket level, captured by \mathrm{Var}(\widehat{\mathcal{P}}_{k,b}) , and (2) Experiment-level variation in bucket mixes c_{k,b}(S) across arms and days. With a sufficiently large calibration sample (e.g., millions of labeled items per bucket), the calibration variation could be made negligible. In practice, however, our calibration budget is constrained by LLM labeling cost, so we work with much smaller bucket-wise samples. Under these realistic calibration sizes, our analytic variance is dominated by the bucket-level calibration variation. As a result, the corresponding confidence intervals for \widehat{\mathcal{P}}_{k}(\mathrm{control}) and \widehat{\mathcal{P}}_{k}(\mathrm{treatment}) are best interpreted as answering: “If we repeated the global LLM calibration many times, where would the absolute prevalence for this arm lie?” For exploratory experiments such as Experiment A (Section 4.1 , used for diagnostic validation rather than launch decisions), where the intervention intentionally produces a large shift in the impression share of the target categories, the calibration-driven uncertainty does not materially affect conclusions: the resulting prevalence changes are large (e.g., 20% or more) and remain clearly detectable even under conservative uncertainty estimates. In most production settings, however, changes are expected to be more incremental because we must balance content-shifting objectives against user engagement and other product constraints. In these more subtle experiments, where a 2%–5% relative change in prevalence would already be meaningful, calibration uncertainty can dominate arm-level confidence intervals and cause small but systematic treatment effects to appear statistically non-significant when evaluated solely using absolute per-arm estimates. 6.1.2. Delta-Focused Inference via Day-Level Sign Tests For evaluating experiment impact, we are usually more interested in the difference between arms than in their absolute levels. In that setting it is natural to focus on experiment-level variation. Concretely, we first compute daily prevalences and the associated daily effects \Delta(d) using the ML-score surrogate with a fixed calibration snapshot. Then aggregates \{\Delta(d)\}_{d=1}^{D} across days to produce: • an empirical 95% confidence interval [\Delta_{0.025},\Delta_{0.975}] from the quantiles of \{\Delta(d)\} , and • a sign-test p -value ( 3 ) based on the fraction of days where \Delta(d) is positive vs. negative or a normalized p -value. • if significant, the observed mean effect \bar{\Delta} is unlikely to be explained by random variation under the null hypothesis; otherwise, we fail to reject the null. This answers the question: “Given the experiment’s traffic and a fixed calibration, how precisely can we estimate the difference between arms?” In our deployments, this approach has proven useful for detecting small but consistent shifts in prevalence, even when the per-arm absolute intervals are relatively wide due to calibration uncertainty. Consider an experiment (Experiment C) that targets a subcategory of category k_{1} . Because this subcategory accounts for only a small fraction of the overall k_{1} impression share, we expect any change in \mathcal{P}_{k_{1}} to be modest but systematic. Using the LLM-based estimator aggregated over the entire experiment window, the estimated prevalences are: \widehat{\mathcal{P}}_{k_{1}}(\mathrm{control})=3.83\%\ [3.69\%,3.96\%], and \widehat{\mathcal{P}}_{k_{1}}(\mathrm{treatment})=3.67\%\ [3.53\%,3.80\%], with overlapping 95% confidence intervals and a two-sided p\text{-value}\approx 0.31 , no significant change. However, given the experimental design, a small shift in exposure to the target subcategory is expected. Using the ML-score surrogate, we can compute daily deltas \Delta(d) . The average delta over the experiment window is around -0.16\% ; relative to a baseline prevalence of about 3.8\% , this corresponds to a \sim 4.1\% relative reduction. As shown in Figure 3 , the daily relative reductions are consistently negative. Aggregating the day-level deltas yields a p -value close to 0 , indicating a statistically significant shift at the experiment level. As a sanity check and to guard against false positives, we also apply the same day-level aggregation to Experiment B (Subsection 4.2 ), where no prevalence shift is expected. In that case, p\text{-value}_{k_{1}}\approx 0.75 and p\text{-value}_{k_{2}}\approx 0.25 , aligning with our expectations. Figure 3 . Relative prevalence reduction vs. calendar day for category k_{1} in Experiment C. The values are consistently negative over the experiment window, and the day-level aggregation yields a statistically significant p -value. 6.2. Choice of Score Buckets The ML-score surrogate relies on a discretization of the model score into buckets \mathcal{B}=\{b_{1},\dots,b_{B}\} , and the choice of bucketization has direct implications for both bias and variance. There are two main trade-offs: (1) Bias vs. resolution. Fewer, wider buckets reduce variance in \widehat{\mathcal{P}}_{k,b} but can blur meaningful differences in the score–label relationship, especially in regions where the conditional prevalence changes rapidly with the score. Conversely, many narrow buckets provide finer resolution but require more calibration labels per bucket and can increase variance. (2) Score distribution skew. On real platforms, model scores are typically highly skewed toward low values. With equal-width buckets (e.g., ten buckets over [0,1] ), most impressions fall into the lowest bucket(s), leaving relatively few examples in the high-score tail. Without appropriate sampling (Section 5.1 ), this can lead to noisy estimates precisely in the buckets that contribute most to measuring changes induced by threshold-based interventions. In this work we adopted a simple default of B=10 equal-width buckets over [0,1] combined with impression \times score weighted sampling to ensure that each bucket receives a sufficient number of calibration labels. That said, alternative bucket schemes may be appropriate in other settings. Examples include: • Non-uniform buckets, with finer resolution near decision thresholds or in the high-score tail, and coarser buckets in low-score regions where the conditional prevalence is approximately constant. • Relative-score (quantile) buckets, where buckets are defined by quantiles of the score distribution (e.g., deciles) rather than fixed raw-score ranges. This yields roughly equal numbers of items per bucket, which can simplify calibration and help avoid buckets that are extremely sparse in terms of labeled items. Quantile-based buckets also tend to be more stable over time: while raw model scores may shift across days, the relative ordering (and hence the score quantiles) is often much more stable, making the calibration more robust. In future work we plan to explore adaptive bucketization strategies that better balance bias and variance under a fixed calibration budget, while retaining the core advantages of the ML-score surrogate framework. More broadly, although we instantiate our surrogate using model-score buckets in this work, the surrogate can represent many types of content or context features beyond raw model score. The most appropriate choice of the surrogate depends on the application and operational constraints. 7. Future Outlook A key motivation for our approach is that large-scale LLM labeling is currently too expensive to serve as a default metric for experiments. In the longer term, as LLM inference becomes cheaper and more tightly integrated into serving stacks, it may become feasible to label much larger fractions of the corpus (or even all content) for multiple content attributes. In that regime, the relative importance of our surrogate may decrease for absolute prevalence estimation. However, we expect the core ideas of reusing calibration and separating absolute levels from deltas to remain useful even when LLM labeling is no longer the primary bottleneck. Acknowledgements. We thank the following collaborators for their support and feedback throughout this project: Minli Zang, Benjamin Thompson, Xiaohan Yang, Wenjun Wang, Faisal Farooq, Andrey Gusev, Aravindh Manickavasagam, Qinglong Zeng, Jianjin Dong, Ziming Yin, Gerardo Gonzalez, Ahmed Fayez, Yasmin ElBaily and Sari Wang