Post-2024 U.S. Presidential Election Analysis of Election and Poll Data: Real-life Validation of Prediction via Small Area Estimation and Uncertainty Quantification
Zheshi Zheng {\rm Zheng}, Yuanyuan Li {\rm Li}, Peter X. K. Song {\rm Song}, and Jiming Jiang {\rm Jiang}, University of Michigan {\rm Michigan}, Munich Re {\rm Re} and, University of California, Davis {\rm Davis} · 2025
Read Post-2024 U.S. Presidential Election Analysis of Election and Poll Data: Real-life Validation of Prediction via Small Area Estimation and Uncertainty Quantification by Zheshi Zheng {\rm Zheng}, Yuanyuan Li {\rm Li}, Peter X. K. Song {\rm Song}, and Jiming Jiang {\rm Jiang}, University of Michigan {\rm Michigan}, Munich Re {\rm Re} and, University of California, Davis {\rm Davis} (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-03555.txt.
Introduction
Zheshi Zheng, Yuanyuan Li, Peter X K Song, and Jiming Jiang revisit their small-area method against the real outcome of the 2024 United States presidential election. Small area estimation borrows strength across regions to sharpen thin local polls. Using poll data gathered one week before election day, the model called every state where polling existed.
Beyond accuracy, the team stresses honest uncertainty. A new measure, the probability of an incorrect call, gets estimated by a conformal method after the standard bootstrap proved too weak. Sensitivity checks then probe pollster bias, finding swing states especially exposed. The post-election audit offers a rare case where a forecasting method can be graded against full ground truth.
Catalog record
| CREATOR | Zheshi Zheng {\rm Zheng}, Yuanyuan Li {\rm Li}, Peter X. K. Song {\rm Song}, and Jiming Jiang {\rm Jiang}, University of Michigan {\rm Michigan}, Munich Re {\rm Re} and, University of California, Davis {\rm Davis} |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-03555
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Post-2024 U.S. Presidential Election Analysis of Election and Poll Data: Real-life Validation of Prediction via Small Area Estimation and Uncertainty Quantification Zheshi Zheng {\rm Zheng}, Yuanyuan Li {\rm Li}, Peter X. K. Song {\rm Song}, and Jiming Jiang {\rm Jiang}, University of Michigan {\rm Michigan}, Munich Re {\rm Re} and, University of California, Davis {\rm Davis} Abstract We carry out a post-election analysis of the 2024 U.S. Presidential Election (USPE) using a prediction model derived from the Small Area Estimation (SAE) methodology. With pollster data obtained one week prior to the election day, retrospectively, our SAE-based prediction model can perfectly predict the Electoral College election results in all 44 states where polling data were available. In addition to such desirable prediction accuracy, we introduce the probability of incorrect prediction (PoIP) to rigorously analyze prediction uncertainty. Since the standard bootstrap method appears inadequate for estimating PoIP, we propose a conformal inference method that yields reliable uncertainty quantification. We further investigate potential pollster biases by the means of sensitivity analyses and conclude that swing states are particularly vulnerable to polling bias in the prediction of the 2024 USPE. Keywords: prediction, SAE, USPE, conformal prediction, sensitivity analysis, transfer learning. 1 Introduction On the evening of November 4, 2024, the night before the 2024 U.S. Presidential Election (USPE), there was considerable optimism in the air, at least among some of the Democrats, that their presidential candidate, Vice President Kamala Harris, would prevail in the election held on the next day, thus producing the first female President of the United States. There were reasons for such optimism: The polls were looking good in favor of Harris’ winning odds, in particular, in most of the battle-ground states: Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin, which showed Harris having slight leads in Michigan, Pennsylvania and Wisconsin, while virtually tied with her opponent, former President Donald Trump, in Georgia and North Carolina. The electoral-college (EC) calculation showed that Harris did not need to win all such states, or even the majority of these battle-ground states. In fact, if she could just “cash in” the lead she had in the polls of Michigan, Pennsylvania and Wisconsin, she would be all but guaranteed to become the first female U.S. President. To add further optimism, the latest polls seemed to be showing Harris even taking a lead in some of the Republican strongholds, such as Iowa. In contrast, Trump would have to win almost every battle ground state in order to win, not to mention losing any of his strongholds—his road to return to the White House appeared to be much tougher. What about the polls’ under-prediction of Trump’s support among the voters, a well-known fact from the 2016 and 2020 elections? Well, the Democrats believed, as did most, if not all, of the election pollsters, that such a potential under-prediction bias had been “factored” into the polling results. In fact, the polls did do better in predicting the 2020 election, as compared to the 2016 election, and after 2020 there were four more years in the making to improve the poll’s prediction accuracy. Finally after eight long years, on the USPE eve (i.e., night before the USPE), someones were optimistically looking forward to seeing the polls, well, do what the polls do, that is, accurately predicting the USPE results. Really? Kamala Harris ended up losing all of the seven battle-ground states. As a result, Donald Trump has made the most astonishing, and mathematically “improbable”, political comeback in the U.S. history. In particular, he easily held the Republican stronghold Iowa, by a large margin, making the latest poll result look embarrassingly misleading. This motivated many statisticians, including us, to curiously investigate possible causes for this landsliding loss to Harris. As noted, after the disastrous failure of the polls in 2016, the pollsters were seemingly making progress, and did better in 2020. But then came this, the 2024 USPE. One could argue that, at the very least, the polls had correctly predicted Hillary Clinton winning the popularity vote in 2016, something they did not even get right in 2024. This brings back, once again, the same old question: Why is it so hard to predict a Trump election? We call it a Trump election because all three elections, 2016, 2020, and 2024, involved the same Republican candidate, Donald Trump. While the pollsters, and political scientists, may be looking for answers, the answers are expected to be anything but simple, and possibly not to be found for many years, if at all. On the other hand, there are methods, including statistical methods, that demonstrably better predict election results, especially for the USPE; see, for example, Ferejohn & Fiorina 1974 , Merrill 1978 , Gelman & King 1993 , Gelman & King 1994 , Rusk 2001 , Katz et al. 2002 , FiveThirtyEight (https://fivethirtyeight.com/), and Jiang et al. 2023 . This may be related to what the pollsters called were “factored in” in the 2024 polls. Yet, it matters what were factored in, and how to factor them in. Interestingly, a statistical method, proposed by Jiang et al. 2023 , performs much better at correctly predicting the 2024 USPE results. Their prediction machinery is built upon the theory of small area estimation (SAE; e.g., Rao & Molina 2015 ). Out of the 50 states of the U.S. plus the District of Columbia (D.C.), seven of them, namely, Alabama, D.C., Hawaii, Idaho, Kentucky, Louisiana, and Mississippi, did not have any polls conducted after Harris replaced Joe Biden as the Democratic candidate on August 5, 2024; see, for example, Associated Press 2024 . For the remaining 44 states of the U.S., the SAE-based prediction (SAEP) method of Jiang et al. 2023 correctly predicts the winner, and therefore the EC result, for every single one of those 44 states, leading to a total of 219 EC votes for Harris vs 277 EC votes for Trump, which is exactly the same outcomes seen in the real election. In contrast, the benchmark method based on average of the polls incorrectly predicted the results in the battleground states of Michigan, Pennsylvania, and Wisconsin, hence yielding a (very) wrong total EC votes, 263 for Harris vs 233 for Trump, which would have changed the outcome of the 2024 USPE. We refer to more detailed results in Section 3 . Notably, the well-known political website, 538 (https://fivethirtyeight.com/) among others, also made incorrect predictions using the polling average method. Although the empirical results suggest that the SAEP achieves remarkably high accuracy, a critical question remains: How confident are we about the prediction results before we know the truth? Also, because the predictions would be made one week before the actual election, unforeseen events during that period may affect the results. It is therefore desirable to quantify the probability of incorrect prediction (PoIP) for the projected EC winner in each state. Adding this additional rigor to the prediction matters as it addresses uncertainty beyond the polling data. This motivates us to invoke and extend conformal prediction to estimate the PoIP for each state-level prediction. We also perform a sensitivity analysis to demonstrate the impact of pollster-specific biases on the reliability of the predictions and distort the associated uncertainty measures. These are the