Using SVM to Estimate and Predict Binary Choice Models

Yoosoon Chang, Joon Y. Park, Guo Yan · 2026

Read Using SVM to Estimate and Predict Binary Choice Models by Yoosoon Chang, Joon Y. Park, Guo Yan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22659.txt.

Introduction

Yoosoon Chang, Joon Park, and Guo Yan use support vector machines to estimate binary choice models. A binary choice model explains yes-no acts, from buys to votes. Their 2026 preprint notes that the SVM, a classic margin-based learner, acts in the limit much like a quasi-likelihood estimator for such data.

That parallel lets SVMs serve as estimators with known traits, not just black-box scorers. It can also yield forecasts of choice from the fitted model. The work joins machine learning and econometrics in a clean way. Each field gains a tool from the other.

Catalog record

CREATORYoosoon Chang, Joon Y. Park, Guo Yan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22659

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Using SVM to Estimate and Predict Binary Choice Models Thanks: Preliminary versions of this paper have been circulated since 2022. We thank Ye Lu and seminar and conference participants of the 2023 SoFie Conference in Seoul, Korea, the 2023 MEG Conference at Cleveland Fed, the AiE Conference and Festschrift in Honor of Joon Y. Park, the 2023 Melbourne Econometrics Workshop, Monash University, University of Essex, and 2025 Yale Econometrics Conference in Honor of Don Andrews for their helpful comments. We thank the 2024 Eminent Research Scholarship, University of Melbourne, for funding Joon Y. Park and Yoosoon Chang to visit the University of Melbourne. Yoosoon Chang, Joon Y. Park, Guo Yan Abstract The support vector machine (SVM) has an asymptotic behavior that parallels that of the quasi-maximum likelihood estimator (QMLE) for binary outcomes generated by a binary choice model (BCM), although it is not a QMLE. We show that, under the linear conditional mean condition for covariates given the systematic component used in the QMLE slope consistency literature, the slope of the separating hyperplane given by the SVM consistently estimates the BCM slope parameter, as long as the class weight is used as required when binary outcomes are severely imbalanced. The SVM slope estimator is asymptotically equivalent to that of logistic regression in this sense. The finite-sample performance of the two estimators can be quite distinct depending on the distributions of covariates and errors, but neither dominates the other. The intercept parameter of the BCM can be consistently estimated once a consistent estimator of its slope parameter is obtained. JEL Classification: C13, C22 Keywords and phrases: machine learning, binary choice model, support vector machine, maximum likelihood estimation, estimation and prediction, asymptotics 1 Introduction Machine learning has become increasingly popular in economics. See, e.g., 14 and 1 , among many others. In machine learning, a variety of algorithms have been developed to analyze data within a flexible framework without an underlying statistical model. Although they were initially proposed as purely computational algorithms, it has recently been shown that some of them can be used to estimate a well-defined class of statistical and econometric models. For instance, it is now widely recognized that random forests and neural networks can be used to consistently estimate nonparametric regression functions under suitable regularity conditions. For the related asymptotic theories of random forests and neural networks, the reader is referred to, e.g., 4 and 6 , respectively, and the references therein. In this paper, we show that the support vector machine (SVM), another widely used algorithm for classification in machine learning, can be used under appropriate conditions to consistently estimate the binary choice model (BCM). The SVM was originally developed by 5 as an algorithm to classify binary outcomes observed with other related features. It is based on a simple idea of using a hyperplane to partition the feature space into two subsets associated with binary outcomes and has thus been regarded as an algorithm entirely independent of any statistical or econometric model. However, it is natural to consider the SVM as a method for analyzing the BCM, since the way the SVM classifies binary observations is essentially the same as that of a BCM with a zero median error term. Indeed, the SVM and the traditional approaches to BCMs use the same type of classifier, the one given by a separating hyperplane, to predict binary outcomes from covariates. For observations generated by the BCM, the SVM classifier will therefore not be asymptotically optimal unless it consistently estimates the underlying BCM. The main finding of this paper is that the slope coefficient of the hyperplane provided by the SVM is a consistent estimator for the slope parameter in the BCM under the same conditions as those required for consistency of the quasi-maximum likelihood estimator (QMLE). The estimator provided by the SVM, which is referred to as the SVM estimator in the paper, is thus asymptotically equivalent to the QMLE in this sense. 1 1 1 The SVM estimator and the QMLE generally do not have the same limit distribution, so they are not fully equivalent even asymptotically. Also, our asymptotic setup does not allow for the number m of features or covariates going to infinity. This is rather surprising, since the SVM has been considered as an algorithm which does not rely on any statistical or econometric model. Given that the SVM is used widely to analyze binary outcomes in machine learning, this asymptotic equivalence of the SVM estimator to the QMLE adds an interesting and important interpretation of the SVM. We establish consistency of the SVM estimator for the slope parameter without assuming that a pseudo-true value exists. This is essential because, for the SVM estimator, a pseudo-true value may not exist if the two classes defined by a binary outcome are severely imbalanced, in which case the SVM fails and simply predicts a majority class. 2 2 2 This is well known. As shown in the paper, the problem disappears if we use class weight , an option for the SVM adopted widely to deal with imbalanced data. The QMLE for the parameter in the BCM is generally inconsistent. It is, however, expected to yield a consistent estimator for the slope parameter in the BCM if the conditional expectation of covariates on its systematic component is given as a linear function. This condition is not as stringent as it may sound. It holds if covariates jointly have an elliptical distribution, and we may give appropriate weights to the observations on covariates so that they may be regarded as being drawn from such a distribution. See, e.g., 18 , 15 . Consistent estimation of the slope parameter is of primary interest for the BCM, especially when its dimension is large with many covariates. This is because we may easily estimate the remaining one-dimensional intercept parameter once a consistent estimator for the multi-dimensional slope parameter is obtained. Under the required linearity in conditional expectation, consistency of the slope parameter in the BCM may be easily established if we assume the existence of a pseudo-true value as in 17 ; 18 . 3 3 3 The existence of a pseudo-true value is not guaranteed. For a rigorous proof of his result, see 2 . The SVM estimator is comparable in many respects to the logit estimator, which is most commonly used QMLE especially for the large dimensional BCM, except that it breaks down in the extreme case where severe class imbalance exists. By simulation, we compare the SVM estimator with the logit estimator in terms of their finite sample performance. In finite samples, the performances of the two estimators could be quite distinct depending upon the distributions of covariates and errors. It seems clear that no one dominates the other. Though their performances are largely comparable, the SVM estimator performs significantly better than the logit estimator in some cases and vice versa in others. Once a consistent estimator for the slope parameter is obtained, we may consistently estimate the intercept parameter in various ways. We may estimate the intercept parameter using, for example, the maximum score estimator of 13 or its smoothed version given by 7 , by replacing the slope parameter with its consistent estimate. In fact,

Read the complete text