A NEW LOOK AT THE ODDS RATIO IN LOGISTIC REGRESSION
JOSÉ RAÚL MARTÍNEZ · 2025
Read A NEW LOOK AT THE ODDS RATIO IN LOGISTIC REGRESSION by JOSÉ RAÚL MARTÍNEZ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17205.txt.
Introduction
José Raúl Martínez takes a fresh look at the odds ratio in a 2025 note on statistics. In logistic regression, the standard odds ratio covers one explanatory variable at a time, a limit he aims to lift. He works out a multivariable odds ratio that spans all the factors in all the ways they can combine, all at once. That broadens a basic gauge of association for use when predictors come in groups.
Catalog record
| CREATOR | JOSÉ RAÚL MARTÍNEZ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17205
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A NEW LOOK AT THE ODDS RATIO IN LOGISTIC REGRESSION JOSÉ RAÚL MARTÍNEZ Abstract . The standard odds ratio of logistic regression is foundational but limited to individual explanatory variables. This work derives a multivariable odds ratio that applies to all the explanatory variables in all their combinations. 1. Introduction In a nutshell, this work is about extracting more information from the standard odds ratios of multivariable logistic regression models whose explanatory variables are binary and have no interactions. For such models, the standard odds ratios by themselves provide incomplete information because each applies only to an individual explanatory variable. In this work, I derive a generalized multivariable odds ratio that treats all the variables jointly in all their combinations. The resulting ensemble of odds ratios tells the complete odds-ratio story. This document’s organization is as follows. Section 2 sets the stage by defining variables, notation, a general binary logistic regression model, and certain mathematical objects that I shall call “events.” Sections 3 – 5 contain the main results. In Section 3 , I derive, analyze, and interpret the known standard odds-ratio formula for individual variables with a view toward recasting it. Recasting is done in Section 4 , in which I derive and validate a new formula for the odds ratio of individual variables. In Section 5 , I derive a formula for a generalized multivariable odds ratio and show that the formula produces the entire ensemble of odds ratios. Section 7 contains conclusions. 2. Preliminaries 2.1. General Binary Logistic Regression Model The variables and associated quantities used in this work are \displaystyle y \displaystyle=\text{Binary response variable, coded }(0,1), (1) \displaystyle x_{n} \displaystyle=n\text{th binary explanatory variable, coded }(0,1), (2) \displaystyle n \displaystyle=1,2,\ldots,N, (3) \displaystyle N \displaystyle=\text{Number of binary explanatory variables}, (4) \displaystyle X_{N} \displaystyle=\{x_{1},x_{2},\dots,x_{N}\}=\text{Set of }N\text{ explanatory % variables}. (5) Hereafter, I shall use “variable(s)” as shorthand for “explanatory variable(s)” unless otherwise indicated. All the variables are binary and noninteracting. The general binary linear logistic regression model is known as the logit model or logit for short. The logit version I use in this work is common in the literature (for example, see [ 1 ] ) and is defined in ( 6 ) using the notation in ( 1 )–( 5 ). \displaystyle\bm{g}(X_{N})=\log\big{(}\mathscr{O}(X_{N})\big{)} \displaystyle=\beta_{0}+\beta_{1}x_{1}+\beta_{2}x_{2}+\cdots+\beta_{N}x_{N}, (6) where “ \log(\cdot) ” denotes the natural logarithm function and \displaystyle\begin{split}\mathscr{O}(X_{N})&=\frac{p\left(y|X_{N}\right)}{1-p% \left(y|X_{N}\right)},\\ &=\text{ Odds of }y\text{ occurring given }X_{N},\\ \end{split} (7) \displaystyle p\left(y|X_{N}\right) \displaystyle=\text{Probability of }y\text{ occurring given }X_{N}, (8) \displaystyle\qquad\qquad\;\;\beta_{k} \displaystyle=k\text{th coefficient},k=0,1,\ldots,N. (9) In ( 7 ) and ( 8 ), the phrase “given X_{N} ” means given that a specific set of values of the variables in X_{N} occurred. The logit ( 6 ) calculates the log-odds of a response y occurring given that X_{N} occurred. I use the generic symbol y in ( 7 )–( 8 ) to match the usage in the literature. Because y has two possible values, I eliminate potential ambiguity in the meaning of y by interpreting y as representing a specified response or outcome , which is either y=1 or y=0 . With this interpretation, the meaning of ( 7 ) is: “Odds of a specified response (outcome) y occurring given X_{N} ,” and likewise for ( 8 ). The logit’s coefficients \beta_{k} are computed with a software package, of which many are available: My software package of choice is R . [ 2 ] Which software package is used is immaterial. In all cases, calculating the coefficients \beta_{k} requires specifying the values of the set of explanatory variables X_{N} from measured or synthetic data or a combination thereof. The constant \beta_{0} in ( 6 ) is commonly called “intercept” in the literature because of its similarity to the term of the same name used in linear regression. The coefficient \beta_{0} is called “intercept” in R and in [ 1 ] . 2.2. More About the Response and Explanatory Variables In ( 1 ) and ( 2 ), the response variable y and the explanatory variable x_{n} are binary quantities, each toggling between two states coded “0” and “1”, respectively. It does not matter that the states of y and x_{n} have identical values because y and x_{n} are categorical variables , hence “0” and “1” are labels . The labels of response and explanatory variables are arbitrary and unrelated, hence their being identical is immaterial. The software package R requires that categorical variables be explicitly identified as such. For purposes of calculation, the labels must be converted into numbers at some point. Using (0,1) to code the response and explanatory variables facilitates treating the labels as numbers as needed. [ 1 ] As explained in [ 1 ] , certain formulas require (0,1) coding, else the formulas have to be modified. All the formulas in this work require (0,1) coding as specified in ( 1 ) and ( 2 ). 2.3. Defining the Events In subsequent sections I use a mathematical object I call event to obtain new expressions for the odds ratio. Events and their properties are defined next. The variables \{x_{1},x_{2},\ldots,x_{N}\} are said to be realized when all N variables have specific identities. For example, a realized set of three variables might be \{x_{1},x_{2},x_{3}\}\rightarrow\{1,0,1\} . Thus, a realized set of variables consists of a sequence of zeros and ones. comprise I define event as a unique realization of the N explanatory variables . For example, there are four events associated with the two-variable set X_{2}=\{x_{1},x_{2}\} , namely, \big{\{}\{0,0\},\{0,1\},\{1,0\},\{1,1\}\big{\}} . All events are realizations of the set X_{N} and vice versa. The sequence of N zeros and ones separated by a comma that makes up an event is equivalent to a positive N -digit binary integer . For example, for the three-variable set X_{3} , the 3-digit binary integer (010)_{2} corresponds to the event \{0,1,0\} . In all the events of the set X_{N} , the binary integers correspond to the decimal integers 0,1,\ldots,2^{N}-1 . Therefore, the number of distinct events is equal to 2^{N} . The set of 2^{N} events exhaustively comprises all possible realizations of the set X_{N} . Events are a property of the logit model and are completely defined by the number of variables N . Hence, events are independent of data . Events treat all the explanatory variables jointly, thereby contextualizing the variables. I use the following notation to denote events. \displaystyle E_{\nu} \displaystyle=\text{The }\nu\text{th event}, (10) \displaystyle\nu \displaystyle=\text{Event number}, (11) \displaystyle\nu \displaystyle=0,1,\ldots,2^{N}-1. (12) A one-to-one correspondence exists between the event number \nu and the event itself. Therefore, an event can be identified from its number and vice versa. As an example, Table 1 lists the 2^{3}=8 events for N=3 ; specifying N=3 sufficed to create the table. The one-to-one correspondence between event numbers and events is evident in the table. Table 1. Events for the set of three explanatory variables X_{3}=\{x_{1},x_{2},x_{3}\} . Decimal Event Event Binary Name Number Event Event \bm{(E_{\nu})} \bm{(\nu)} Number \bm{\{x_{1},x_{2},x_{3}\}} E_{0} 0 000 \{0,0,0\} E_{1} 1 001 \{0,0,1\} E_{2} 2 010 \{0,1,0\} E_{3} 3 011 \{0,1,1\} E_{4} 4 100 \{1,0,0\} E_{5} 5 101 \{1,0,1\} E_{6} 6 110 \{1,1,0\} E_{7} 7 111 \{1,1,1\} Events similar to E_{0}=\{0,0,0\} and E_{7}=\{1,1,1\} in Table 1 have recurring roles in subsequent discussions and I have named them as