Instatistics,binomial regression is aregression analysis technique in which theresponse (often referred to asY) has abinomial distribution: it is the number of successes in a series of⁠${\displaystyle n}$⁠ independentBernoulli trials, where each trial has probability of success⁠${\displaystyle p}$⁠.^[1] In binomial regression, the probability of a success is related toexplanatory variables: the corresponding concept in ordinary regression is to relate the mean value of the unobserved response to explanatory variables.

Binomial regression is closely related tobinary regression: a binary regression can be considered a binomial regression with${\displaystyle n=1}$, or a regression onungrouped binary data, while a binomial regression can be considered a regression ongrouped binary data (seecomparison).^[2] Binomial regression models are essentially the same asbinary choice models, one type ofdiscrete choice model: the primary difference is in the theoretical motivation (seecomparison). Inmachine learning, binomial regression is considered a special case ofprobabilistic classification, and thus a generalization ofbinary classification.

In one published example of an application of binomial regression,^[3] the details were as follows. The observed outcome variable was whether or not a fault occurred in an industrial process. There were two explanatory variables: the first was a simple two-case factor representing whether or not a modified version of the process was used and the second was an ordinary quantitative variable measuring the purity of the material being supplied for the process.

The response variableY is assumed to bebinomially distributed conditional on the explanatory variablesX. The number of trialsn is known, and the probability of success for each trialp is specified as a functionθ(X). This implies that theconditional expectation andconditional variance of the observed fraction of successes,Y/n, are

${\displaystyle E(Y/n\mid X)=\theta (X)}$

${\displaystyle \mathrm{Var}(Y/n\mid X)=\theta (X)(1-\theta (X))/n}$

The goal of binomial regression is to estimate the functionθ(X). Typically the statistician assumes${\displaystyle \theta (X)=m({\beta }^{\mathrm{T}}X)}$, for a known functionm, and estimatesβ. Common choices form include thelogistic function.^[1]

The data are often fitted as ageneralised linear model where the predicted values μ are the probabilities that any individual event will result in a success. Thelikelihood of the predictions is then given by

${\displaystyle L(\mathit{\mu }\mid Y)=\prod _{i=1}^n\left(1_{y_i=1}({\mu }_i)+1_{y_i=0}(1-{\mu }_i)\right),}$

where1_A is theindicator function which takes on the value one when the eventA occurs, and zero otherwise: in this formulation, for any given observationy_i, only one of the two terms inside the product contributes, according to whethery_i=0 or 1. The likelihood function is more fully specified by defining the formal parametersμ_i as parameterised functions of the explanatory variables: this defines the likelihood in terms of a much reduced number of parameters. Fitting of the model is usually achieved by employing the method ofmaximum likelihood to determine these parameters. In practice, the use of a formulation as a generalised linear model allows advantage to be taken of certain algorithmic ideas which are applicable across the whole class of more general models but which do not apply to all maximum likelihood problems.

Models used in binomial regression can often be extended to multinomial data.

There are many methods of generating the values ofμ in systematic ways that allow for interpretation of the model; they are discussed below.

There is a requirement that the modelling linking the probabilities μ to the explanatory variables should be of a form which only produces values in the range 0 to 1. Many models can be fitted into the form

${\displaystyle \mathit{\mu }=g(\mathit{\eta }).}$

Hereη is an intermediate variable representing a linear combination, containing the regression parameters, of the explanatory variables. The functiong is thecumulative distribution function (cdf) of someprobability distribution. Usually this probability distribution has asupport from minus infinity to plus infinity so that any finite value ofη is transformed by the functiong to a value inside the range 0 to 1.

In the case oflogistic regression, the link function is the log of theodds ratio orlogistic function. In the case ofprobit, the link is the cdf of thenormal distribution. Thelinear probability model is not a proper binomial regression specification because predictions need not be in the range of zero to one; it is sometimes used for this type of data when the probability space is where interpretation occurs or when the analyst lacks sufficient sophistication to fit or calculate approximate linearizations of probabilities for interpretation.

Binomial regression is closely connected with binary regression. If the response is abinary variable (two possible outcomes), then these alternatives can be coded as 0 or 1 by considering one of the outcomes as "success" and the other as "failure" and considering these ascount data: "success" is 1 success out of 1 trial, while "failure" is 0 successes out of 1 trial. This can now be considered a binomial distribution with${\displaystyle n=1}$ trial, so a binary regression is a special case of a binomial regression. If these data aregrouped (by adding counts), they are no longer binary data, but are count data for each group, and can still be modeled by a binomial regression; the individual binary outcomes are then referred to as "ungrouped data". An advantage of working with grouped data is that one can test the goodness of fit of the model;^[2] for example, grouped data may exhibitoverdispersion relative to the variance estimated from the ungrouped data.

A binary choice model assumes alatent variableU_n, the utility (or net benefit) that personn obtains from taking an action (as opposed to not taking the action). The utility the person obtains from taking the action depends on the characteristics of the person, some of which are observed by the researcher and some are not:

${\displaystyle U_n=\mathit{\beta }\cdot {\mathbf{s}}_{\mathbf{n}}+{\epsilon }_n}$

where${\displaystyle \mathit{\beta }}$ is a set ofregression coefficients and${\displaystyle {\mathbf{s}}_{\mathbf{n}}}$ is a set ofindependent variables (also known as "features") describing personn, which may be either discrete "dummy variables" or regular continuous variables.${\displaystyle {\epsilon }_n}$ is arandom variable specifying "noise" or "error" in the prediction, assumed to be distributed according to some distribution. Normally, if there is a mean or variance parameter in the distribution, it cannot beidentified, so the parameters are set to convenient values — by convention usually mean 0, variance 1.

The person takes the action,y_n = 1, ifU_n > 0. The unobserved term,ε_n, is assumed to have alogistic distribution.

The specification is written succinctly as:

• • U_n =βs_n +ε_n
  • 
  • ε ∼logistic, standardnormal, etc.

Let us write it slightly differently:

• • U_n =βs_n −e_n
  • 
  • e ∼logistic, standardnormal, etc.

Here we have made the substitutione_n = −ε_n. This changes a random variable into a slightly different one, defined over a negated domain. As it happens, the error distributions we usually consider (e.g.logistic distribution, standardnormal distribution, standardStudent's t-distribution, etc.) are symmetric about 0, and hence the distribution overe_n is identical to the distribution overε_n.

Denote thecumulative distribution function (CDF) of${\displaystyle e}$ as${\displaystyle F_e,}$ and thequantile function (inverse CDF) of${\displaystyle e}$ as${\displaystyle F_e^{-1}.}$

Note that

Since${\displaystyle Y_n}$ is aBernoulli trial, where${\displaystyle \mathbb{E}[Y_n]=Pr(Y_n=1),}$ we have

${\displaystyle \mathbb{E}[Y_n]=F_e(\mathit{\beta }\cdot {\mathbf{s}}_{\mathbf{n}})}$

or equivalently

${\displaystyle F_e^{-1}(\mathbb{E}[Y_n])=\mathit{\beta }\cdot {\mathbf{s}}_{\mathbf{n}}.}$

Note that this is exactly equivalent to the binomial regression model expressed in the formalism of thegeneralized linear model.

If${\displaystyle e_n\sim \mathcal{N}(0,1),}$ i.e. distributed as astandard normal distribution, then

${\displaystyle {\mathrm{\Phi }}^{-1}(\mathbb{E}[Y_n])=\mathit{\beta }\cdot {\mathbf{s}}_{\mathbf{n}}}$

which is exactly aprobit model.

If${\displaystyle e_n\sim \mathrm{Logistic}(0,1),}$ i.e. distributed as a standardlogistic distribution with mean 0 andscale parameter 1, then the correspondingquantile function is thelogit function, and

${\displaystyle \mathrm{logit}(\mathbb{E}[Y_n])=\mathit{\beta }\cdot {\mathbf{s}}_{\mathbf{n}}}$

which is exactly alogit model.

Note that the two different formalisms —generalized linear models (GLM's) anddiscrete choice models — are equivalent in the case of simple binary choice models, but can be extended if differing ways:

• GLM's can easily handle arbitrarily distributedresponse variables (dependent variables), not justcategorical variables orordinal variables, which discrete choice models are limited to by their nature. GLM's are also not limited to link functions that arequantile functions of some distribution, unlike the use of anerror variable, which must by assumption have aprobability distribution.
• On the other hand, because discrete choice models are described as types ofgenerative models, it is conceptually easier to extend them to complicated situations with multiple, possibly correlated, choices for each person, or other variations.

Alatent variable model involving a binomial observed variableY can be constructed such thatY is related to the latent variableY* via

The latent variableY* is then related to a set of regression variablesX by the model

${\displaystyle Y^{\ast }=X\beta +ϵ.}$

This results in a binomial regression model.

The variance ofϵ can not be identified and when it is not of interest is often assumed to be equal to one. Ifϵ is normally distributed, then a probit is the appropriate model and ifϵ islog-Weibull distributed, then a logit is appropriate. Ifϵ is uniformly distributed, then a linear probability model is appropriate.

• Linear probability model
• Poisson regression
• Predictive modelling

• Cox, D. R.;Snell, E. J. (1981).Applied Statistics: Principles and Examples. Chapman and Hall.ISBN 0-412-16570-8.
• Rodríguez, Germán (2007)."Lecture Notes on Generalized Linear Models".

• Dean, C. B. (1992). "Testing for Overdispersion in Poisson and Binomial Regression Models".Journal of the American Statistical Association.87 (418). Informa UK Limited:451–457.doi:10.1080/01621459.1992.10475225.ISSN 0162-1459.JSTOR 2290276.

• Factorial and binomial topics
• Generalized linear models

• Articles with short description
• Short description matches Wikidata