Instatistics, thedelta method is a method of deriving the asymptotic distribution of a random variable. It is applicable when the random variable being considered can be defined as adifferentiable function of a random variable which isasymptoticallyGaussian.

The delta method was derived frompropagation of error, and the idea behind was known in the early 20th century.^[1] Its statistical application can be traced as far back as 1928 byT. L. Kelley.^[2] A formal description of the method was presented byJ. L. Doob in 1935.^[3]Robert Dorfman also described a version of it in 1938.^[4]

While the delta method generalizes easily to a multivariate setting, careful motivation of the technique is more easily demonstrated inunivariate terms. Roughly, if there is asequence of random variablesX_n satisfying

${\displaystyle \sqrt{n}[X_n-\theta ]\stackrel{D}{\to }\mathcal{N}(0,{\sigma }^2),}$

whereθ andσ² are finite valued constants and${\displaystyle \stackrel{D}{\to }}$ denotesconvergence in distribution, then

${\displaystyle \sqrt{n}[g(X_n)-g(\theta )]\stackrel{D}{\to }\mathcal{N}(0,{\sigma }^2\cdot [g^{\prime }(\theta ){]}^2)}$

for any functiong satisfying the property that its first derivative, evaluated at${\displaystyle \theta }$,${\displaystyle g^{\prime }(\theta )}$ exists and is non-zero valued.

The intuition of the delta method is that any suchg function, in a "small enough" range of the function, can be approximated via a first orderTaylor series (which is basically a linear function). If the random variable is roughly normal then alinear transformation of it is also normal. Small range can be achieved when approximating the function around the mean, when the variance is "small enough". When g is applied to a random variable such as the mean, the delta method would tend to work better as the sample size increases, since it would help reduce the variance, and thus the Taylor approximation would be applied to a smaller range of the function g at the point of interest.

Demonstration of this result is fairly straightforward under the assumption that${\displaystyle g(x)}$ is differentiable near the neighborhood of${\displaystyle \theta }$ and${\displaystyle g^{\prime }(x)}$ is continuous at${\displaystyle \theta }$ with${\displaystyle g^{\prime }(\theta )\ne 0}$. To begin, we use themean value theorem (i.e.: the first order approximation of aTaylor series usingTaylor's theorem):

${\displaystyle g(X_n)=g(\theta )+g^{\prime }(\tilde{\theta })(X_n-\theta ),}$

where${\displaystyle \tilde{\theta }}$ lies betweenX_n andθ.Note that since${\displaystyle X_n\stackrel{P}{\to }\theta }$ and, it must be that${\displaystyle \tilde{\theta }\stackrel{P}{\to }\theta }$ and sinceg′(θ) is continuous, applying thecontinuous mapping theorem yields

${\displaystyle g^{\prime }(\tilde{\theta })\stackrel{P}{\to }{g}^{\prime }(\theta ),}$

where${\displaystyle \stackrel{P}{\to }}$ denotesconvergence in probability.

Rearranging the terms and multiplying by${\displaystyle \sqrt{n}}$ gives

${\displaystyle \sqrt{n}[g(X_n)-g(\theta )]=g^{\prime }\left(\tilde{\theta }\right)\sqrt{n}[X_n-\theta ].}$

Since

${\displaystyle \sqrt{n}[X_n-\theta ]\stackrel{D}{\to }\mathcal{N}(0,{\sigma }^2)}$

by assumption, it follows immediately from appeal toSlutsky's theorem that

${\displaystyle \sqrt{n}[g(X_n)-g(\theta )]\stackrel{D}{\to }\mathcal{N}(0,{\sigma }^2[g^{\prime }(\theta ){]}^2).}$

This concludes the proof.

Alternatively, one can add one more step at the end, to obtain theorder of approximation:

This suggests that the error in the approximation converges to 0 in probability.

By definition, aconsistent estimatorBconverges in probability to its true valueβ, and often acentral limit theorem can be applied to obtainasymptotic normality:

${\displaystyle \sqrt{n}\left(B-\beta \right)\stackrel{D}{\to }N\left(0,\mathrm{\Sigma }\right),}$

wheren is the number of observations and Σ is a (symmetric positive semi-definite)covariance matrix. Suppose we want to estimate the variance of a scalar-valued functionh of the estimatorB. Keeping only the first two terms of theTaylor series, and usingvector notation for thegradient, we can estimateh(B) as

${\displaystyle h(B)\approx h(\beta )+\mathrm{\nabla }h(\beta {)}^T\cdot (B-\beta )}$

which implies the variance ofh(B) is approximately

One can use themean value theorem (for real-valued functions of many variables) to see that this does not rely on taking first order approximation.

The delta method therefore implies that

${\displaystyle \sqrt{n}\left(h(B)-h(\beta )\right)\stackrel{D}{\to }N\left(0,\mathrm{\nabla }h(\beta {)}^T\cdot \mathrm{\Sigma }\cdot \mathrm{\nabla }h(\beta )\right)}$

or in univariate terms,

${\displaystyle \sqrt{n}\left(h(B)-h(\beta )\right)\stackrel{D}{\to }N\left(0,{\sigma }^2\cdot {\left(h^{\mathrm{\prime }}(\beta )\right)}^2\right).}$

SupposeX_n isbinomial with parameters${\displaystyle p\in (0,1]}$ andn. Since

${\displaystyle \sqrt{n}\left[\frac{X_n}{n}-p\right]\stackrel{D}{\to }N(0,p(1-p)),}$

we can apply the Delta method withg(θ) = log(θ) to see

${\displaystyle \sqrt{n}\left[\log \left(\frac{X_n}{n}\right)-\log (p)\right]\stackrel{D}{\to }N(0,p(1-p)[1/p{]}^2)}$

Hence, even though for any finiten, the variance of${\displaystyle \log \left(\frac{X_n}{n}\right)}$ does not actually exist (sinceX_n can be zero), the asymptotic variance of${\displaystyle \log \left(\frac{X_n}{n}\right)}$ does exist and is equal to

${\displaystyle \frac{1-p}{np}.}$

Note that sincep>0,${\displaystyle Pr\left(\frac{X_n}{n}>0\right)\to 1}$ as${\displaystyle n\to \mathrm{\infty }}$, so with probability converging to one,${\displaystyle \log \left(\frac{X_n}{n}\right)}$ is finite for largen.

Moreover, if${\displaystyle \hat{p}}$ and${\displaystyle \hat{q}}$ are estimates of different group rates from independent samples of sizesn andm respectively, then the logarithm of the estimatedrelative risk${\displaystyle \frac{\hat{p}}{\hat{q}}}$ has asymptotic variance equal to

${\displaystyle \frac{1-p}{pn}+\frac{1-q}{qm}.}$

This is useful to construct a hypothesis test or to make aconfidence interval for the relative risk.

The delta method is often used in a form that is essentially identical to that above, but without the assumption thatX_n orB is asymptotically normal. Often the only context is that the variance is "small". The results then just give approximations to the means and covariances of the transformed quantities. For example, the formulae presented in Klein (1953, p. 258) are:^[5]

whereh_r is therth element ofh(B) andB_i is theith element ofB.

Wheng′(θ) = 0 the delta method cannot be applied. However, ifg′′(θ) exists and is not zero, the second-order delta method can be applied. By the Taylor expansion,${\displaystyle n[g(X_n)-g(\theta )]=\frac{1}{2}n[X_n-\theta {]}^2\left[g^{″}(\theta )\right]+o_p(1)}$, so that the variance of${\displaystyle g\left(X_n\right)}$ relies on up to the 4th moment of${\displaystyle X_n}$.

The second-order delta method is also useful in conducting a more accurate approximation of${\displaystyle g\left(X_n\right)}$'s distribution when sample size is small.${\displaystyle \sqrt{n}[g(X_n)-g(\theta )]=\sqrt{n}[X_n-\theta ]g^{\prime }(\theta )+\frac{1}{2}\frac{n[X_n-\theta {]}^2}{\sqrt{n}}{g}^{″}(\theta )+o_p(1)}$.For example, when${\displaystyle X_n}$ follows the standard normal distribution,${\displaystyle g\left(X_n\right)}$ can be approximated as theweighted sum of a standard normal and a chi-square with 1 degree of freedom.

A version of the delta method exists innonparametric statistics. Let${\displaystyle X_i\sim F}$ be anindependent and identically distributed random variable with a sample of size${\displaystyle n}$ with anempirical distribution function${\displaystyle {\hat{F}}_n}$, and let${\displaystyle T}$ be a functional. If${\displaystyle T}$ isHadamard differentiable with respect to theChebyshev metric, then

${\displaystyle \frac{T({\hat{F}}_n)-T(F)}{\widehat{\text{se}}}\stackrel{D}{\to }N(0,1)}$

where${\displaystyle \widehat{\text{se}}=\frac{\hat{\tau }}{\sqrt{n}}}$ and${\displaystyle {\hat{\tau }}^2=\frac{1}{n}\sum _{i=1}^n{\hat{L}}^2(X_i)}$, with${\displaystyle \hat{L}(x)=L_{{\hat{F}}_n}({\delta }_x)}$ denoting the empiricalinfluence function for${\displaystyle T}$. A nonparametric${\displaystyle (1-\alpha )}$ pointwise asymptotic confidence interval for${\displaystyle T(F)}$ is therefore given by

${\displaystyle T({\hat{F}}_n)\pm z_{\alpha /2}\widehat{\text{se}}}$

where${\displaystyle z_q}$ denotes the${\displaystyle q}$-quantile of the standard normal. See Wasserman (2006) p. 19f. for details and examples.

• Taylor expansions for the moments of functions of random variables
• Variance-stabilizing transformation

• Oehlert, G. W. (1992). "A Note on the Delta Method".The American Statistician.46 (1):27–29.doi:10.1080/00031305.1992.10475842.JSTOR 2684406.
• Wolter, Kirk M. (1985)."Taylor Series Methods".Introduction to Variance Estimation. New York: Springer. pp. 221–247.ISBN 0-387-96119-4.
• Wasserman, Larry (2006).All of Nonparametric Statistics. New York: Springer. pp. 19–20.ISBN 0-387-25145-6.

• Asmussen, Søren (2005)."Some Applications of the Delta Method"(PDF).Lecture notes. Aarhus University. Archived fromthe original(PDF) on May 25, 2015.
• Feiveson, Alan H."Explanation of the delta method". Stata Corp.

• Estimation methods
• Statistical approximations

• CS1 errors: ISBN date
• Articles with short description
• Short description matches Wikidata
• Articles containing proofs
• Statistics articles needing expert attention