Movatterモバイル変換

[0]ホーム
URL:

Jump to content
WikipediaThe Free Encyclopedia
Search

Normal distribution

From Wikipedia, the free encyclopedia
Probability distribution
"Bell curve" redirects here. For other uses, seeBell curve (disambiguation).
icon
This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Normal distribution" – news ·newspapers ·books ·scholar ·JSTOR(December 2024) (Learn how and when to remove this message)

Normal distribution
Probability density function
The red curve is thestandard normal distribution.
Cumulative distribution function
NotationN(μ,σ2){\displaystyle {\mathcal {N}}(\mu ,\sigma ^{2})}
Parametersμ∈R{\displaystyle \mu \in \mathbb {R} } =mean (location)
σ2∈R>0{\displaystyle \sigma ^{2}\in \mathbb {R} _{>0}} =variance (squaredscale)
Supportx∈R{\displaystyle x\in \mathbb {R} }
PDF12πσ2e−(x−μ)22σ2{\displaystyle {\frac {1}{\sqrt {2\pi \sigma ^{2}}}}e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}}}
CDFΦ(x−μσ)=12[1+erf⁡(x−μσ2)]{\displaystyle \Phi \left({\frac {x-\mu }{\sigma }}\right)={\frac {1}{2}}\left[1+\operatorname {erf} \left({\frac {x-\mu }{\sigma {\sqrt {2}}}}\right)\right]}
Quantileμ+σ2erf−1⁡(2p−1){\displaystyle \mu +\sigma {\sqrt {2}}\operatorname {erf} ^{-1}(2p-1)}
Meanμ{\displaystyle \mu }
Medianμ{\displaystyle \mu }
Modeμ{\displaystyle \mu }
Varianceσ2{\displaystyle \sigma ^{2}}
MADσ2erf−1⁡(1/2){\displaystyle \sigma {\sqrt {2}}\,\operatorname {erf} ^{-1}(1/2)}
AADσ2/π{\textstyle \sigma {\sqrt {2/\pi }}}
Skewness0{\displaystyle 0}
Excess kurtosis0{\displaystyle 0}
Entropy12log⁡(2πeσ2){\textstyle {\tfrac {1}{2}}\log(2\pi e\sigma ^{2})}
MGFexp⁡(μt+σ2t2/2){\displaystyle \exp(\mu t+\sigma ^{2}t^{2}/2)}
CFexp⁡(iμt−σ2t2/2){\displaystyle \exp(i\mu t-\sigma ^{2}t^{2}/2)}
Fisher information

I(μ,σ)=(1/σ2002/σ2){\displaystyle {\mathcal {I}}(\mu ,\sigma )={\begin{pmatrix}1/\sigma ^{2}&0\\0&2/\sigma ^{2}\end{pmatrix}}}

I(μ,σ2)=(1/σ2001/(2σ4)){\displaystyle {\mathcal {I}}(\mu ,\sigma ^{2})={\begin{pmatrix}1/\sigma ^{2}&0\\0&1/(2\sigma ^{4})\end{pmatrix}}}
Kullback–Leibler divergence12{(σ0σ1)2+(μ1−μ0)2σ12−1+ln⁡σ12σ02}{\displaystyle {1 \over 2}\left\{\left({\frac {\sigma _{0}}{\sigma _{1}}}\right)^{2}+{\frac {(\mu _{1}-\mu _{0})^{2}}{\sigma _{1}^{2}}}-1+\ln {\sigma _{1}^{2} \over \sigma _{0}^{2}}\right\}}
Expected shortfallμ+σ12πe−(qp(X−μσ))221−p{\displaystyle \mu +\sigma {\frac {{\frac {1}{\sqrt {2\pi }}}e^{\frac {-\left(q_{p}\left({\frac {X-\mu }{\sigma }}\right)\right)^{2}}{2}}}{1-p}}}[1]
Part of a series onstatistics
Probability theory

Inprobability theory andstatistics, anormal distribution orGaussian distribution is a type ofcontinuous probability distribution for areal-valuedrandom variable. The general form of itsprobability density function is[2][3][4]f(x)=12πσ2exp⁡(−(x−μ)22σ2).{\displaystyle f(x)={\frac {1}{\sqrt {2\pi \sigma ^{2}}}}\exp {\left(-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right)}\,.}The parameter⁠μ{\displaystyle \mu }⁠ is themean orexpectation of the distribution (and also itsmedian andmode), while the parameterσ2{\textstyle \sigma ^{2}} is thevariance. Thestandard deviation of the distribution is⁠σ{\displaystyle \sigma }⁠ (sigma). A random variable with a Gaussian distribution is said to benormally distributed and is called anormal deviate.

Normal distributions are important instatistics and are often used in thenatural andsocial sciences to represent real-valuedrandom variables whose distributions are not known.[5][6] Their importance is partly due to thecentral limit theorem. It states that, under some conditions, the average of many samples (observations) of a random variable with finite mean and variance is itself a random variable—whose distributionconverges to a normal distribution as the number of samples increases. Therefore, physical quantities that are expected to be the sum of many independent processes, such asmeasurement errors, often have distributions that are nearly normal.[7]

Moreover, Gaussian distributions have some unique properties that are valuable in analytic studies. For instance, anylinear combination of a fixed collection of independent normal deviates is a normal deviate. Many results and methods, such aspropagation of uncertainty andleast squares[8] parameter fitting, can be derived analytically in explicit form when the relevant variables are normally distributed.

A normal distribution is sometimes informally called abell curve.[9][10] However, many other distributions arebell-shaped (such as theCauchy,Student'st, andlogistic distributions). (For other names, seeNaming.)

Theunivariate probability distribution is generalized forvectors in themultivariate normal distribution and for matrices in thematrix normal distribution.

Definitions

[edit]

Standard normal distribution

[edit]

The simplest case of a normal distribution is known as thestandard normal distribution orunit normal distribution. This is a special case whenμ=0{\textstyle \mu =0} andσ2=1{\textstyle \sigma ^{2}=1}, and it is described by thisprobability density function (or density):[11]φ(z)=e−z2/22π.{\displaystyle \varphi (z)={\frac {e^{-z^{2}/2}}{\sqrt {2\pi }}}\,.}The variable⁠z{\displaystyle z}⁠ has a mean of 0 and a variance and standard deviation of 1. The densityφ(z){\textstyle \varphi (z)} has its peak12π{\textstyle {\frac {1}{\sqrt {2\pi }}}} atz=0{\textstyle z=0} andinflection points atz=+1{\textstyle z=+1} and⁠z=−1{\displaystyle z=-1}⁠.

Although the density above is most commonly known as thestandard normal, a few authors have used that term to describe other versions of the normal distribution.Carl Friedrich Gauss, for example, once defined the standard normal asφ(z)=e−z2π,{\displaystyle \varphi (z)={\frac {e^{-z^{2}}}{\sqrt {\pi }}},}which has a variance of⁠12{\displaystyle {\frac {1}{2}}}⁠, andStephen Stigler[12] once defined the standard normal asφ(z)=e−πz2,{\displaystyle \varphi (z)=e^{-\pi z^{2}},}which has a simple functional form and a variance ofσ2=12π.{\textstyle \sigma ^{2}={\frac {1}{2\pi }}.}

General normal distribution

[edit]

Every normal distribution is a version of the standard normal distribution whose domain has been stretched by a factor⁠σ{\displaystyle \sigma }⁠ (the standard deviation) and then translated by⁠μ{\displaystyle \mu }⁠ (the mean value):f(x∣μ,σ2)=1σφ(x−μσ).{\displaystyle f(x\mid \mu ,\sigma ^{2})={\frac {1}{\sigma }}\varphi \left({\frac {x-\mu }{\sigma }}\right)\,.}

The probability density must be scaled by1/σ{\textstyle 1/\sigma } so that theintegral is still 1.

If⁠Z{\displaystyle Z}⁠ is astandard normal deviate, thenX=σZ+μ{\textstyle X=\sigma Z+\mu } will have a normal distribution with expected value⁠μ{\displaystyle \mu }⁠ and standard deviation⁠σ{\displaystyle \sigma }⁠. This is equivalent to saying that the standard normal distribution⁠Z{\displaystyle Z}⁠ can be scaled/stretched by a factor of⁠σ{\displaystyle \sigma }⁠ and shifted by⁠μ{\displaystyle \mu }⁠ to yield a different normal distribution, called⁠X{\displaystyle X}⁠. Conversely, if⁠X{\displaystyle X}⁠ is a normal deviate with parameters⁠μ{\displaystyle \mu }⁠ andσ2{\textstyle \sigma ^{2}}, then this⁠X{\displaystyle X}⁠ distribution can be re-scaled and shifted via the formulaZ=(X−μ)/σ{\textstyle Z=(X-\mu )/\sigma } to convert it to the standard normal distribution. This variate is also called the standardized form of⁠X{\displaystyle X}⁠.

Notation

[edit]

The probability density of the standard Gaussian distribution (standard normal distribution, with zero mean and unit variance) is often denoted with the Greek letter⁠ϕ{\displaystyle \phi }⁠ (phi).[13] The alternative form of the Greek letter phi,⁠φ{\displaystyle \varphi }⁠, is also used quite often.

The normal distribution is often referred to asN(μ,σ2){\textstyle N(\mu ,\sigma ^{2})} or⁠N(μ,σ2){\displaystyle {\mathcal {N}}(\mu ,\sigma ^{2})}⁠.[14] Thus when a random variable⁠X{\displaystyle X}⁠ is normally distributed with mean⁠μ{\displaystyle \mu }⁠ and standard deviation⁠σ{\displaystyle \sigma }⁠, one may write

X∼N(μ,σ2).{\displaystyle X\sim {\mathcal {N}}(\mu ,\sigma ^{2}).}

Alternative parameterizations

[edit]

Some authors advocate using theprecision⁠τ{\displaystyle \tau }⁠ as the parameter defining the width of the distribution, instead of the standard deviation⁠σ{\displaystyle \sigma }⁠ or the variance⁠σ2{\displaystyle \sigma ^{2}}⁠. The precision is normally defined as the reciprocal of the variance,⁠1/σ2{\displaystyle 1/\sigma ^{2}}⁠.[15] The formula for the distribution then becomesf(x)=τ2πe−τ(x−μ)2/2.{\displaystyle f(x)={\sqrt {\frac {\tau }{2\pi }}}e^{-\tau (x-\mu )^{2}/2}.}

This choice is claimed to have advantages in numerical computations when⁠σ{\displaystyle \sigma }⁠ is very close to zero, and simplifies formulas in some contexts, such as in theBayesian inference of variables withmultivariate normal distribution.

Alternatively, the reciprocal of the standard deviationτ′=1/σ{\textstyle \tau '=1/\sigma } might be defined as theprecision, in which case the expression of the normal distribution becomesf(x)=τ′2πe−(τ′)2(x−μ)2/2.{\displaystyle f(x)={\frac {\tau '}{\sqrt {2\pi }}}e^{-(\tau ')^{2}(x-\mu )^{2}/2}.}

According to Stigler, this formulation is advantageous because of a much simpler and easier-to-remember formula, and simple approximate formulas for thequantiles of the distribution.

Normal distributions form anexponential family withnatural parametersθ1=μσ2{\textstyle \textstyle \theta _{1}={\frac {\mu }{\sigma ^{2}}}} andθ2=−12σ2{\textstyle \textstyle \theta _{2}={\frac {-1}{2\sigma ^{2}}}}, and natural statisticsx andx2. The dual expectation parameters for normal distribution areη1 =μ andη2 =μ2 +σ2.

Cumulative distribution function

[edit]

Thecumulative distribution function (CDF) of the standard normal distribution, usually denoted with the capital Greek letter⁠Φ{\displaystyle \Phi }⁠, is the integralΦ(x)=12π∫−∞xe−t2/2dt.{\displaystyle \Phi (x)={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{x}e^{-t^{2}/2}\,dt\,.}

The relatederror functionerf⁡(x){\textstyle \operatorname {erf} (x)} gives the probability of a random variable, with normal distribution of mean 0 and variance 1/2, falling in the range⁠[−x,x]{\displaystyle [-x,x]}⁠. That is:erf⁡(x)=1π∫−xxe−t2dt=2π∫0xe−t2dt.{\displaystyle \operatorname {erf} (x)={\frac {1}{\sqrt {\pi }}}\int _{-x}^{x}e^{-t^{2}}\,dt={\frac {2}{\sqrt {\pi }}}\int _{0}^{x}e^{-t^{2}}\,dt\,.}

These integrals cannot be expressed in terms of elementary functions, and are often said to bespecial functions. However, many numerical approximations are known; seebelow for more.

The two functions are closely related, namelyΦ(x)=12[1+erf⁡(x2)].{\displaystyle \Phi (x)={\frac {1}{2}}\left[1+\operatorname {erf} \left({\frac {x}{\sqrt {2}}}\right)\right].}

For a generic normal distribution with density⁠f{\displaystyle f}⁠, mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}}, the cumulative distribution function isF(x)=Φ(x−μσ)=12[1+erf⁡(x−μσ2)].{\displaystyle F(x)=\Phi {\left({\frac {x-\mu }{\sigma }}\right)}={\frac {1}{2}}\left[1+\operatorname {erf} \left({\frac {x-\mu }{\sigma {\sqrt {2}}}}\right)\right].}

The probability thatx lies in the semi-closedinterval(a,b], where(a <b], is therefore[16]: 84 P⁡(a<x≤b)=erf⁡(b−μσ2)−erf⁡(a−μσ2)2=erf⁡(b−μσ2)+erf⁡(μ−aσ2)2{\displaystyle {\begin{aligned}\operatorname {P} (a<x\leq b)&={\frac {\operatorname {erf} \left({\frac {b-\mu }{\sigma {\sqrt {2}}}}\right)-\operatorname {erf} \left({\frac {a-\mu }{\sigma {\sqrt {2}}}}\right)}{2}}\\&={\frac {\operatorname {erf} \left({\frac {b-\mu }{\sigma {\sqrt {2}}}}\right)+\operatorname {erf} \left({\frac {\mu -a}{\sigma {\sqrt {2}}}}\right)}{2}}\\\end{aligned}}}

The complement of the standard normal cumulative distribution function,Q(x)=1−Φ(x){\textstyle Q(x)=1-\Phi (x)}, is often called theQ-function, especially in engineering texts.[17][18] It gives the probability that the value of a standard normal random variable⁠X{\displaystyle X}⁠ will exceed⁠x{\displaystyle x}⁠:⁠P(X>x){\displaystyle P(X>x)}⁠. Other definitions of the⁠Q{\displaystyle Q}⁠-function, all of which are simple transformations of⁠Φ{\displaystyle \Phi }⁠, are also used occasionally.[19]

Thegraph of the standard normal cumulative distribution function⁠Φ{\displaystyle \Phi }⁠ has 2-foldrotational symmetry around the point (0,1/2); that is,⁠Φ(−x)=1−Φ(x){\displaystyle \Phi (-x)=1-\Phi (x)}⁠. Itsantiderivative (indefinite integral) can be expressed as follows:∫Φ(x)dx=xΦ(x)+φ(x)+C.{\displaystyle \int \Phi (x)\,dx=x\Phi (x)+\varphi (x)+C.}

The cumulative distribution function of the standard normal distribution can be expanded byintegration by parts into a series:Φ(x)=12+12π⋅e−x2/2[x+x33+x53⋅5+⋯+x2n+1(2n+1)!!+⋯].{\displaystyle \Phi (x)={\frac {1}{2}}+{\frac {1}{\sqrt {2\pi }}}\cdot e^{-x^{2}/2}\left[x+{\frac {x^{3}}{3}}+{\frac {x^{5}}{3\cdot 5}}+\cdots +{\frac {x^{2n+1}}{(2n+1)!!}}+\cdots \right]\,.}where!!{\textstyle !!} denotes thedouble factorial.

Anasymptotic expansion of the cumulative distribution function for largex can also be derived using integration by parts. For more, seeError function § Asymptotic expansion.[20]

The standard normal distribution's cumulative distribution function can be found via its Taylor series approximation:Φ(x)=12+12π∑k=0∞(−1)kx(2k+1)2kk!(2k+1).{\displaystyle \Phi (x)={\frac {1}{2}}+{\frac {1}{\sqrt {2\pi }}}\sum _{k=0}^{\infty }{\frac {(-1)^{k}x^{(2k+1)}}{2^{k}k!(2k+1)}}\,.}

Recursive computation with Taylor series expansion

[edit]

The recursive nature of theeax2{\textstyle e^{ax^{2}}}family of derivatives may be used to easily construct a rapidly convergingTaylor series expansion using recursive entries about any point of known value of the distribution,Φ(x0){\textstyle \Phi (x_{0})}:Φ(x)=∑n=0∞Φ(n)(x0)n!(x−x0)n,{\displaystyle \Phi (x)=\sum _{n=0}^{\infty }{\frac {\Phi ^{(n)}(x_{0})}{n!}}(x-x_{0})^{n}\,,}where:Φ(0)(x0)=12π∫−∞x0e−t2/2dtΦ(1)(x0)=12πe−x02/2Φ(n)(x0)=−(x0Φ(n−1)(x0)+(n−2)Φ(n−2)(x0)),n≥2.{\displaystyle {\begin{aligned}\Phi ^{(0)}(x_{0})&={\frac {1}{\sqrt {2\pi }}}\int _{-\infty }^{x_{0}}e^{-t^{2}/2}\,dt\\\Phi ^{(1)}(x_{0})&={\frac {1}{\sqrt {2\pi }}}e^{-x_{0}^{2}/2}\\\Phi ^{(n)}(x_{0})&=-\left(x_{0}\Phi ^{(n-1)}(x_{0})+(n-2)\Phi ^{(n-2)}(x_{0})\right),&n\geq 2\,.\end{aligned}}}

Using the Taylor series and Newton's method for the inverse function

[edit]

An application for the above Taylor series expansion is to useNewton's method to reverse the computation. That is, if we have a value for thecumulative distribution function,Φ(x){\textstyle \Phi (x)}, but do not know the x needed to obtain theΦ(x){\textstyle \Phi (x)}, we can use Newton's method to find x, and use the Taylor series expansion above to minimize the number of computations. Newton's method is ideal to solve this problem because the first derivative ofΦ(x){\textstyle \Phi (x)}, which is an integral of the normal standard distribution, is the normal standard distribution, and is readily available to use in the Newton's method solution.

To solve, select a known approximate solution,x0{\textstyle x_{0}}, to the desired⁠Φ(x){\displaystyle \Phi (x)}⁠.x0{\textstyle x_{0}} may be a value from a distribution table, or an intelligent estimate followed by a computation ofΦ(x0){\textstyle \Phi (x_{0})} using any desired means to compute. Use this value ofx0{\textstyle x_{0}} and the Taylor series expansion above to minimize computations.

Repeat the following process until the difference between the computedΦ(xn){\textstyle \Phi (x_{n})} and the desired⁠Φ{\displaystyle \Phi }⁠, which we will callΦ(desired){\textstyle \Phi ({\text{desired}})}, is below a chosen acceptably small error, such as 10−5, 10−15, etc.:xn+1=xn−Φ(xn,x0,Φ(x0))−Φ(desired)Φ′(xn),{\displaystyle x_{n+1}=x_{n}-{\frac {\Phi (x_{n},x_{0},\Phi (x_{0}))-\Phi ({\text{desired}})}{\Phi '(x_{n})}}\,,}where

Φ(x,x0,Φ(x0)){\textstyle \Phi (x,x_{0},\Phi (x_{0}))} is theΦ(x){\textstyle \Phi (x)} from a Taylor series solution usingx0{\textstyle x_{0}} andΦ(x0){\textstyle \Phi (x_{0})}

Φ′(xn)=12πe−xn2/2.{\displaystyle \Phi '(x_{n})={\frac {1}{\sqrt {2\pi }}}e^{-x_{n}^{2}/2}\,.}

When the repeated computations converge to an error below the chosen acceptably small value,x will be the value needed to obtain aΦ(x){\textstyle \Phi (x)} of the desired value,⁠Φ(desired){\displaystyle \Phi ({\text{desired}})}⁠.

Standard deviation and coverage

[edit]
Further information:Interval estimation andCoverage probability
For the normal distribution, the values less than one standard deviation from the mean account for 68.27% of the set; while two standard deviations from the mean account for 95.45%; and three standard deviations account for 99.73%.

About 68% of values drawn from a normal distribution are within one standard deviationσ from the mean; about 95% of the values lie within two standard deviations; and about 99.7% are within three standard deviations.[9] This is known as the68–95–99.7 (empirical) rule, or the3-sigma rule.

More precisely, the probability that a normal deviate lies in the range betweenμ−nσ{\textstyle \mu -n\sigma } andμ+nσ{\textstyle \mu +n\sigma } is given byF(μ+nσ)−F(μ−nσ)=Φ(n)−Φ(−n)=erf⁡(n2).{\displaystyle F(\mu +n\sigma )-F(\mu -n\sigma )=\Phi (n)-\Phi (-n)=\operatorname {erf} \left({\frac {n}{\sqrt {2}}}\right).}To 12 significant digits, the values forn=1,2,…,6{\textstyle n=1,2,\ldots ,6} are:

⁠n{\displaystyle n}⁠p=F(μ+nσ)−F(μ−nσ){\textstyle p=F(\mu +n\sigma )-F(\mu -n\sigma )}1−p{\textstyle 1-p}or 1 in (1−p){\textstyle {\text{or }}1{\text{ in }}(1-p)}OEIS
10.6826894921370.317310507863
3.15148718753
OEIS: A178647
20.9544997361040.045500263896
21.9778945080
OEIS: A110894
30.9973002039370.002699796063
370.398347345
OEIS: A270712
40.9999366575160.000063342484
15787.1927673
50.9999994266970.000000573303
1744277.89362
60.9999999980270.000000001973
506797345.897

For large⁠n{\displaystyle n}⁠, one can use the approximation1−p≈2nπen2{\displaystyle 1-p\approx {\frac {\sqrt {2}}{n{\sqrt {\pi e^{n^{2}}}}}}}

Quantile function

[edit]
Further information:Quantile function § Normal distribution

Thequantile function of a distribution is the inverse of the cumulative distribution function. The quantile function of the standard normal distribution is called theprobit function, and can be expressed in terms of the inverseerror function:Φ−1(p)=2erf−1⁡(2p−1),p∈(0,1).{\displaystyle \Phi ^{-1}(p)={\sqrt {2}}\operatorname {erf} ^{-1}(2p-1),\quad p\in (0,1).}For a normal random variable with mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}}, the quantile function isF−1(p)=μ+σΦ−1(p)=μ+σ2erf−1⁡(2p−1),p∈(0,1).{\displaystyle F^{-1}(p)=\mu +\sigma \Phi ^{-1}(p)=\mu +\sigma {\sqrt {2}}\operatorname {erf} ^{-1}(2p-1),\quad p\in (0,1).}ThequantileΦ−1(p){\textstyle \Phi ^{-1}(p)} of the standard normal distribution is commonly denoted as⁠zp{\displaystyle z_{p}}⁠. These values are used inhypothesis testing, construction ofconfidence intervals andQ–Q plots. A normal random variable⁠X{\displaystyle X}⁠ will exceedμ+zpσ{\textstyle \mu +z_{p}\sigma } with probability1−p{\textstyle 1-p}, and will lie outside the intervalμ±zpσ{\textstyle \mu \pm z_{p}\sigma } with probability⁠2(1−p){\displaystyle 2(1-p)}⁠. In particular, the quantilez0.975{\textstyle z_{0.975}} is1.96; therefore a normal random variable will lie outside the intervalμ±1.96σ{\textstyle \mu \pm 1.96\sigma } in only 5% of cases.

The following table gives the quantilezp{\textstyle z_{p}} such that⁠X{\displaystyle X}⁠ will lie in the rangeμ±zpσ{\textstyle \mu \pm z_{p}\sigma } with a specified probability⁠p{\displaystyle p}⁠. These values are useful to determinetolerance interval forsample averages and other statisticalestimators with normal (orasymptotically normal) distributions.[21] The following table shows2erf−1⁡(p)=Φ−1(p+12){\textstyle {\sqrt {2}}\operatorname {erf} ^{-1}(p)=\Phi ^{-1}\left({\frac {p+1}{2}}\right)}, notΦ−1(p){\textstyle \Phi ^{-1}(p)} as defined above.

⁠p{\displaystyle p}⁠zp{\textstyle z_{p}} ⁠p{\displaystyle p}⁠zp{\textstyle z_{p}}
0.801.2815515655450.9993.290526731492
0.901.6448536269510.99993.890591886413
0.951.9599639845400.999994.417173413469
0.982.3263478740410.9999994.891638475699
0.992.5758293035490.99999995.326723886384
0.9952.8070337683440.999999995.730728868236
0.9983.0902323061680.9999999996.109410204869

For small⁠p{\displaystyle p}⁠, the quantile function has the usefulasymptotic expansionΦ−1(p)=−ln⁡1p2−ln⁡ln⁡1p2−ln⁡(2π)+o(1).{\textstyle \Phi ^{-1}(p)=-{\sqrt {\ln {\frac {1}{p^{2}}}-\ln \ln {\frac {1}{p^{2}}}-\ln(2\pi )}}+{\mathcal {o}}(1).}[citation needed]

Properties

[edit]

The normal distribution is the only distribution whosecumulants beyond the first two (i.e., other than the mean andvariance) are zero. It is also the continuous distribution with themaximum entropy for a specified mean and variance.[22][23] Geary has shown, assuming that the mean and variance are finite, that the normal distribution is the only distribution where the mean and variance calculated from a set of independent draws are independent of each other.[24][25]

The normal distribution is a subclass of theelliptical distributions. The normal distribution issymmetric about its mean, and is non-zero over the entire real line. As such it may not be a suitable model for variables that are inherently positive or strongly skewed, such as theweight of a person or the price of ashare. Such variables may be better described by other distributions, such as thelog-normal distribution or thePareto distribution.

The value of the normal density is practically zero when the value⁠x{\displaystyle x}⁠ lies more than a fewstandard deviations away from the mean (e.g., a spread of three standard deviations covers all but 0.27% of the total distribution). Therefore, it may not be an appropriate model when one expects a significant fraction ofoutliers—values that lie many standard deviations away from the mean—and least squares and otherstatistical inference methods that are optimal for normally distributed variables often become highly unreliable when applied to such data. In those cases, a moreheavy-tailed distribution should be assumed and the appropriaterobust statistical inference methods applied.

The Gaussian distribution belongs to the family ofstable distributions which are the attractors of sums ofindependent, identically distributed distributions whether or not the mean or variance is finite. Except for the Gaussian which is a limiting case, all stable distributions have heavy tails and infinite variance. It is one of the few distributions that are stable and that have probability density functions that can be expressed analytically, the others being theCauchy distribution and theLévy distribution.

Symmetries and derivatives

[edit]

The normal distribution with densityf(x){\textstyle f(x)} (mean⁠μ{\displaystyle \mu }⁠ and varianceσ2>0{\textstyle \sigma ^{2}>0}) has the following properties:

Furthermore, the density⁠φ{\displaystyle \varphi }⁠ of the standard normal distribution (i.e.μ=0{\textstyle \mu =0} andσ=1{\textstyle \sigma =1}) also has the following properties:

Moments

[edit]
See also:List of integrals of Gaussian functions

The plain and absolutemoments of a variable⁠X{\displaystyle X}⁠ are the expected values ofXp{\textstyle X^{p}} and|X|p{\textstyle |X|^{p}}, respectively. If the expected value⁠μ{\displaystyle \mu }⁠ of⁠X{\displaystyle X}⁠ is zero, these parameters are calledcentral moments; otherwise, these parameters are callednon-central moments. Usually we are interested only in moments with integer order⁠p{\displaystyle p}⁠.

If⁠X{\displaystyle X}⁠ has a normal distribution, the non-central moments exist and are finite for any⁠p{\displaystyle p}⁠ whose real part is greater than −1. For any non-negative integer⁠p{\displaystyle p}⁠, the plain central moments are:[29]E⁡[(X−μ)p]={0if p is odd,σp(p−1)!!if p is even.{\displaystyle \operatorname {E} \left[(X-\mu )^{p}\right]={\begin{cases}0&{\text{if }}p{\text{ is odd,}}\\\sigma ^{p}(p-1)!!&{\text{if }}p{\text{ is even.}}\end{cases}}}Heren!!{\textstyle n!!} denotes thedouble factorial, that is, the product of all numbers from⁠n{\displaystyle n}⁠ to 1 that have the same parity asn.{\textstyle n.}

The central absolute moments coincide with plain moments for all even orders, but are nonzero for odd orders. For any non-negative integerp,{\textstyle p,}

E⁡[|X−μ|p]=σp(p−1)!!⋅{2πif p is odd1if p is even=σp⋅2p/2Γ(p+12)π.{\displaystyle {\begin{aligned}\operatorname {E} \left[|X-\mu |^{p}\right]&=\sigma ^{p}(p-1)!!\cdot {\begin{cases}{\sqrt {\frac {2}{\pi }}}&{\text{if }}p{\text{ is odd}}\\1&{\text{if }}p{\text{ is even}}\end{cases}}\\[8pt]&=\sigma ^{p}\cdot {\frac {2^{p/2}\Gamma \left({\frac {p+1}{2}}\right)}{\sqrt {\pi }}}.\end{aligned}}}The last formula is valid also for any non-integerp>−1.{\textstyle p>-1.} When the meanμ≠0,{\textstyle \mu \neq 0,} the plain and absolute moments can be expressed in terms ofconfluent hypergeometric functions1F1{\textstyle {}_{1}F_{1}} andU.{\textstyle U.}[30]E⁡[Xp]=σp⋅(−i2)pU(−p2,12,−μ22σ2),E⁡[|X|p]=σp⋅2p/2Γ(1+p2)π1F1(−p2,12,−μ22σ2).{\displaystyle {\begin{aligned}\operatorname {E} \left[X^{p}\right]&=\sigma ^{p}\cdot {\left(-i{\sqrt {2}}\right)}^{p}\,U{\left(-{\frac {p}{2}},{\frac {1}{2}},-{\frac {\mu ^{2}}{2\sigma ^{2}}}\right)},\\\operatorname {E} \left[|X|^{p}\right]&=\sigma ^{p}\cdot 2^{p/2}{\frac {\Gamma {\left({\frac {1+p}{2}}\right)}}{\sqrt {\pi }}}\,{}_{1}F_{1}{\left(-{\frac {p}{2}},{\frac {1}{2}},-{\frac {\mu ^{2}}{2\sigma ^{2}}}\right)}.\end{aligned}}}

These expressions remain valid even if⁠p{\displaystyle p}⁠ is not an integer. See alsogeneralized Hermite polynomials.

OrderNon-central moment,E⁡[Xp]{\displaystyle \operatorname {E} \left[X^{p}\right]}Central moment,E⁡[(X−μ)p]{\displaystyle \operatorname {E} \left[(X-\mu )^{p}\right]}
0⁠1{\displaystyle 1}⁠⁠1{\displaystyle 1}⁠
1⁠μ{\displaystyle \mu }⁠⁠0{\displaystyle 0}⁠
2μ2+σ2{\textstyle \mu ^{2}+\sigma ^{2}}σ2{\textstyle \sigma ^{2}}
3μ3+3μσ2{\textstyle \mu ^{3}+3\mu \sigma ^{2}}⁠0{\displaystyle 0}⁠
4μ4+6μ2σ2+3σ4{\textstyle \mu ^{4}+6\mu ^{2}\sigma ^{2}+3\sigma ^{4}}3σ4{\textstyle 3\sigma ^{4}}
5μ5+10μ3σ2+15μσ4{\textstyle \mu ^{5}+10\mu ^{3}\sigma ^{2}+15\mu \sigma ^{4}}⁠0{\displaystyle 0}⁠
6μ6+15μ4σ2+45μ2σ4+15σ6{\textstyle \mu ^{6}+15\mu ^{4}\sigma ^{2}+45\mu ^{2}\sigma ^{4}+15\sigma ^{6}}15σ6{\textstyle 15\sigma ^{6}}
7μ7+21μ5σ2+105μ3σ4+105μσ6{\textstyle \mu ^{7}+21\mu ^{5}\sigma ^{2}+105\mu ^{3}\sigma ^{4}+105\mu \sigma ^{6}}⁠0{\displaystyle 0}⁠
8μ8+28μ6σ2+210μ4σ4+420μ2σ6+105σ8{\textstyle \mu ^{8}+28\mu ^{6}\sigma ^{2}+210\mu ^{4}\sigma ^{4}+420\mu ^{2}\sigma ^{6}+105\sigma ^{8}}105σ8{\textstyle 105\sigma ^{8}}

The expectation of⁠X{\displaystyle X}⁠ conditioned on the event that⁠X{\displaystyle X}⁠ lies in an interval[a,b]{\textstyle [a,b]} is given byE⁡[X∣a<X<b]=μ−σ2f(b)−f(a)F(b)−F(a),{\displaystyle \operatorname {E} \left[X\mid a<X<b\right]=\mu -\sigma ^{2}{\frac {f(b)-f(a)}{F(b)-F(a)}}\,,}where⁠f{\displaystyle f}⁠ and⁠F{\displaystyle F}⁠ respectively are the density and the cumulative distribution function of⁠X{\displaystyle X}⁠. Forb=∞{\textstyle b=\infty } this is known as theinverse Mills ratio. Note that above, density⁠f{\displaystyle f}⁠ of⁠X{\displaystyle X}⁠ is used instead of standard normal density as in inverse Mills ratio, so here we haveσ2{\textstyle \sigma ^{2}} instead of⁠σ{\displaystyle \sigma }⁠.

Fourier transform and characteristic function

[edit]

TheFourier transform of a normal density⁠f{\displaystyle f}⁠ with mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}} is[31]

f^(t)=∫−∞∞f(x)e−itxdx=e−iμte−12σ2t2,{\displaystyle {\hat {f}}(t)=\int _{-\infty }^{\infty }f(x)e^{-itx}\,dx=e^{-i\mu t}e^{-{\frac {1}{2}}\sigma ^{2}t^{2}}\,,}

where⁠i{\displaystyle i}⁠ is theimaginary unit. If the meanμ=0{\textstyle \mu =0}, the first factor is 1, and the Fourier transform is, apart from a constant factor, a normal density on thefrequency domain, with mean 0 and variance⁠1/σ2{\displaystyle 1/\sigma ^{2}}⁠. In particular, the standard normal distribution⁠φ{\displaystyle \varphi }⁠ is aneigenfunction of the Fourier transform.

In probability theory, the Fourier transform of the probability distribution of a real-valued random variable⁠X{\displaystyle X}⁠ is closely connected to thecharacteristic functionφX(t){\textstyle \varphi _{X}(t)} of that variable, which is defined as theexpected value ofeitX{\textstyle e^{itX}}, as a function of the real variable⁠t{\displaystyle t}⁠ (thefrequency parameter of the Fourier transform). This definition can be analytically extended to a complex-value variable⁠t{\displaystyle t}⁠.[32] The relation between both is:φX(t)=f^(−t).{\displaystyle \varphi _{X}(t)={\hat {f}}(-t)\,.}

The real and imaginary parts off^(t)=E⁡[e−itx]=e−iμte−12σ2t2{\displaystyle {\hat {f}}(t)=\operatorname {E} [e^{-itx}]=e^{-i\mu t}e^{-{\frac {1}{2}}\sigma ^{2}t^{2}}} give:E⁡[cos⁡(tx)]=cos⁡(μt)e−12σ2t2{\displaystyle \operatorname {E} [\cos(tx)]=\cos(\mu t)e^{-{\frac {1}{2}}\sigma ^{2}t^{2}}} andE⁡[sin⁡(tx)]=sin⁡(μt)e−12σ2t2.{\displaystyle \operatorname {E} [\sin(tx)]=\sin(\mu t)e^{-{\frac {1}{2}}\sigma ^{2}t^{2}}.}

Similarly,E⁡[cosh⁡(tx)]=cosh⁡(μt)e12σ2t2{\displaystyle \operatorname {E} [\cosh(tx)]=\cosh(\mu t)e^{{\frac {1}{2}}\sigma ^{2}t^{2}}} andE⁡[sinh⁡(tx)]=sinh⁡(μt)e12σ2t2.{\displaystyle \operatorname {E} [\sinh(tx)]=\sinh(\mu t)e^{{\frac {1}{2}}\sigma ^{2}t^{2}}.}

These formulas evaluated att=1{\displaystyle t=1} give the expected value of these basic trigonometric and hyperbolic functions over a Gaussian random variableX∼N(μ,σ2){\displaystyle X\sim N(\mu ,\sigma ^{2})}, which also could be seen as consequences of theIsserlis's theorem.

Moment- and cumulant-generating functions

[edit]

Themoment generating function of a real random variable⁠X{\displaystyle X}⁠ is the expected value ofetX{\textstyle e^{tX}}, as a function of the real parameter⁠t{\displaystyle t}⁠. For a normal distribution with density⁠f{\displaystyle f}⁠, mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}}, the moment generating function exists and is equal to

M(t)=E⁡[etX]=f^(it)=eμteσ2t2/2.{\displaystyle M(t)=\operatorname {E} \left[e^{tX}\right]={\hat {f}}(it)=e^{\mu t}e^{\sigma ^{2}t^{2}/2}\,.}For any⁠k{\displaystyle k}⁠, the coefficient of⁠tk/k!{\displaystyle t^{k}/k!}⁠ in the moment generating function (expressed as anexponential power series in⁠t{\displaystyle t}⁠) is the normal distribution's expected value⁠E⁡[Xk]{\displaystyle \operatorname {E} [X^{k}]}⁠.

Thecumulant generating function is the logarithm of the moment generating function, namelyg(t)=ln⁡M(t)=μt+12σ2t2.{\displaystyle g(t)=\ln M(t)=\mu t+{\tfrac {1}{2}}\sigma ^{2}t^{2}\,.}

The coefficients of this exponential power series define the cumulants, but because this is a quadratic polynomial in⁠t{\displaystyle t}⁠, only the first twocumulants are nonzero, namely the mean ⁠μ{\displaystyle \mu }⁠ and the variance ⁠σ2{\displaystyle \sigma ^{2}}⁠.

Some authors prefer to instead work with thecharacteristic functionE[eitX] =eiμt −σ2t2/2 andln E[eitX] =iμt −⁠1/2⁠σ2t2.

Stein operator and class

[edit]

WithinStein's method the Stein operator and class of a random variableX∼N(μ,σ2){\textstyle X\sim {\mathcal {N}}(\mu ,\sigma ^{2})} areAf(x)=σ2f′(x)−(x−μ)f(x){\textstyle {\mathcal {A}}f(x)=\sigma ^{2}f'(x)-(x-\mu )f(x)} andF{\textstyle {\mathcal {F}}} the class of all absolutely continuous functions⁠f:R→R{\displaystyle \textstyle f:\mathbb {R} \to \mathbb {R} }⁠ such that⁠E⁡[|f′(X)|]<∞{\displaystyle \operatorname {E} [\vert f'(X)\vert ]<\infty }⁠.

Zero-variance limit

[edit]

In thelimit whenσ2{\textstyle \sigma ^{2}} approaches zero, the probability densityf{\textstyle f} approaches zero everywhere except atμ{\textstyle \mu }, where it approaches∞{\textstyle \infty }, while its integral remains equal to 1. An extension of the normal distribution to the case with zero variance can be defined using theDirac delta measureδμ{\textstyle \delta _{\mu }}, although the resulting random variables are notabsolutely continuous and thus do not haveprobability density functions.The cumulative distribution function of such a random variable is then theHeaviside step function translated by the meanμ{\textstyle \mu }, namelyF(x)={0if x<μ1if x≥μ.{\displaystyle F(x)={\begin{cases}0&{\text{if }}x<\mu \\1&{\text{if }}x\geq \mu .\end{cases}}}

Maximum entropy

[edit]

Of all probability distributions over the reals with a specified finite mean⁠μ{\displaystyle \mu }⁠ and finite variance⁠σ2{\displaystyle \sigma ^{2}}⁠, the normal distributionN(μ,σ2){\textstyle N(\mu ,\sigma ^{2})} is the one withmaximum entropy.[22] To see this, let⁠X{\displaystyle X}⁠ be acontinuous random variable withprobability density⁠f(x){\displaystyle f(x)}⁠. The entropy of⁠X{\displaystyle X}⁠ is defined as[33][34][35]H(X)=−∫−∞∞f(x)ln⁡f(x)dx,{\displaystyle H(X)=-\int _{-\infty }^{\infty }f(x)\ln f(x)\,dx\,,}wheref(x)log⁡f(x){\textstyle f(x)\log f(x)} is understood to be zero whenever⁠f(x)=0{\displaystyle f(x)=0}⁠. This functional can be maximized, subject to the constraints that the distribution is properly normalized and has a specified mean and variance, by usingvariational calculus. A function with threeLagrange multipliers is defined:L=−∫−∞∞f(x)ln⁡f(x)dx−λ0(1−∫−∞∞f(x)dx)−λ1(μ−∫−∞∞f(x)xdx)−λ2(σ2−∫−∞∞f(x)(x−μ)2dx).{\displaystyle L=-\int _{-\infty }^{\infty }f(x)\ln f(x)\,dx-\lambda _{0}\left(1-\int _{-\infty }^{\infty }f(x)\,dx\right)-\lambda _{1}\left(\mu -\int _{-\infty }^{\infty }f(x)x\,dx\right)-\lambda _{2}\left(\sigma ^{2}-\int _{-\infty }^{\infty }f(x)(x-\mu )^{2}\,dx\right)\,.}

At maximum entropy, a small variationδf(x){\textstyle \delta f(x)} aboutf(x){\textstyle f(x)} will produce a variationδL{\textstyle \delta L} about⁠L{\displaystyle L}⁠ which is equal to 0:0=δL=∫−∞∞δf(x)(−ln⁡f(x)−1+λ0+λ1x+λ2(x−μ)2)dx.{\displaystyle 0=\delta L=\int _{-\infty }^{\infty }\delta f(x)\left(-\ln f(x)-1+\lambda _{0}+\lambda _{1}x+\lambda _{2}(x-\mu )^{2}\right)\,dx\,.}

Since this must hold for any small⁠δf(x){\displaystyle \delta f(x)}⁠, the factor multiplying⁠δf(x){\displaystyle \delta f(x)}⁠ must be zero, and solving for⁠f(x){\displaystyle f(x)}⁠ yields:f(x)=exp⁡(−1+λ0+λ1x+λ2(x−μ)2).{\displaystyle f(x)=\exp \left(-1+\lambda _{0}+\lambda _{1}x+\lambda _{2}(x-\mu )^{2}\right)\,.}

The Lagrange constraints that⁠f(x){\displaystyle f(x)}⁠ is properly normalized and has the specified mean and variance are satisfied if and only if⁠λ0{\displaystyle \lambda _{0}}⁠,⁠λ1{\displaystyle \lambda _{1}}⁠, and⁠λ2{\displaystyle \lambda _{2}}⁠ are chosen so thatf(x)=12πσ2e−(x−μ)22σ2.{\displaystyle f(x)={\frac {1}{\sqrt {2\pi \sigma ^{2}}}}e^{-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}}\,.}The entropy of a normal distributionX∼N(μ,σ2){\textstyle X\sim N(\mu ,\sigma ^{2})} is equal toH(X)=12(1+ln⁡2σ2π),{\displaystyle H(X)={\tfrac {1}{2}}(1+\ln 2\sigma ^{2}\pi )\,,}which is independent of the mean⁠μ{\displaystyle \mu }⁠.

Other properties

[edit]
  1. If the characteristic functionϕX{\textstyle \phi _{X}} of some random variable⁠X{\displaystyle X}⁠ is of the formϕX(t)=exp⁡Q(t){\textstyle \phi _{X}(t)=\exp Q(t)} in a neighborhood of zero, whereQ(t){\textstyle Q(t)} is apolynomial, then theMarcinkiewicz theorem (named afterJózef Marcinkiewicz) asserts that⁠Q{\displaystyle Q}⁠ can be at most a quadratic polynomial, and therefore⁠X{\displaystyle X}⁠ is a normal random variable.[36] The consequence of this result is that the normal distribution is the only distribution with a finite number (two) of non-zerocumulants.
  2. If⁠X{\displaystyle X}⁠ and⁠Y{\displaystyle Y}⁠ arejointly normal anduncorrelated, then they areindependent. The requirement that⁠X{\displaystyle X}⁠ and⁠Y{\displaystyle Y}⁠ should bejointly normal is essential; without it the property does not hold.[37][38][proof] For non-normal random variables uncorrelatedness does not imply independence.
  3. TheKullback–Leibler divergence of one normal distributionX1∼N(μ1,σ12){\textstyle X_{1}\sim N(\mu _{1},\sigma _{1}^{2})} from anotherX2∼N(μ2,σ22){\textstyle X_{2}\sim N(\mu _{2},\sigma _{2}^{2})} is given by:[39]DKL(X1∥X2)=(μ1−μ2)22σ22+12(σ12σ22−1−ln⁡σ12σ22){\displaystyle D_{\mathrm {KL} }(X_{1}\parallel X_{2})={\frac {(\mu _{1}-\mu _{2})^{2}}{2\sigma _{2}^{2}}}+{\frac {1}{2}}\left({\frac {\sigma _{1}^{2}}{\sigma _{2}^{2}}}-1-\ln {\frac {\sigma _{1}^{2}}{\sigma _{2}^{2}}}\right)}TheHellinger distance between the same distributions is equal toH2(X1,X2)=1−2σ1σ2σ12+σ22exp⁡(−14(μ1−μ2)2σ12+σ22){\displaystyle H^{2}(X_{1},X_{2})=1-{\sqrt {\frac {2\sigma _{1}\sigma _{2}}{\sigma _{1}^{2}+\sigma _{2}^{2}}}}\exp \left(-{\frac {1}{4}}{\frac {(\mu _{1}-\mu _{2})^{2}}{\sigma _{1}^{2}+\sigma _{2}^{2}}}\right)}
  4. TheFisher information matrix for a normal distribution w.r.t.⁠μ{\displaystyle \mu }⁠ andσ2{\textstyle \sigma ^{2}} is diagonal and takes the formI(μ,σ2)=(1σ20012σ4){\displaystyle {\mathcal {I}}(\mu ,\sigma ^{2})={\begin{pmatrix}{\frac {1}{\sigma ^{2}}}&0\\0&{\frac {1}{2\sigma ^{4}}}\end{pmatrix}}}
  5. Theconjugate prior of the mean of a normal distribution is another normal distribution.[40] Specifically, ifx1,…,xn{\textstyle x_{1},\ldots ,x_{n}} are iid∼N(μ,σ2){\textstyle \sim N(\mu ,\sigma ^{2})} and the prior isμ∼N(μ0,σ02){\textstyle \mu \sim N(\mu _{0},\sigma _{0}^{2})}, then the posterior distribution for the estimator of⁠μ{\displaystyle \mu }⁠ will beμ∣x1,…,xn∼N(σ2nμ0+σ02x¯σ2n+σ02,(nσ2+1σ02)−1){\displaystyle \mu \mid x_{1},\ldots ,x_{n}\sim {\mathcal {N}}\left({\frac {{\frac {\sigma ^{2}}{n}}\mu _{0}+\sigma _{0}^{2}{\bar {x}}}{{\frac {\sigma ^{2}}{n}}+\sigma _{0}^{2}}},\left({\frac {n}{\sigma ^{2}}}+{\frac {1}{\sigma _{0}^{2}}}\right)^{-1}\right)}
  6. The family of normal distributions not only forms anexponential family (EF), but in fact forms anatural exponential family (NEF) with quadraticvariance function (NEF-QVF). Many properties of normal distributions generalize to properties of NEF-QVF distributions, NEF distributions, or EF distributions generally. NEF-QVF distributions comprises 6 families, including Poisson, Gamma, binomial, and negative binomial distributions, while many of the common families studied in probability and statistics are NEF or EF.
  7. Ininformation geometry, the family of normal distributions forms astatistical manifold withconstant curvature⁠−1{\displaystyle -1}⁠. The same family isflat with respect to the (±1)-connections∇(e){\textstyle \nabla ^{(e)}} and∇(m){\textstyle \nabla ^{(m)}}.[41]
  8. IfX1,…,Xn{\textstyle X_{1},\dots ,X_{n}} are distributed according toN(0,σ2){\textstyle N(0,\sigma ^{2})}, thenE[maxiXi]≤σ2ln⁡n{\textstyle E[\max _{i}X_{i}]\leq \sigma {\sqrt {2\ln n}}}. Note that there is no assumption of independence.[42]

Related distributions

[edit]

Central limit theorem

[edit]
As the number of discrete events increases, the function begins to resemble a normal distribution.
Comparison of probability density functions,p(k) for the sum ofn fair 6-sided dice to show their convergence to a normal distribution with increasingna, in accordance to the central limit theorem. In the bottom-right graph, smoothed profiles of the previous graphs are rescaled, superimposed and compared with a normal distribution (black curve).
Main article:Central limit theorem

The central limit theorem states that under certain (fairly common) conditions, the sum of many random variables will have an approximately normal distribution. More specifically, whereX1,…,Xn{\textstyle X_{1},\ldots ,X_{n}} areindependent and identically distributed random variables with the same arbitrary distribution, zero mean, and varianceσ2{\textstyle \sigma ^{2}} and⁠Z{\displaystyle Z}⁠ is theirmean scaled byn{\textstyle {\sqrt {n}}}Z=n(1n∑i=1nXi){\displaystyle Z={\sqrt {n}}\left({\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right)}Then, as⁠n{\displaystyle n}⁠ increases, the probability distribution of⁠Z{\displaystyle Z}⁠ will tend to the normal distribution with zero mean and variance⁠σ2{\displaystyle \sigma ^{2}}⁠.

The theorem can be extended to variables(Xi){\textstyle (X_{i})} that are not independent and/or not identically distributed if certain constraints are placed on the degree of dependence and the moments of the distributions.

Manytest statistics,scores, andestimators encountered in practice contain sums of certain random variables in them, and even more estimators can be represented as sums of random variables through the use ofinfluence functions. The central limit theorem implies that those statistical parameters will have asymptotically normal distributions.

The central limit theorem also implies that certain distributions can be approximated by the normal distribution, for example:

Whether these approximations are sufficiently accurate depends on the purpose for which they are needed, and the rate of convergence to the normal distribution. It is typically the case that such approximations are less accurate in the tails of the distribution.

A general upper bound for the approximation error in the central limit theorem is given by theBerry–Esseen theorem, improvements of the approximation are given by theEdgeworth expansions.

This theorem can also be used to justify modeling the sum of many uniform noise sources asGaussian noise. SeeAWGN.

Operations and functions of normal variables

[edit]

Operations on a single normal variable

[edit]

If⁠X{\displaystyle X}⁠ is distributed normally with mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}}, then

Operations on two independent normal variables
[edit]
Operations on two independent standard normal variables
[edit]

IfX1{\textstyle X_{1}} andX2{\textstyle X_{2}} are two independent standard normal random variables with mean 0 and variance 1, then

Operations on multiple independent normal variables

[edit]

Operations on multiple correlated normal variables

[edit]

Operations on the density function

[edit]

Thesplit normal distribution is most directly defined in terms of joining scaled sections of the density functions of different normal distributions and rescaling the density to integrate to one. Thetruncated normal distribution results from rescaling a section of a single density function.

Infinite divisibility and Cramér's theorem

[edit]

For any positive integern, any normal distribution with mean⁠μ{\displaystyle \mu }⁠ and varianceσ2{\textstyle \sigma ^{2}} is the distribution of the sum ofn independent normal deviates, each with meanμn{\textstyle {\frac {\mu }{n}}} and varianceσ2n{\textstyle {\frac {\sigma ^{2}}{n}}}. This property is calledinfinite divisibility.[49]

Conversely, ifX1{\textstyle X_{1}} andX2{\textstyle X_{2}} are independent random variables and their sumX1+X2{\textstyle X_{1}+X_{2}} has a normal distribution, then bothX1{\textstyle X_{1}} andX2{\textstyle X_{2}} must be normal deviates.[50]

This result is known asCramér's decomposition theorem, and is equivalent to saying that theconvolution of two distributions is normal if and only if both are normal. Cramér's theorem implies that a linear combination of independent non-Gaussian variables will never have an exactly normal distribution, although it may approach it arbitrarily closely.[36]

The Kac–Bernstein theorem

[edit]

TheKac–Bernstein theorem states that ifX{\textstyle X} and⁠Y{\displaystyle Y}⁠ are independent andX+Y{\textstyle X+Y} andX−Y{\textstyle X-Y} are also independent, then bothX andY must necessarily have normal distributions.[51][52]

More generally, ifX1,…,Xn{\textstyle X_{1},\ldots ,X_{n}} are independent random variables, then two distinct linear combinations∑akXk{\textstyle \sum {a_{k}X_{k}}} and∑bkXk{\textstyle \sum {b_{k}X_{k}}}will be independent if and only if allXk{\textstyle X_{k}} are normal and∑akbkσk2=0{\textstyle \sum {a_{k}b_{k}\sigma _{k}^{2}=0}}, whereσk2{\textstyle \sigma _{k}^{2}} denotes the variance ofXk{\textstyle X_{k}}.[51]

Extensions

[edit]

The notion of normal distribution, being one of the most important distributions in probability theory, has been extended far beyond the standard framework of the univariate (that is one-dimensional) case (Case 1). All these extensions are also callednormal orGaussian laws, so a certain ambiguity in names exists.

A random variableX has a two-piece normal distribution if it has a distributionfX(x)={N(μ,σ12), if x≤μN(μ,σ22), if x≥μ{\displaystyle f_{X}(x)={\begin{cases}N(\mu ,\sigma _{1}^{2}),&{\text{ if }}x\leq \mu \\N(\mu ,\sigma _{2}^{2}),&{\text{ if }}x\geq \mu \end{cases}}}whereμ is the mean andσ2
1  andσ2
2  are the variances of the distribution to the left and right of the mean respectively.

The meanE(X), varianceV(X), and third central momentT(X) of this distribution have been determined[53]E⁡(X)=μ+2π(σ2−σ1),V⁡(X)=(1−2π)(σ2−σ1)2+σ1σ2,T⁡(X)=2π(σ2−σ1)[(4π−1)(σ2−σ1)2+σ1σ2].{\displaystyle {\begin{aligned}\operatorname {E} (X)&=\mu +{\sqrt {\frac {2}{\pi }}}(\sigma _{2}-\sigma _{1}),\\\operatorname {V} (X)&=\left(1-{\frac {2}{\pi }}\right)(\sigma _{2}-\sigma _{1})^{2}+\sigma _{1}\sigma _{2},\\\operatorname {T} (X)&={\sqrt {\frac {2}{\pi }}}(\sigma _{2}-\sigma _{1})\left[\left({\frac {4}{\pi }}-1\right)(\sigma _{2}-\sigma _{1})^{2}+\sigma _{1}\sigma _{2}\right].\end{aligned}}}

One of the main practical uses of the Gaussian law is to model the empirical distributions of many different random variables encountered in practice. In such case a possible extension would be a richer family of distributions, having more than two parameters and therefore being able to fit the empirical distribution more accurately. The examples of such extensions are:

  • Pearson distribution — a four-parameter family of probability distributions that extend the normal law to include different skewness and kurtosis values.
  • Thegeneralized normal distribution, also known as the exponential power distribution, allows for distribution tails with thicker or thinner asymptotic behaviors.

Statistical inference

[edit]

Estimation of parameters

[edit]
See also:Maximum likelihood § Continuous distribution, continuous parameter space; andGaussian function § Estimation of parameters

It is often the case that we do not know the parameters of the normal distribution, but instead want toestimate them. That is, having a sample(x1,…,xn){\textstyle (x_{1},\ldots ,x_{n})} from a normalN(μ,σ2){\textstyle {\mathcal {N}}(\mu ,\sigma ^{2})} population we would like to learn the approximate values of parameters⁠μ{\displaystyle \mu }⁠ andσ2{\textstyle \sigma ^{2}}. The standard approach to this problem is themaximum likelihood method, which requires maximization of thelog-likelihood function:ln⁡L(μ,σ2)=∑i=1nln⁡f(xi∣μ,σ2)=−n2ln⁡(2π)−n2ln⁡σ2−12σ2∑i=1n(xi−μ)2.{\displaystyle \ln {\mathcal {L}}(\mu ,\sigma ^{2})=\sum _{i=1}^{n}\ln f(x_{i}\mid \mu ,\sigma ^{2})=-{\frac {n}{2}}\ln(2\pi )-{\frac {n}{2}}\ln \sigma ^{2}-{\frac {1}{2\sigma ^{2}}}\sum _{i=1}^{n}(x_{i}-\mu )^{2}.}Taking derivatives with respect to⁠μ{\displaystyle \mu }⁠ andσ2{\textstyle \sigma ^{2}} and solving the resulting system of first order conditions yields themaximum likelihood estimates:μ^=x¯≡1n∑i=1nxi,σ^2=1n∑i=1n(xi−x¯)2.{\displaystyle {\hat {\mu }}={\overline {x}}\equiv {\frac {1}{n}}\sum _{i=1}^{n}x_{i},\qquad {\hat {\sigma }}^{2}={\frac {1}{n}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}.}

Thenln⁡L(μ^,σ^2){\textstyle \ln {\mathcal {L}}({\hat {\mu }},{\hat {\sigma }}^{2})} is as follows:ln⁡L(μ^,σ^2)=(−n/2)[ln⁡(2πσ^2)+1]{\displaystyle \ln {\mathcal {L}}({\hat {\mu }},{\hat {\sigma }}^{2})=(-n/2)[\ln(2\pi {\hat {\sigma }}^{2})+1]}

Sample mean

[edit]
See also:Standard error of the mean

Estimatorμ^{\displaystyle \textstyle {\hat {\mu }}} is called thesample mean, since it is the arithmetic mean of all observations. The statisticx¯{\displaystyle \textstyle {\overline {x}}} iscomplete andsufficient for⁠μ{\displaystyle \mu }⁠, and therefore by theLehmann–Scheffé theorem,μ^{\displaystyle \textstyle {\hat {\mu }}} is theuniformly minimum variance unbiased (UMVU) estimator.[54] In finite samples it is distributed normally:μ^∼N(μ,σ2/n).{\displaystyle {\hat {\mu }}\sim {\mathcal {N}}(\mu ,\sigma ^{2}/n).}The variance of this estimator is equal to theμμ-element of the inverseFisher information matrixI−1{\displaystyle \textstyle {\mathcal {I}}^{-1}}. This implies that the estimator isfinite-sample efficient. Of practical importance is thestandard error ofμ^{\displaystyle \textstyle {\hat {\mu }}} being proportional to1/n{\displaystyle \textstyle 1/{\sqrt {n}}}, that is, if one wishes to decrease the standard error by a factor of 10, one must increase the number of points in the sample by a factor of 100. This fact is widely used in determining sample sizes for opinion polls and the number of trials inMonte Carlo simulations.

From the standpoint of theasymptotic theory,μ^{\displaystyle \textstyle {\hat {\mu }}} isconsistent, that is, itconverges in probability to⁠μ{\displaystyle \mu }⁠ asn→∞{\textstyle n\rightarrow \infty }. The estimator is alsoasymptotically normal, which is a simple corollary of it being normal in finite samples:n(μ^−μ)→dN(0,σ2).{\displaystyle {\sqrt {n}}({\hat {\mu }}-\mu )\,\xrightarrow {d} \,{\mathcal {N}}(0,\sigma ^{2}).}

Sample variance

[edit]
See also:Standard deviation § Estimation, andVariance § Estimation

The estimatorσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} is called thesample variance, since it is the variance of the sample ((x1,…,xn){\textstyle (x_{1},\ldots ,x_{n})}). In practice, another estimator is often used instead of theσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}}. This other estimator is denoteds2{\textstyle s^{2}}, and is also called thesample variance, which represents a certain ambiguity in terminology; its square root⁠s{\displaystyle s}⁠ is called thesample standard deviation. The estimators2{\textstyle s^{2}} differs fromσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} by having(n − 1) instead of n in the denominator (the so-calledBessel's correction):s2=nn−1σ^2=1n−1∑i=1n(xi−x¯)2.{\displaystyle s^{2}={\frac {n}{n-1}}{\hat {\sigma }}^{2}={\frac {1}{n-1}}\sum _{i=1}^{n}(x_{i}-{\overline {x}})^{2}.}The difference betweens2{\textstyle s^{2}} andσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} becomes negligibly small for largen's. In finite samples however, the motivation behind the use ofs2{\textstyle s^{2}} is that it is anunbiased estimator of the underlying parameterσ2{\textstyle \sigma ^{2}}, whereasσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} is biased. Also, by the Lehmann–Scheffé theorem the estimators2{\textstyle s^{2}} is uniformly minimum variance unbiased (UMVU),[54] which makes it the "best" estimator among all unbiased ones. However it can be shown that the biased estimatorσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} is better than thes2{\textstyle s^{2}} in terms of themean squared error (MSE) criterion. In finite samples boths2{\textstyle s^{2}} andσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} have scaledchi-squared distribution with(n − 1) degrees of freedom:s2∼σ2n−1⋅χn−12,σ^2∼σ2n⋅χn−12.{\displaystyle s^{2}\sim {\frac {\sigma ^{2}}{n-1}}\cdot \chi _{n-1}^{2},\qquad {\hat {\sigma }}^{2}\sim {\frac {\sigma ^{2}}{n}}\cdot \chi _{n-1}^{2}.}The first of these expressions shows that the variance ofs2{\textstyle s^{2}} is equal to2σ4/(n−1){\textstyle 2\sigma ^{4}/(n-1)}, which is slightly greater than theσσ-element of the inverse Fisher information matrixI−1{\displaystyle \textstyle {\mathcal {I}}^{-1}}, which is2σ4/n{\textstyle 2\sigma ^{4}/n}. Thus,s2{\textstyle s^{2}} is not an efficient estimator forσ2{\textstyle \sigma ^{2}}, and moreover, sinces2{\textstyle s^{2}} is UMVU, we can conclude that the finite-sample efficient estimator forσ2{\textstyle \sigma ^{2}} does not exist.

Applying the asymptotic theory, both estimatorss2{\textstyle s^{2}} andσ^2{\displaystyle \textstyle {\hat {\sigma }}^{2}} are consistent, that is they converge in probability toσ2{\textstyle \sigma ^{2}} as the sample sizen→∞{\textstyle n\rightarrow \infty }. The two estimators are also both asymptotically normal:n(σ^2−σ2)≃n(s2−σ2)→dN(0,2σ4).{\displaystyle {\sqrt {n}}({\hat {\sigma }}^{2}-\sigma ^{2})\simeq {\sqrt {n}}(s^{2}-\sigma ^{2})\,\xrightarrow {d} \,{\mathcal {N}}(0,2\sigma ^{4}).}In particular, both estimators are asymptotically efficient forσ2{\textstyle \sigma ^{2}}.

Confidence intervals

[edit]
See also:Studentization and3-sigma rule

ByCochran's theorem, for normal distributions the sample meanμ^{\displaystyle \textstyle {\hat {\mu }}} and the sample variances2 areindependent, which means there can be no gain in considering theirjoint distribution. There is also a converse theorem: if in a sample the sample mean and sample variance are independent, then the sample must have come from the normal distribution. The independence betweenμ^{\displaystyle \textstyle {\hat {\mu }}} ands can be employed to construct the so-calledt-statistic:t=μ^−μs/n=x¯−μ1n(n−1)∑(xi−x¯)2∼tn−1{\displaystyle t={\frac {{\hat {\mu }}-\mu }{s/{\sqrt {n}}}}={\frac {{\overline {x}}-\mu }{\sqrt {{\frac {1}{n(n-1)}}\sum (x_{i}-{\overline {x}})^{2}}}}\sim t_{n-1}}This quantityt has theStudent's t-distribution with(n − 1) degrees of freedom, and it is anancillary statistic (independent of the value of the parameters). Inverting the distribution of thist-statistics will allow us to construct theconfidence interval forμ;[55] similarly, inverting theχ2 distribution of the statistics2 will give us the confidence interval forσ2:[56]μ∈[μ^−tn−1,1−α/2sn,μ^+tn−1,1−α/2sn]{\displaystyle \mu \in \left[{\hat {\mu }}-t_{n-1,1-\alpha /2}{\frac {s}{\sqrt {n}}},\,{\hat {\mu }}+t_{n-1,1-\alpha /2}{\frac {s}{\sqrt {n}}}\right]}σ2∈[n−1χn−1,1−α/22s2,n−1χn−1,α/22s2]{\displaystyle \sigma ^{2}\in \left[{\frac {n-1}{\chi _{n-1,1-\alpha /2}^{2}}}s^{2},\,{\frac {n-1}{\chi _{n-1,\alpha /2}^{2}}}s^{2}\right]}wheretk,p andχ 2
k,p  are thepthquantiles of thet- andχ2-distributions respectively. These confidence intervals are of theconfidence level1 −α, meaning that the true valuesμ andσ2 fall outside of these intervals with probability (orsignificance level)α. In practice people usually takeα = 5%, resulting in the 95% confidence intervals. The confidence interval forσ can be found by taking the square root of the interval bounds forσ2.

Approximate formulas can be derived from the asymptotic distributions ofμ^{\displaystyle \textstyle {\hat {\mu }}} ands2:μ∈[μ^−|zα/2|ns,μ^+|zα/2|ns]{\displaystyle \mu \in \left[{\hat {\mu }}-{\frac {|z_{\alpha /2}|}{\sqrt {n}}}s,\,{\hat {\mu }}+{\frac {|z_{\alpha /2}|}{\sqrt {n}}}s\right]}σ2∈[s2−2|zα/2|ns2,s2+2|zα/2|ns2]{\displaystyle \sigma ^{2}\in \left[s^{2}-{\sqrt {2}}{\frac {|z_{\alpha /2}|}{\sqrt {n}}}s^{2},\,s^{2}+{\sqrt {2}}{\frac {|z_{\alpha /2}|}{\sqrt {n}}}s^{2}\right]}The approximate formulas become valid for large values ofn, and are more convenient for the manual calculation since the standard normal quantileszα/2 do not depend onn. In particular, the most popular value ofα = 5%, results in|z0.025| =1.96.

Normality tests

[edit]
Main article:Normality tests

Normality tests assess the likelihood that the given data set{x1, ...,xn} comes from a normal distribution. Typically thenull hypothesisH0 is that the observations are distributed normally with unspecified meanμ and varianceσ2, versus the alternativeHa that the distribution is arbitrary. Many tests (over 40) have been devised for this problem. The more prominent of them are outlined below:

Diagnostic plots are more intuitively appealing but subjective at the same time, as they rely on informal human judgement to accept or reject the null hypothesis.

  • Q–Q plot, also known asnormal probability plot orrankit plot—is a plot of the sorted values from the data set against the expected values of the corresponding quantiles from the standard normal distribution. That is, it is a plot of point of the form(Φ−1(pk),x(k)), where plotting pointspk are equal topk = (k −α)/(n + 1 − 2α) andα is an adjustment constant, which can be anything between 0 and 1. If the null hypothesis is true, the plotted points should approximately lie on a straight line.
  • P–P plot – similar to the Q–Q plot, but used much less frequently. This method consists of plotting the points(Φ(z(k)),pk), wherez(k)=(x(k)−μ^)/σ^{\textstyle \textstyle z_{(k)}=(x_{(k)}-{\hat {\mu }})/{\hat {\sigma }}}. For normally distributed data this plot should lie on a straight line between(0, 0) and (1, 1).

Goodness-of-fit tests:

Moment-based tests:

Tests based on the empirical distribution function:

Bayesian analysis of the normal distribution

[edit]

Bayesian analysis of normally distributed data is complicated by the many different possibilities that may be considered:

  • Either the mean, or the variance, or neither, may be considered a fixed quantity.
  • When the variance is unknown, analysis may be done directly in terms of the variance, or in terms of theprecision, the reciprocal of the variance. The reason for expressing the formulas in terms of precision is that the analysis of most cases is simplified.
  • Both univariate andmultivariate cases need to be considered.
  • Eitherconjugate orimproperprior distributions may be placed on the unknown variables.
  • An additional set of cases occurs inBayesian linear regression, where in the basic model the data is assumed to be normally distributed, and normal priors are placed on theregression coefficients. The resulting analysis is similar to the basic cases ofindependent identically distributed data.

The formulas for the non-linear-regression cases are summarized in theconjugate prior article.

Sum of two quadratics

[edit]
Scalar form
[edit]

The following auxiliary formula is useful for simplifying theposterior update equations, which otherwise become fairly tedious.

a(x−y)2+b(x−z)2=(a+b)(x−ay+bza+b)2+aba+b(y−z)2{\displaystyle a(x-y)^{2}+b(x-z)^{2}=(a+b)\left(x-{\frac {ay+bz}{a+b}}\right)^{2}+{\frac {ab}{a+b}}(y-z)^{2}}

This equation rewrites the sum of two quadratics inx by expanding the squares, grouping the terms inx, andcompleting the square. Note the following about the complex constant factors attached to some of the terms:

  1. The factoray+bza+b{\textstyle {\frac {ay+bz}{a+b}}} has the form of aweighted average ofy andz.
  2. aba+b=11a+1b=(a−1+b−1)−1.{\textstyle {\frac {ab}{a+b}}={\frac {1}{{\frac {1}{a}}+{\frac {1}{b}}}}=(a^{-1}+b^{-1})^{-1}.} This shows that this factor can be thought of as resulting from a situation where thereciprocals of quantitiesa andb add directly, so to combinea andb themselves, it is necessary to reciprocate, add, and reciprocate the result again to get back into the original units. This is exactly the sort of operation performed by theharmonic mean, so it is not surprising thataba+b{\textstyle {\frac {ab}{a+b}}} is one-half theharmonic mean ofa andb.
Vector form
[edit]

A similar formula can be written for the sum of two vector quadratics: Ifx,y,z are vectors of lengthk, andA andB aresymmetric,invertible matrices of sizek×k{\textstyle k\times k}, then

(y−x)′A(y−x)+(x−z)′B(x−z)=(x−c)′(A+B)(x−c)+(y−z)′(A−1+B−1)−1(y−z){\displaystyle {\begin{aligned}&(\mathbf {y} -\mathbf {x} )'\mathbf {A} (\mathbf {y} -\mathbf {x} )+(\mathbf {x} -\mathbf {z} )'\mathbf {B} (\mathbf {x} -\mathbf {z} )\\={}&(\mathbf {x} -\mathbf {c} )'(\mathbf {A} +\mathbf {B} )(\mathbf {x} -\mathbf {c} )+(\mathbf {y} -\mathbf {z} )'(\mathbf {A} ^{-1}+\mathbf {B} ^{-1})^{-1}(\mathbf {y} -\mathbf {z} )\end{aligned}}}wherec=(A+B)−1(Ay+Bz){\displaystyle \mathbf {c} =(\mathbf {A} +\mathbf {B} )^{-1}(\mathbf {A} \mathbf {y} +\mathbf {B} \mathbf {z} )}

The formx′Ax is called aquadratic form and is ascalar:x′Ax=∑i,jaijxixj{\displaystyle \mathbf {x} '\mathbf {A} \mathbf {x} =\sum _{i,j}a_{ij}x_{i}x_{j}}In other words, it sums up all possible combinations of products of pairs of elements fromx, with a separate coefficient for each. In addition, sincexixj=xjxi{\textstyle x_{i}x_{j}=x_{j}x_{i}}, only the sumaij+aji{\textstyle a_{ij}+a_{ji}} matters for any off-diagonal elements ofA, and there is no loss of generality in assuming thatA issymmetric. Furthermore, ifA is symmetric, then the formx′Ay=y′Ax.{\textstyle \mathbf {x} '\mathbf {A} \mathbf {y} =\mathbf {y} '\mathbf {A} \mathbf {x} .}

Sum of differences from the mean

[edit]

Another useful formula is as follows:∑i=1n(xi−μ)2=∑i=1n(xi−x¯)2+n(x¯−μ)2{\displaystyle \sum _{i=1}^{n}(x_{i}-\mu )^{2}=\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+n({\bar {x}}-\mu )^{2}}wherex¯=1n∑i=1nxi.{\textstyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}.}

With known variance

[edit]

For a set ofi.i.d. normally distributed data pointsX of sizen where each individual pointx followsx∼N(μ,σ2){\textstyle x\sim {\mathcal {N}}(\mu ,\sigma ^{2})} with knownvarianceσ2, theconjugate prior distribution is also normally distributed.

This can be shown more easily by rewriting the variance as theprecision, i.e. usingτ = 1/σ2. Then ifx∼N(μ,1/τ){\textstyle x\sim {\mathcal {N}}(\mu ,1/\tau )} andμ∼N(μ0,1/τ0),{\textstyle \mu \sim {\mathcal {N}}(\mu _{0},1/\tau _{0}),} we proceed as follows.

First, thelikelihood function is (using the formula above for the sum of differences from the mean):p(X∣μ,τ)=∏i=1nτ2πexp⁡(−12τ(xi−μ)2)=(τ2π)n/2exp⁡(−12τ∑i=1n(xi−μ)2)=(τ2π)n/2exp⁡[−12τ(∑i=1n(xi−x¯)2+n(x¯−μ)2)].{\displaystyle {\begin{aligned}p(\mathbf {X} \mid \mu ,\tau )&=\prod _{i=1}^{n}{\sqrt {\frac {\tau }{2\pi }}}\exp \left(-{\frac {1}{2}}\tau (x_{i}-\mu )^{2}\right)\\&=\left({\frac {\tau }{2\pi }}\right)^{n/2}\exp \left(-{\frac {1}{2}}\tau \sum _{i=1}^{n}(x_{i}-\mu )^{2}\right)\\&=\left({\frac {\tau }{2\pi }}\right)^{n/2}\exp \left[-{\frac {1}{2}}\tau \left(\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+n({\bar {x}}-\mu )^{2}\right)\right].\end{aligned}}}

Then, we proceed as follows:p(μ∣X)∝p(X∣μ)p(μ)=(τ2π)n/2exp⁡[−12τ(∑i=1n(xi−x¯)2+n(x¯−μ)2)]τ02πexp⁡(−12τ0(μ−μ0)2)∝exp⁡(−12(τ(∑i=1n(xi−x¯)2+n(x¯−μ)2)+τ0(μ−μ0)2))∝exp⁡(−12(nτ(x¯−μ)2+τ0(μ−μ0)2))=exp⁡(−12(nτ+τ0)(μ−nτx¯+τ0μ0nτ+τ0)2+nττ0nτ+τ0(x¯−μ0)2)∝exp⁡(−12(nτ+τ0)(μ−nτx¯+τ0μ0nτ+τ0)2){\displaystyle {\begin{aligned}p(\mu \mid \mathbf {X} )&\propto p(\mathbf {X} \mid \mu )p(\mu )\\&=\left({\frac {\tau }{2\pi }}\right)^{n/2}\exp \left[-{\frac {1}{2}}\tau \left(\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+n({\bar {x}}-\mu )^{2}\right)\right]{\sqrt {\frac {\tau _{0}}{2\pi }}}\exp \left(-{\frac {1}{2}}\tau _{0}(\mu -\mu _{0})^{2}\right)\\&\propto \exp \left(-{\frac {1}{2}}\left(\tau \left(\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+n({\bar {x}}-\mu )^{2}\right)+\tau _{0}(\mu -\mu _{0})^{2}\right)\right)\\&\propto \exp \left(-{\frac {1}{2}}\left(n\tau ({\bar {x}}-\mu )^{2}+\tau _{0}(\mu -\mu _{0})^{2}\right)\right)\\&=\exp \left(-{\frac {1}{2}}(n\tau +\tau _{0})\left(\mu -{\dfrac {n\tau {\bar {x}}+\tau _{0}\mu _{0}}{n\tau +\tau _{0}}}\right)^{2}+{\frac {n\tau \tau _{0}}{n\tau +\tau _{0}}}({\bar {x}}-\mu _{0})^{2}\right)\\&\propto \exp \left(-{\frac {1}{2}}(n\tau +\tau _{0})\left(\mu -{\dfrac {n\tau {\bar {x}}+\tau _{0}\mu _{0}}{n\tau +\tau _{0}}}\right)^{2}\right)\end{aligned}}}

In the above derivation, we used the formula above for the sum of two quadratics and eliminated all constant factors not involving μ. The result is thekernel of a normal distribution, with meannτx¯+τ0μ0nτ+τ0{\textstyle {\frac {n\tau {\bar {x}}+\tau _{0}\mu _{0}}{n\tau +\tau _{0}}}} and precisionnτ+τ0{\textstyle n\tau +\tau _{0}}, i.e.p(μ∣X)∼N(nτx¯+τ0μ0nτ+τ0,1nτ+τ0){\displaystyle p(\mu \mid \mathbf {X} )\sim {\mathcal {N}}\left({\frac {n\tau {\bar {x}}+\tau _{0}\mu _{0}}{n\tau +\tau _{0}}},{\frac {1}{n\tau +\tau _{0}}}\right)}

This can be written as a set of Bayesian update equations for the posterior parameters in terms of the prior parameters:τ0′=τ0+nτμ0′=nτx¯+τ0μ0nτ+τ0x¯=1n∑i=1nxi{\displaystyle {\begin{aligned}\tau _{0}'&=\tau _{0}+n\tau \\[5pt]\mu _{0}'&={\frac {n\tau {\bar {x}}+\tau _{0}\mu _{0}}{n\tau +\tau _{0}}}\\[5pt]{\bar {x}}&={\frac {1}{n}}\sum _{i=1}^{n}x_{i}\end{aligned}}}

That is, to combinen data points with total precision ofnτ (or equivalently, total variance ofn/σ2) and mean of valuesx¯{\textstyle {\bar {x}}}, derive a new total precision simply by adding the total precision of the data to the prior total precision, and form a new mean through aprecision-weighted average, i.e. aweighted average of the data mean and the prior mean, each weighted by the associated total precision. This makes logical sense if the precision is thought of as indicating the certainty of the observations: In the distribution of the posterior mean, each of the input components is weighted by its certainty, and the certainty of this distribution is the sum of the individual certainties. (For the intuition of this, compare the expression "the whole is (or is not) greater than the sum of its parts". In addition, consider that the knowledge of the posterior comes from a combination of the knowledge of the prior and likelihood, so it makes sense that we are more certain of it than of either of its components.)

The above formula reveals why it is more convenient to doBayesian analysis ofconjugate priors for the normal distribution in terms of the precision. The posterior precision is simply the sum of the prior and likelihood precisions, and the posterior mean is computed through a precision-weighted average, as described above. The same formulas can be written in terms of variance by reciprocating all the precisions, yielding the more ugly formulasσ02′=1nσ2+1σ02μ0′=nx¯σ2+μ0σ02nσ2+1σ02x¯=1n∑i=1nxi{\displaystyle {\begin{aligned}{\sigma _{0}^{2}}'&={\frac {1}{{\frac {n}{\sigma ^{2}}}+{\frac {1}{\sigma _{0}^{2}}}}}\\[5pt]\mu _{0}'&={\frac {{\frac {n{\bar {x}}}{\sigma ^{2}}}+{\frac {\mu _{0}}{\sigma _{0}^{2}}}}{{\frac {n}{\sigma ^{2}}}+{\frac {1}{\sigma _{0}^{2}}}}}\\[5pt]{\bar {x}}&={\frac {1}{n}}\sum _{i=1}^{n}x_{i}\end{aligned}}}

With known mean

[edit]

For a set ofi.i.d. normally distributed data pointsX of sizen where each individual pointx followsx∼N(μ,σ2){\textstyle x\sim {\mathcal {N}}(\mu ,\sigma ^{2})} with known meanμ, theconjugate prior of thevariance has aninverse gamma distribution or ascaled inverse chi-squared distribution. The two are equivalent except for having differentparameterizations. Although the inverse gamma is more commonly used, we use the scaled inverse chi-squared for the sake of convenience. The prior forσ2 is as follows:p(σ2∣ν0,σ02)=(σ02ν02)ν0/2Γ(ν02) exp⁡[−ν0σ022σ2](σ2)1+ν02∝exp⁡[−ν0σ022σ2](σ2)1+ν02{\displaystyle p(\sigma ^{2}\mid \nu _{0},\sigma _{0}^{2})={\frac {(\sigma _{0}^{2}{\frac {\nu _{0}}{2}})^{\nu _{0}/2}}{\Gamma \left({\frac {\nu _{0}}{2}}\right)}}~{\frac {\exp \left[{\frac {-\nu _{0}\sigma _{0}^{2}}{2\sigma ^{2}}}\right]}{(\sigma ^{2})^{1+{\frac {\nu _{0}}{2}}}}}\propto {\frac {\exp \left[{\frac {-\nu _{0}\sigma _{0}^{2}}{2\sigma ^{2}}}\right]}{(\sigma ^{2})^{1+{\frac {\nu _{0}}{2}}}}}}

Thelikelihood function from above, written in terms of the variance, is:p(X∣μ,σ2)=(12πσ2)n/2exp⁡[−12σ2∑i=1n(xi−μ)2]=(12πσ2)n/2exp⁡[−S2σ2]{\displaystyle {\begin{aligned}p(\mathbf {X} \mid \mu ,\sigma ^{2})&=\left({\frac {1}{2\pi \sigma ^{2}}}\right)^{n/2}\exp \left[-{\frac {1}{2\sigma ^{2}}}\sum _{i=1}^{n}(x_{i}-\mu )^{2}\right]\\&=\left({\frac {1}{2\pi \sigma ^{2}}}\right)^{n/2}\exp \left[-{\frac {S}{2\sigma ^{2}}}\right]\end{aligned}}}whereS=∑i=1n(xi−μ)2.{\displaystyle S=\sum _{i=1}^{n}(x_{i}-\mu )^{2}.}

Then:p(σ2∣X)∝p(X∣σ2)p(σ2)=(12πσ2)n/2exp⁡[−S2σ2](σ02ν02)ν02Γ(ν02) exp⁡[−ν0σ022σ2](σ2)1+ν02∝(1σ2)n/21(σ2)1+ν02exp⁡[−S2σ2+−ν0σ022σ2]=1(σ2)1+ν0+n2exp⁡[−ν0σ02+S2σ2]{\displaystyle {\begin{aligned}p(\sigma ^{2}\mid \mathbf {X} )&\propto p(\mathbf {X} \mid \sigma ^{2})p(\sigma ^{2})\\&=\left({\frac {1}{2\pi \sigma ^{2}}}\right)^{n/2}\exp \left[-{\frac {S}{2\sigma ^{2}}}\right]{\frac {(\sigma _{0}^{2}{\frac {\nu _{0}}{2}})^{\frac {\nu _{0}}{2}}}{\Gamma \left({\frac {\nu _{0}}{2}}\right)}}~{\frac {\exp \left[{\frac {-\nu _{0}\sigma _{0}^{2}}{2\sigma ^{2}}}\right]}{(\sigma ^{2})^{1+{\frac {\nu _{0}}{2}}}}}\\&\propto \left({\frac {1}{\sigma ^{2}}}\right)^{n/2}{\frac {1}{(\sigma ^{2})^{1+{\frac {\nu _{0}}{2}}}}}\exp \left[-{\frac {S}{2\sigma ^{2}}}+{\frac {-\nu _{0}\sigma _{0}^{2}}{2\sigma ^{2}}}\right]\\&={\frac {1}{(\sigma ^{2})^{1+{\frac {\nu _{0}+n}{2}}}}}\exp \left[-{\frac {\nu _{0}\sigma _{0}^{2}+S}{2\sigma ^{2}}}\right]\end{aligned}}}

The above is also a scaled inverse chi-squared distribution whereν0′=ν0+nν0′σ02′=ν0σ02+∑i=1n(xi−μ)2{\displaystyle {\begin{aligned}\nu _{0}'&=\nu _{0}+n\\\nu _{0}'{\sigma _{0}^{2}}'&=\nu _{0}\sigma _{0}^{2}+\sum _{i=1}^{n}(x_{i}-\mu )^{2}\end{aligned}}}or equivalentlyν0′=ν0+nσ02′=ν0σ02+∑i=1n(xi−μ)2ν0+n{\displaystyle {\begin{aligned}\nu _{0}'&=\nu _{0}+n\\{\sigma _{0}^{2}}'&={\frac {\nu _{0}\sigma _{0}^{2}+\sum _{i=1}^{n}(x_{i}-\mu )^{2}}{\nu _{0}+n}}\end{aligned}}}

Reparameterizing in terms of aninverse gamma distribution, the result is:α′=α+n2β′=β+∑i=1n(xi−μ)22{\displaystyle {\begin{aligned}\alpha '&=\alpha +{\frac {n}{2}}\\\beta '&=\beta +{\frac {\sum _{i=1}^{n}(x_{i}-\mu )^{2}}{2}}\end{aligned}}}

With unknown mean and unknown variance

[edit]

For a set ofi.i.d. normally distributed data pointsX of sizen where each individual pointx followsx∼N(μ,σ2){\textstyle x\sim {\mathcal {N}}(\mu ,\sigma ^{2})} with unknown meanμ and unknownvarianceσ2, a combined (multivariate)conjugate prior is placed over the mean and variance, consisting of anormal-inverse-gamma distribution.Logically, this originates as follows:

  1. From the analysis of the case with unknown mean but known variance, we see that the update equations involvesufficient statistics computed from the data consisting of the mean of the data points and the total variance of the data points, computed in turn from the known variance divided by the number of data points.
  2. From the analysis of the case with unknown variance but known mean, we see that the update equations involve sufficient statistics over the data consisting of the number of data points andsum of squared deviations.
  3. Keep in mind that the posterior update values serve as the prior distribution when further data is handled. Thus, we should logically think of our priors in terms of the sufficient statistics just described, with the same semantics kept in mind as much as possible.
  4. To handle the case where both mean and variance are unknown, we could place independent priors over the mean and variance, with fixed estimates of the average mean, total variance, number of data points used to compute the variance prior, and sum of squared deviations. Note however that in reality, the total variance of the mean depends on the unknown variance, and the sum of squared deviations that goes into the variance prior (appears to) depend on the unknown mean. In practice, the latter dependence is relatively unimportant: Shifting the actual mean shifts the generated points by an equal amount, and on average the squared deviations will remain the same. This is not the case, however, with the total variance of the mean: As the unknown variance increases, the total variance of the mean will increase proportionately, and we would like to capture this dependence.
  5. This suggests that we create aconditional prior of the mean on the unknown variance, with a hyperparameter specifying the mean of thepseudo-observations associated with the prior, and another parameter specifying the number of pseudo-observations. This number serves as a scaling parameter on the variance, making it possible to control the overall variance of the mean relative to the actual variance parameter. The prior for the variance also has two hyperparameters, one specifying the sum of squared deviations of the pseudo-observations associated with the prior, and another specifying once again the number of pseudo-observations. Each of the priors has a hyperparameter specifying the number of pseudo-observations, and in each case this controls the relative variance of that prior. These are given as two separate hyperparameters so that the variance (aka the confidence) of the two priors can be controlled separately.
  6. This leads immediately to thenormal-inverse-gamma distribution, which is the product of the two distributions just defined, withconjugate priors used (aninverse gamma distribution over the variance, and a normal distribution over the mean,conditional on the variance) and with the same four parameters just defined.

The priors are normally defined as follows:p(μ∣σ2;μ0,n0)∼N(μ0,σ2/n0)p(σ2;ν0,σ02)∼Iχ2(ν0,σ02)=IG(ν0/2,ν0σ02/2){\displaystyle {\begin{aligned}p(\mu \mid \sigma ^{2};\mu _{0},n_{0})&\sim {\mathcal {N}}(\mu _{0},\sigma ^{2}/n_{0})\\p(\sigma ^{2};\nu _{0},\sigma _{0}^{2})&\sim I\chi ^{2}(\nu _{0},\sigma _{0}^{2})=IG(\nu _{0}/2,\nu _{0}\sigma _{0}^{2}/2)\end{aligned}}}

The update equations can be derived, and look as follows:x¯=1n∑i=1nxiμ0′=n0μ0+nx¯n0+nn0′=n0+nν0′=ν0+nν0′σ02′=ν0σ02+∑i=1n(xi−x¯)2+n0nn0+n(μ0−x¯)2{\displaystyle {\begin{aligned}{\bar {x}}&={\frac {1}{n}}\sum _{i=1}^{n}x_{i}\\\mu _{0}'&={\frac {n_{0}\mu _{0}+n{\bar {x}}}{n_{0}+n}}\\n_{0}'&=n_{0}+n\\\nu _{0}'&=\nu _{0}+n\\\nu _{0}'{\sigma _{0}^{2}}'&=\nu _{0}\sigma _{0}^{2}+\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}+{\frac {n_{0}n}{n_{0}+n}}(\mu _{0}-{\bar {x}})^{2}\end{aligned}}}The respective numbers of pseudo-observations add the number of actual observations to them. The new mean hyperparameter is once again a weighted average, this time weighted by the relative numbers of observations. Finally, the update forν0′σ02′{\textstyle \nu _{0}'{\sigma _{0}^{2}}'} is similar to the case with known mean, but in this case the sum of squared deviations is taken with respect to the observed data mean rather than the true mean, and as a result a new interaction term needs to be added to take care of the additional error source stemming from the deviation between prior and data mean.

Occurrence and applications

[edit]

The occurrence of normal distribution in practical problems can be loosely classified into four categories:

  1. Exactly normal distributions;
  2. Approximately normal laws, for example when such approximation is justified by thecentral limit theorem; and
  3. Distributions modeled as normal – the normal distribution being the distribution withmaximum entropy for a given mean and variance.
  4. Regression problems – the normal distribution being found after systematic effects have been modeled sufficiently well.

Exact normality

[edit]
The ground state of aquantum harmonic oscillator has the Gaussian distribution.

A normal distribution occurs in somephysical theories:

Approximate normality

[edit]

Approximately normal distributions occur in many situations, as explained by thecentral limit theorem. When the outcome is produced by many small effects actingadditively and independently, its distribution will be close to normal. The normal approximation will not be valid if the effects act multiplicatively (instead of additively), or if there is a single external influence that has a considerably larger magnitude than the rest of the effects.

Assumed normality

[edit]
Histogram of sepal widths forIris versicolor from Fisher'sIris flower data set, with superimposed best-fitting normal distribution

I can only recognize the occurrence of the normal curve – the Laplacian curve of errors – as a very abnormal phenomenon. It is roughly approximated to in certain distributions; for this reason, and on account for its beautiful simplicity, we may, perhaps, use it as a first approximation, particularly in theoretical investigations.

— Pearson (1901)

There are statistical methods to empirically test that assumption; see the aboveNormality tests section.

  • Inbiology, thelogarithm of various variables tend to have a normal distribution, that is, they tend to have alog-normal distribution (after separation on male/female subpopulations), with examples including:
    • Measures of size of living tissue (length, height, skin area, weight);[60]
    • Thelength ofinert appendages (hair, claws, nails, teeth) of biological specimens,in the direction of growth; presumably the thickness of tree bark also falls under this category;
    • Certain physiological measurements, such as blood pressure of adult humans.
  • In finance, in particular theBlack–Scholes model, changes in thelogarithm of exchange rates, price indices, and stock market indices are assumed normal (these variables behave likecompound interest, not like simple interest, and so are multiplicative). Some mathematicians such asBenoit Mandelbrot have argued thatlog-Levy distributions, which possessheavy tails, would be a more appropriate model, in particular for the analysis forstock market crashes. The use of the assumption of normal distribution occurring in financial models has also been criticized byNassim Nicholas Taleb in his works.
  • Measurement errors in physical experiments are often modeled by a normal distribution. This use of a normal distribution does not imply that one is assuming the measurement errors are normally distributed, rather using the normal distribution produces the most conservative predictions possible given only knowledge about the mean and variance of the errors.[61]
  • Instandardized testing, results can be made to have a normal distribution by either selecting the number and difficulty of questions (as in theIQ test) or transforming the raw test scores into output scores by fitting them to the normal distribution. For example, theSAT's traditional range of 200–800 is based on a normal distribution with a mean of 500 and a standard deviation of 100.
Fitted cumulative normal distribution to October rainfalls, seedistribution fitting

Methodological problems and peer review

[edit]

John Ioannidisargued that using normally distributed standard deviations as standards for validating research findings leavefalsifiable predictions about phenomena that are not normally distributed untested. This includes, for example, phenomena that only appear when all necessary conditions are present and one cannot be a substitute for another in an addition-like way and phenomena that are not randomly distributed. Ioannidis argues that standard deviation-centered validation gives a false appearance of validity to hypotheses and theories where some but not all falsifiable predictions are normally distributed since the portion of falsifiable predictions that there is evidence against may and in some cases are in the non-normally distributed parts of the range of falsifiable predictions, as well as baselessly dismissing hypotheses for which none of the falsifiable predictions are normally distributed as if they were unfalsifiable when in fact they do make falsifiable predictions. It is argued by Ioannidis that many cases of mutually exclusive theories being accepted as validated by research journals are caused by failure of the journals to take in empirical falsifications of non-normally distributed predictions, and not because mutually exclusive theories are true, which they cannot be, although two mutually exclusive theories can both be wrong and a third one correct.[63]

Computational methods

[edit]

Generating values from normal distribution

[edit]
Thebean machine, a device invented byFrancis Galton, can be called the first generator of normal random variables. This machine consists of a vertical board with interleaved rows of pins. Small balls are dropped from the top and then bounce randomly left or right as they hit the pins. The balls are collected into bins at the bottom and settle down into a pattern resembling the Gaussian curve.

In computer simulations, especially in applications of theMonte-Carlo method, it is often desirable to generate values that are normally distributed. The algorithms listed below all generate the standard normal deviates, since aN(μ,σ2) can be generated asX =μ +σZ, whereZ is standard normal. All these algorithms rely on the availability of arandom number generatorU capable of producinguniform random variates.

  • The most straightforward method is based on theprobability integral transform property: ifU is distributed uniformly on (0,1), thenΦ−1(U) will have the standard normal distribution. The drawback of this method is that it relies on calculation of theprobit function Φ−1, which cannot be done analytically. Some approximate methods are described inHart (1968) and in theerf article. Wichura gives a fast algorithm for computing this function to 16 decimal places,[64] which is used byR to compute random variates of the normal distribution.
  • An easy-to-program approximate approach that relies on thecentral limit theorem is as follows: generate 12 uniformU(0,1) deviates, add them all up, and subtract 6 – the resulting random variable will have approximately standard normal distribution. In truth, the distribution will beIrwin–Hall, which is a 12-section eleventh-order polynomial approximation to the normal distribution. This random deviate will have a limited range of(−6, 6).[65] Note that in a true normal distribution, only 0.00034% of all samples will fall outside ±6σ.
  • TheBox–Muller method uses two independent random numbersU andV distributeduniformly on (0,1). Then the two random variablesX andYX=−2ln⁡Ucos⁡(2πV),Y=−2ln⁡Usin⁡(2πV).{\displaystyle X={\sqrt {-2\ln U}}\,\cos(2\pi V),\qquad Y={\sqrt {-2\ln U}}\,\sin(2\pi V).} will both have the standard normal distribution, and will beindependent. This formulation arises because for abivariate normal random vector(X,Y) the squared normX2 +Y2 will have thechi-squared distribution with two degrees of freedom, which is an easily generatedexponential random variable corresponding to the quantity−2 ln(U) in these equations; and the angle is distributed uniformly around the circle, chosen by the random variableV.
  • TheMarsaglia polar method is a modification of the Box–Muller method which does not require computation of the sine and cosine functions. In this method,U andV are drawn from the uniform (−1,1) distribution, and thenS =U2 +V2 is computed. IfS is greater or equal to 1, then the method starts over, otherwise the two quantitiesX=U−2ln⁡SS,Y=V−2ln⁡SS{\displaystyle X=U{\sqrt {\frac {-2\ln S}{S}}},\qquad Y=V{\sqrt {\frac {-2\ln S}{S}}}} are returned. Again,X andY are independent, standard normal random variables.
  • The Ratio method[66] is a rejection method. The algorithm proceeds as follows:
    • Generate two independent uniform deviatesU andV;
    • ComputeX =√8/e (V − 0.5)/U;
    • Optional: ifX2 ≤ 5 − 4e1/4U then acceptX and terminate algorithm;
    • Optional: ifX2 ≥ 4e−1.35/U + 1.4 then rejectX and start over from step 1;
    • IfX2 ≤ −4 lnU then acceptX, otherwise start over the algorithm.
    The two optional steps allow the evaluation of the logarithm in the last step to be avoided in most cases. These steps can be greatly improved[67] so that the logarithm is rarely evaluated.
  • Theziggurat algorithm[68] is faster than the Box–Muller transform and still exact. In about 97% of all cases it uses only two random numbers, one random integer and one random uniform, one multiplication and an if-test. Only in 3% of the cases, where the combination of those two falls outside the "core of the ziggurat" (a kind of rejection sampling using logarithms), do exponentials and more uniform random numbers have to be employed.
  • Integer arithmetic can be used to sample from the standard normal distribution.[69][70] This method is exact in the sense that it satisfies the conditions ofideal approximation;[71] i.e., it is equivalent to sampling a real number from the standard normal distribution and rounding this to the nearest representable floating point number.
  • There is also some investigation[72] into the connection between the fastHadamard transform and the normal distribution, since the transform employs just addition and subtraction and by the central limit theorem random numbers from almost any distribution will be transformed into the normal distribution. In this regard a series of Hadamard transforms can be combined with random permutations to turn arbitrary data sets into a normally distributed data.

Numerical approximations for the normal cumulative distribution function and normal quantile function

[edit]

The standard normalcumulative distribution function is widely used in scientific and statistical computing.

The valuesΦ(x) may be approximated very accurately by a variety of methods, such asnumerical integration,Taylor series,asymptotic series andcontinued fractions. Different approximations are used depending on the desired level of accuracy.

1−Φ(x)=1−(1−Φ(−x)){\displaystyle 1-\Phi \left(x\right)=1-\left(1-\Phi \left(-x\right)\right)}

Shore (1982) introduced simple approximations that may be incorporated in stochastic optimization models of engineering and operations research, like reliability engineering and inventory analysis. Denotingp =Φ(z), the simplest approximation for the quantile function is:z=Φ−1(p)=5.5556[1−(1−pp)0.1186],p≥1/2{\displaystyle z=\Phi ^{-1}(p)=5.5556\left[1-\left({\frac {1-p}{p}}\right)^{0.1186}\right],\qquad p\geq 1/2}

This approximation delivers forz a maximum absolute error of 0.026 (for0.5 ≤p ≤ 0.9999, corresponding to0 ≤z ≤ 3.719). Forp < 1/2 replacep by1 −p and change sign. Another approximation, somewhat less accurate, is the single-parameter approximation:z=−0.4115{1−pp+log⁡[1−pp]−1},p≥1/2{\displaystyle z=-0.4115\left\{{\frac {1-p}{p}}+\log \left[{\frac {1-p}{p}}\right]-1\right\},\qquad p\geq 1/2}

The latter had served to derive a simple approximation for the loss integral of the normal distribution, defined byL(z)=∫z∞(u−z)φ(u)du=∫z∞[1−Φ(u)]duL(z)≈{0.4115(p1−p)−z,p<1/2,0.4115(1−pp),p≥1/2.or, equivalently,L(z)≈{0.4115{1−log⁡[p1−p]},p<1/2,0.41151−pp,p≥1/2.{\displaystyle {\begin{aligned}L(z)&=\int _{z}^{\infty }(u-z)\varphi (u)\,du=\int _{z}^{\infty }[1-\Phi (u)]\,du\\[5pt]L(z)&\approx {\begin{cases}0.4115\left({\dfrac {p}{1-p}}\right)-z,&p<1/2,\\\\0.4115\left({\dfrac {1-p}{p}}\right),&p\geq 1/2.\end{cases}}\\[5pt]{\text{or, equivalently,}}\\L(z)&\approx {\begin{cases}0.4115\left\{1-\log \left[{\frac {p}{1-p}}\right]\right\},&p<1/2,\\\\0.4115{\dfrac {1-p}{p}},&p\geq 1/2.\end{cases}}\end{aligned}}}

This approximation is particularly accurate for the right far-tail (maximum error of 10−3 forz ≥ 1.4). Highly accurate approximations for the cumulative distribution function, based onResponse Modeling Methodology (RMM, Shore, 2011, 2012), are shown in Shore (2005).

Some more approximations can be found at:Error function#Approximation with elementary functions. In particular, smallrelative error on the whole domain for the cumulative distribution function⁠Φ{\displaystyle \Phi }⁠ and the quantile functionΦ−1{\textstyle \Phi ^{-1}} as well, is achieved via an explicitly invertible formula by Sergei Winitzki in 2008.

History

[edit]

Development

[edit]

Some authors[73][74] attribute the discovery of the normal distribution tode Moivre, who in 1738[note 2] published in the second edition of hisThe Doctrine of Chances the study of the coefficients in thebinomial expansion of(a +b)n. De Moivre proved that the middle term in this expansion has the approximate magnitude of2n/2πn{\textstyle 2^{n}/{\sqrt {2\pi n}}}, and that "Ifm or⁠1/2⁠n be a Quantity infinitely great, then the Logarithm of the Ratio, which a Term distant from the middle by the Intervalℓ, has to the middle Term, is−2ℓℓn{\textstyle -{\frac {2\ell \ell }{n}}}."[75] Although this theorem can be interpreted as the first obscure expression for the normal probability law,Stigler points out that de Moivre himself did not interpret his results as anything more than the approximate rule for the binomial coefficients, and in particular de Moivre lacked the concept of the probability density function.[76]

In 1809,Carl Friedrich Gauss showed that the normal distribution provides a way to rationalize themethod of least squares.

In 1823Gauss published his monograph"Theoria combinationis observationum erroribus minimis obnoxiae" where among other things he introduces several important statistical concepts, such as themethod of least squares, themethod of maximum likelihood, and thenormal distribution. Gauss usedM,M′,M″, ... to denote the measurements of some unknown quantity V, and sought the most probable estimator of that quantity: the one that maximizes the probabilityφ(M −V) ·φ(M′ −V) ·φ(M″ −V) · ... of obtaining the observed experimental results. In his notation φΔ is the probability density function of the measurement errors of magnitude Δ. Not knowing what the functionφ is, Gauss requires that his method should reduce to the well-known answer: the arithmetic mean of the measured values.[note 3] Starting from these principles, Gauss demonstrates that the only law that rationalizes the choice of arithmetic mean as an estimator of the location parameter, is the normal law of errors:[77]φΔ=h√πe−hhΔΔ,{\displaystyle \varphi {\mathit {\Delta }}={\frac {h}{\surd \pi }}\,e^{-\mathrm {hh} \Delta \Delta },} whereh is "the measure of the precision of the observations". Using this normal law as a generic model for errors in the experiments, Gauss formulates what is now known as thenon-linearweighted least squares method.[78]

Pierre-Simon Laplace proved thecentral limit theorem in 1810, consolidating the importance of the normal distribution in statistics.

Although Gauss was the first to suggest the normal distribution law,Laplace made significant contributions.[note 4] It was Laplace who first posed the problem of aggregating several observations in 1774,[79] although his own solution led to theLaplacian distribution. It was Laplace who first calculated the value of theintegral∫e−t2dt =√π in 1782, providing the normalization constant for the normal distribution.[80] For this accomplishment, Gauss acknowledged the priority of Laplace.[81] Finally, it was Laplace who in 1810 proved and presented to the academy the fundamentalcentral limit theorem, which emphasized the theoretical importance of the normal distribution.[82]

It is of interest to note that in 1809 an Irish-American mathematicianRobert Adrain published two insightful but flawed derivations of the normal probability law, simultaneously and independently from Gauss.[83] His works remained largely unnoticed by the scientific community, until in 1871 they were exhumed byAbbe.[84]

In the middle of the 19th centuryMaxwell demonstrated that the normal distribution is not just a convenient mathematical tool, but may also occur in natural phenomena:[57] The number of particles whose velocity, resolved in a certain direction, lies betweenx andx +dx isN⁡1απe−x2α2dx{\displaystyle \operatorname {N} {\frac {1}{\alpha \;{\sqrt {\pi }}}}\;e^{-{\frac {x^{2}}{\alpha ^{2}}}}\,dx}

Naming

[edit]

Today, the concept is usually known in English as thenormal distribution orGaussian distribution. Other less common names include Gauss distribution, Laplace–Gauss distribution, the law of error, the law of facility of errors, Laplace's second law, and Gaussian law.

Gauss himself apparently coined the term with reference to the "normal equations" involved in its applications, with normal having its technical meaning of orthogonal rather than usual.[85] However, by the end of the 19th century some authors[note 5] had started using the namenormal distribution, where the word "normal" was used as an adjective – the term now being seen as a reflection of this distribution being seen as typical, common – and thus normal.Peirce (one of those authors) once defined "normal" thus: "... the 'normal' is not the average (or any other kind of mean) of what actually occurs, but of whatwould, in the long run, occur under certain circumstances."[86] Around the turn of the 20th centuryPearson popularized the termnormal as a designation for this distribution.[87]

Many years ago I called the Laplace–Gaussian curve thenormal curve, which name, while it avoids an international question of priority, has the disadvantage of leading people to believe that all other distributions of frequency are in one sense or another 'abnormal'.

— Pearson (1920)

Also, it was Pearson who first wrote the distribution in terms of the standard deviationσ as in modern notation. Soon after this, in year 1915,Fisher added the location parameter to the formula for normal distribution, expressing it in the way it is written nowadays:df=12σ2πe−(x−m)2/(2σ2)dx.{\displaystyle df={\frac {1}{\sqrt {2\sigma ^{2}\pi }}}e^{-(x-m)^{2}/(2\sigma ^{2})}\,dx.}

The termstandard normal distribution, which denotes the normal distribution with zero mean and unit variance came into general use around the 1950s, appearing in the popular textbooks by P. G. Hoel (1947)Introduction to Mathematical Statistics andAlexander M. Mood (1950)Introduction to the Theory of Statistics.[88][89][90]

See also

[edit]

Notes

[edit]
  1. ^For example, this algorithm is given in the articleBc programming language.
  2. ^De Moivre first published his findings in 1733, in a pamphletApproximatio ad Summam Terminorum Binomii(a +b)nin Seriem Expansi that was designated for private circulation only. But it was not until the year 1738 that he made his results publicly available. The original pamphlet was reprinted several times, see for exampleWalker (1985).
  3. ^"It has been customary certainly to regard as an axiom the hypothesis that if any quantity has been determined by several direct observations, made under the same circumstances and with equal care, the arithmetical mean of the observed values affords the most probable value, if not rigorously, yet very nearly at least, so that it is always most safe to adhere to it." —Gauss (1809, section 177)
  4. ^"My custom of terming the curve the Gauss–Laplacian ornormal curve saves us from proportioning the merit of discovery between the two great astronomer mathematicians." quote fromPearson (1905, p. 189)
  5. ^Besides those specifically referenced here, such use is encountered in the works ofPeirce,Galton (Galton (1889, chapter V)) andLexis (Lexis (1878),Rohrbasser & Véron (2003)) c. 1875.[citation needed]

References

[edit]

Citations

[edit]
  1. ^Norton, Matthew; Khokhlov, Valentyn; Uryasev, Stan (2019)."Calculating CVaR and bPOE for common probability distributions with application to portfolio optimization and density estimation"(PDF).Annals of Operations Research.299 (1–2). Springer:1281–1315.arXiv:1811.11301.doi:10.1007/s10479-019-03373-1.S2CID 254231768. Archived fromthe original(PDF) on March 31, 2023. RetrievedFebruary 27, 2023.
  2. ^Tsokos, Chris; Wooten, Rebecca (January 1, 2016). Tsokos, Chris; Wooten, Rebecca (eds.).The Joy of Finite Mathematics. Boston: Academic Press. pp. 231–263.doi:10.1016/b978-0-12-802967-1.00007-3.ISBN 978-0-12-802967-1.
  3. ^Harris, Frank E. (January 1, 2014). Harris, Frank E. (ed.).Mathematics for Physical Science and Engineering. Boston: Academic Press. pp. 663–709.doi:10.1016/b978-0-12-801000-6.00018-3.ISBN 978-0-12-801000-6.
  4. ^Hoel (1947,p. 31) andMood (1950,p. 109) give this definition with slightly different notation.
  5. ^Normal Distribution, Gale Encyclopedia of Psychology
  6. ^Casella & Berger (2001, p. 102)
  7. ^Lyon, A. (2014).Why are Normal Distributions Normal?, The British Journal for the Philosophy of Science.
  8. ^Jorge, Nocedal; Stephan, J. Wright (2006).Numerical Optimization (2nd ed.). Springer. p. 249.ISBN 978-0387-30303-1.
  9. ^ab"Normal Distribution".www.mathsisfun.com. RetrievedAugust 15, 2020.
  10. ^"bell curve".Merriam-Webster.com Dictionary. RetrievedMay 25, 2025.
  11. ^Mood (1950,p. 112) explicitly defines thestandard normal distribution. In contrast,Hoel (1947) explicitly defines thestandard normal curve(p. 33) and introduces the termstandard normal distribution(p. 69).
  12. ^Stigler (1982)
  13. ^Halperin, Hartley & Hoel (1965, item 7)
  14. ^McPherson (1990, p. 110)
  15. ^Bernardo & Smith (2000, p. 121)
  16. ^Park, Kun Il (2018).Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer.ISBN 978-3-319-68074-3.
  17. ^Scott, Clayton; Nowak, Robert (August 7, 2003)."The Q-function".Connexions.
  18. ^Barak, Ohad (April 6, 2006)."Q Function and Error Function"(PDF). Tel Aviv University. Archived fromthe original(PDF) on March 25, 2009.
  19. ^Weisstein, Eric W."Normal Distribution Function".MathWorld.
  20. ^Abramowitz, Milton;Stegun, Irene Ann, eds. (1983) [June 1964]."Chapter 26, eqn 26.2.12".Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Applied Mathematics Series. Vol. 55 (Ninth reprint with additional corrections of tenth original printing with corrections (December 1972); first ed.). Washington D.C.; New York: United States Department of Commerce, National Bureau of Standards; Dover Publications. p. 932.ISBN 978-0-486-61272-0.LCCN 64-60036.MR 0167642.LCCN 65-12253.
  21. ^Vaart, A. W. van der (October 13, 1998).Asymptotic Statistics. Cambridge University Press.doi:10.1017/cbo9780511802256.ISBN 978-0-511-80225-6.
  22. ^abCover & Thomas (2006), p. 254.
  23. ^Park, Sung Y.; Bera, Anil K. (2009)."Maximum Entropy Autoregressive Conditional Heteroskedasticity Model"(PDF).Journal of Econometrics.150 (2):219–230.Bibcode:2009JEcon.150..219P.CiteSeerX 10.1.1.511.9750.doi:10.1016/j.jeconom.2008.12.014. Archived fromthe original(PDF) on March 7, 2016. RetrievedJune 2, 2011.
  24. ^Geary RC(1936) The distribution of the "Student's ratio for the non-normal samples". Supplement to the Journal of the Royal Statistical Society 3 (2): 178–184
  25. ^Lukacs, Eugene (March 1942)."A Characterization of the Normal Distribution".Annals of Mathematical Statistics.13 (1):91–93.doi:10.1214/AOMS/1177731647.ISSN 0003-4851.JSTOR 2236166.MR 0006626.Zbl 0060.28509.Wikidata Q55897617.
  26. ^abcPatel & Read (1996, [2.1.4])
  27. ^Fan (1991, p. 1258)
  28. ^Patel & Read (1996, [2.1.8])
  29. ^Papoulis, Athanasios.Probability, Random Variables and Stochastic Processes (4th ed.). p. 148.
  30. ^Winkelbauer, Andreas (2012). "Moments and Absolute Moments of the Normal Distribution".arXiv:1209.4340 [math.ST].
  31. ^Bryc (1995, p. 23)
  32. ^Bryc (1995, p. 24)
  33. ^Williams, David (2001).Weighing the odds : a course in probability and statistics (Reprinted. ed.). Cambridge [u.a.]: Cambridge Univ. Press. pp. 197–199.ISBN 978-0-521-00618-7.
  34. ^José M. Bernardo; Adrian F. M. Smith (2000).Bayesian theory (Reprint ed.). Chichester [u.a.]: Wiley. pp. 209, 366.ISBN 978-0-471-49464-5.
  35. ^O'Hagan, A. (1994)Kendall's Advanced Theory of statistics, Vol 2B, Bayesian Inference, Edward Arnold.ISBN 0-340-52922-9 (Section 5.40)
  36. ^abBryc (1995, p. 35)
  37. ^UIUC, Lecture 21.The Multivariate Normal Distribution, 21.6:"Individually Gaussian Versus Jointly Gaussian".
  38. ^Edward L. Melnick and Aaron Tenenbein, "Misspecifications of the Normal Distribution",The American Statistician, volume 36, number 4 November 1982, pages 372–373
  39. ^"Kullback Leibler (KL) Distance of Two Normal (Gaussian) Probability Distributions".Allisons.org. December 5, 2007. RetrievedMarch 3, 2017.
  40. ^Jordan, Michael I. (February 8, 2010)."Stat260: Bayesian Modeling and Inference: The Conjugate Prior for the Normal Distribution"(PDF).
  41. ^Amari & Nagaoka (2000)
  42. ^"Expectation of the maximum of gaussian random variables".Mathematics Stack Exchange. RetrievedApril 7, 2024.
  43. ^"Normal Approximation to Poisson Distribution".Stat.ucla.edu. RetrievedMarch 3, 2017.
  44. ^Bryc (1995, p. 27)
  45. ^Weisstein, Eric W."Normal Product Distribution".MathWorld. wolfram.com.
  46. ^Lukacs, Eugene (1942)."A Characterization of the Normal Distribution".The Annals of Mathematical Statistics.13 (1):91–3.doi:10.1214/aoms/1177731647.ISSN 0003-4851.JSTOR 2236166.
  47. ^Basu, D.; Laha, R. G. (1954). "On Some Characterizations of the Normal Distribution".Sankhyā.13 (4):359–62.ISSN 0036-4452.JSTOR 25048183.
  48. ^Lehmann, E. L. (1997).Testing Statistical Hypotheses (2nd ed.). Springer. p. 199.ISBN 978-0-387-94919-2.
  49. ^Patel & Read (1996, [2.3.6])
  50. ^Galambos & Simonelli (2004, Theorem 3.5)
  51. ^abLukacs & King (1954)
  52. ^Quine, M.P. (1993)."On three characterisations of the normal distribution".Probability and Mathematical Statistics.14 (2):257–263.
  53. ^John, S (1982). "The three parameter two-piece normal family of distributions and its fitting".Communications in Statistics – Theory and Methods.11 (8):879–885.doi:10.1080/03610928208828279.
  54. ^abKrishnamoorthy (2006, p. 127)
  55. ^Krishnamoorthy (2006, p. 130)
  56. ^Krishnamoorthy (2006, p. 133)
  57. ^abMaxwell (1860), p. 23.
  58. ^Bryc (1995), p. 1.
  59. ^Larkoski, Andrew J. (2023).Quantum Mechanics: A Mathematical Introduction. United Kingdom: Cambridge University Press. pp. 120–121.ISBN 978-1-009-12222-1. RetrievedMay 30, 2025.
  60. ^Huxley (1932)
  61. ^Jaynes, Edwin T. (2003).Probability Theory: The Logic of Science. Cambridge University Press. pp. 592–593.ISBN 9780521592710.
  62. ^Oosterbaan, Roland J. (1994)."Chapter 6: Frequency and Regression Analysis of Hydrologic Data"(PDF). In Ritzema, Henk P. (ed.).Drainage Principles and Applications, Publication 16 (second revised ed.). Wageningen, The Netherlands: International Institute for Land Reclamation and Improvement (ILRI). pp. 175–224.ISBN 978-90-70754-33-4.
  63. ^Why Most Published Research Findings Are False, John P. A. Ioannidis, 2005
  64. ^Wichura, Michael J. (1988). "Algorithm AS241: The Percentage Points of the Normal Distribution".Applied Statistics.37 (3):477–84.doi:10.2307/2347330.JSTOR 2347330.
  65. ^Johnson, Kotz & Balakrishnan (1995, Equation (26.48))
  66. ^Kinderman & Monahan (1977)
  67. ^Leva (1992)
  68. ^Marsaglia & Tsang (2000)
  69. ^Karney (2016)
  70. ^Du, Fan & Wei (2022)
  71. ^Monahan (1985, section 2)
  72. ^Wallace (1996)
  73. ^Johnson, Kotz & Balakrishnan (1994, p. 85)
  74. ^Le Cam & Lo Yang (2000, p. 74)
  75. ^De Moivre, Abraham (1733), Corollary I – seeWalker (1985, p. 77)
  76. ^Stigler (1986,p. 76)
  77. ^Gauss (1809, section 177)
  78. ^Gauss (1809, section 179)
  79. ^Laplace (1774, Problem III)
  80. ^Pearson (1905, p. 189)
  81. ^Gauss (1809, section 177)
  82. ^Stigler (1986, p. 144)
  83. ^Stigler (1978, p. 243)
  84. ^Stigler (1978, p. 244)
  85. ^Jaynes, Edwin J.;Probability Theory: The Logic of Science,Ch. 7.
  86. ^Peirce, Charles S. (c. 1909 MS),Collected Papers v. 6, paragraph 327.
  87. ^Kruskal & Stigler (1997).
  88. ^"Earliest Uses... (Entry Standard Normal Curve)".
  89. ^Hoel (1947) introduces the termsstandard normal curve(p. 33) andstandard normal distribution(p. 69).
  90. ^Mood (1950) explicitly defines thestandard normal distribution(p. 112).
  91. ^Sun, Jingchao; Kong, Maiying; Pal, Subhadip (June 22, 2021)."The Modified-Half-Normal distribution: Properties and an efficient sampling scheme".Communications in Statistics – Theory and Methods.52 (5):1591–1613.doi:10.1080/03610926.2021.1934700.ISSN 0361-0926.S2CID 237919587.

Sources

[edit]


External links

[edit]
Wikimedia Commons has media related toNormal distribution.
Discrete
univariate
with finite
support
with infinite
support
Continuous
univariate
supported on a
bounded interval
supported on a
semi-infinite
interval
supported
on the whole
real line
with support
whose type varies
Mixed
univariate
continuous-
discrete
Multivariate
(joint)
Directional
Degenerate
andsingular
Degenerate
Dirac delta function
Singular
Cantor
Families
International
National
Other
Retrieved from "https://en.wikipedia.org/w/index.php?title=Normal_distribution&oldid=1337109975"
Categories:
Hidden categories:
[8]ページ先頭
©2009-2026 Movatter.jp