Movatterモバイル変換

[0]ホーム
URL:

Jump to content
WikipediaThe Free Encyclopedia
Search

Variance

From Wikipedia, the free encyclopedia
Statistical measure of how far values spread from their average
This article is about the mathematical concept. For other uses, seeVariance (disambiguation).
Example of samples from two populations with the same mean but different variances. The red population has mean 100 and variance 100 (SD=10) while the blue population has mean 100 and variance 2500 (SD=50) where SD stands for Standard Deviation.

Inprobability theory andstatistics,variance is theexpected value of thesquared deviation from the mean of arandom variable. Thestandard deviation (SD) is obtained as the square root of the variance. Variance is a measure ofdispersion, meaning it is a measure of how far a set of numbers are spread out from their average value. It is the secondcentral moment of adistribution, and thecovariance of the random variable with itself, and it is often represented byσ2{\displaystyle \sigma ^{2}},s2{\displaystyle s^{2}},Var⁡(X){\displaystyle \operatorname {Var} (X)},V(X){\displaystyle V(X)}, orV(X){\displaystyle \mathbb {V} (X)}.[1]

An advantage of variance as a measure of dispersion is that it is more amenable to algebraic manipulation than other measures of dispersion such as theexpected absolute deviation; for example, the variance of a sum of uncorrelated random variables is equal to the sum of their variances. A disadvantage of the variance for practical applications is that, unlike the standard deviation, its units differ from the random variable, which is why the standard deviation is more commonly reported as a measure of dispersion once the calculation is finished. Another disadvantage is that the variance is not finite for many distributions.

There are two distinct concepts that are both called "variance". One, as discussed above, is part of a theoreticalprobability distribution and is defined by an equation. The other variance is a characteristic of a set of observations. When variance is calculated from observations, those observations are typically measured from a real-world system. If all possible observations of the system are present, then the calculated variance is called the population variance. Normally, however, only a subset is available, and the variance calculated from this is called the sample variance. The variance calculated from a sample is considered an estimate of the full population variance. There are multiple ways to estimate the population variance on the basis of the sample variance, as discussed in the section below.

The two kinds of variance are closely related. To see how, consider that a theoretical probability distribution can be used as a generator of hypothetical observations. If an infinite number of observations are generated using a distribution, then the sample variance calculated from that infinite set will match the value calculated using the distribution's equation for variance. Variance has a central role in statistics, where some ideas that use it includedescriptive statistics,statistical inference,hypothesis testing,goodness of fit, andMonte Carlo sampling.

Geometric visualisation of the variance of an arbitrary distribution (2, 4, 4, 4, 5, 5, 7, 9):
  1. A frequency distribution is constructed.
  2. The centroid of the distribution gives its mean.
  3. A square with sides equal to the difference of each value from the mean is formed for each value.
  4. Arranging the squares into a rectangle with one side equal to the number of values,n, results in the other side being the distribution's variance,σ2.

Definition

[edit]

The variance of a random variableX{\displaystyle X} is theexpected value of thesquared deviation from the mean ofX{\displaystyle X},μ=E⁡[X]{\displaystyle \mu =\operatorname {E} [X]}:Var⁡(X)=E⁡[(X−μ)2].{\displaystyle \operatorname {Var} (X)=\operatorname {E} \left[(X-\mu )^{2}\right].}This definition encompasses random variables that are generated by processes that arediscrete,continuous,neither, or mixed. The variance can also be thought of as thecovariance of a random variable with itself:

Var⁡(X)=Cov⁡(X,X).{\displaystyle \operatorname {Var} (X)=\operatorname {Cov} (X,X).} The variance is also equivalent to the secondcumulant of a probability distribution that generatesX{\displaystyle X}. The variance is typically designated asVar⁡(X){\displaystyle \operatorname {Var} (X)}, or sometimes asV(X){\displaystyle V(X)} orV(X){\displaystyle \mathbb {V} (X)}, or symbolically asσX2{\displaystyle \sigma _{X}^{2}} or simplyσ2{\displaystyle \sigma ^{2}} (pronounced "sigma squared"). The expression for the variance can be expanded as follows:Var⁡(X)=E⁡[(X−E⁡[X])2]=E⁡[X2−2XE⁡[X]+E⁡[X]2]=E⁡[X2]−2E⁡[X]E⁡[X]+E⁡[X]2=E⁡[X2]−2E⁡[X]2+E⁡[X]2=E⁡[X2]−E⁡[X]2{\displaystyle {\begin{aligned}\operatorname {Var} (X)&=\operatorname {E} \left[{\left(X-\operatorname {E} [X]\right)}^{2}\right]\\[4pt]&=\operatorname {E} \left[X^{2}-2X\operatorname {E} [X]+\operatorname {E} [X]^{2}\right]\\[4pt]&=\operatorname {E} \left[X^{2}\right]-2\operatorname {E} [X]\operatorname {E} [X]+\operatorname {E} [X]^{2}\\[4pt]&=\operatorname {E} \left[X^{2}\right]-2\operatorname {E} [X]^{2}+\operatorname {E} [X]^{2}\\[4pt]&=\operatorname {E} \left[X^{2}\right]-\operatorname {E} [X]^{2}\end{aligned}}}

In other words, the variance ofX is equal to the mean of the square ofX minus the square of the mean ofX. This equation should not be used for computations usingfloating-point arithmetic, because it suffers fromcatastrophic cancellation if the two components of the equation are similar in magnitude. For other numerically stable alternatives, seealgorithms for calculating variance.

Discrete random variable

[edit]

If the generator of random variableX{\displaystyle X} isdiscrete withprobability mass functionx1↦p1,x2↦p2,…,xn↦pn{\displaystyle x_{1}\mapsto p_{1},x_{2}\mapsto p_{2},\ldots ,x_{n}\mapsto p_{n}}, then

Var⁡(X)=∑i=1npi⋅(xi−μ)2,{\displaystyle \operatorname {Var} (X)=\sum _{i=1}^{n}p_{i}\cdot {\left(x_{i}-\mu \right)}^{2},}

whereμ{\displaystyle \mu } is the expected value. That is,

μ=∑i=1npixi.{\displaystyle \mu =\sum _{i=1}^{n}p_{i}x_{i}.}

(When such a discreteweighted variance is specified by weights whose sum is not 1, then one divides by the sum of the weights.)

The variance of a collection ofn{\displaystyle n} equally likely values can be written as

Var⁡(X)=1n∑i=1n(xi−μ)2{\displaystyle \operatorname {Var} (X)={\frac {1}{n}}\sum _{i=1}^{n}(x_{i}-\mu )^{2}}

whereμ{\displaystyle \mu } is the average value. That is,

μ=1n∑i=1nxi.{\displaystyle \mu ={\frac {1}{n}}\sum _{i=1}^{n}x_{i}.}

The variance of a set ofn{\displaystyle n} equally likely values can be equivalently expressed, without directly referring to the mean, in terms of squared deviations of all pairwise squared distances of points from each other:[2]

Var⁡(X)=1n2∑i=1n∑j=1n12(xi−xj)2=1n2∑i∑j>i(xi−xj)2.{\displaystyle \operatorname {Var} (X)={\frac {1}{n^{2}}}\sum _{i=1}^{n}\sum _{j=1}^{n}{\frac {1}{2}}{\left(x_{i}-x_{j}\right)}^{2}={\frac {1}{n^{2}}}\sum _{i}\sum _{j>i}{\left(x_{i}-x_{j}\right)}^{2}.}

Absolutely continuous random variable

[edit]

If the random variableX{\displaystyle X} has aprobability density functionf(x){\displaystyle f(x)}, andF(x){\displaystyle F(x)} is the correspondingcumulative distribution function, then

Var⁡(X)=σ2=∫R(x−μ)2f(x)dx=∫Rx2f(x)dx−2μ∫Rxf(x)dx+μ2∫Rf(x)dx=∫Rx2dF(x)−2μ∫RxdF(x)+μ2∫RdF(x)=∫Rx2dF(x)−2μ⋅μ+μ2⋅1=∫Rx2dF(x)−μ2,{\displaystyle {\begin{aligned}\operatorname {Var} (X)=\sigma ^{2}&=\int _{\mathbb {R} }{\left(x-\mu \right)}^{2}f(x)\,dx\\[4pt]&=\int _{\mathbb {R} }x^{2}f(x)\,dx-2\mu \int _{\mathbb {R} }xf(x)\,dx+\mu ^{2}\int _{\mathbb {R} }f(x)\,dx\\[4pt]&=\int _{\mathbb {R} }x^{2}\,dF(x)-2\mu \int _{\mathbb {R} }x\,dF(x)+\mu ^{2}\int _{\mathbb {R} }\,dF(x)\\[4pt]&=\int _{\mathbb {R} }x^{2}\,dF(x)-2\mu \cdot \mu +\mu ^{2}\cdot 1\\[4pt]&=\int _{\mathbb {R} }x^{2}\,dF(x)-\mu ^{2},\end{aligned}}}

or equivalently,

Var⁡(X)=∫Rx2f(x)dx−μ2,{\displaystyle \operatorname {Var} (X)=\int _{\mathbb {R} }x^{2}f(x)\,dx-\mu ^{2},}

whereμ{\displaystyle \mu } is the expected value ofX{\displaystyle X} given by

μ=∫Rxf(x)dx=∫RxdF(x).{\displaystyle \mu =\int _{\mathbb {R} }xf(x)\,dx=\int _{\mathbb {R} }x\,dF(x).}

In these formulas, the integrals with respect todx{\displaystyle dx} anddF(x){\displaystyle dF(x)}areLebesgue andLebesgue–Stieltjes integrals, respectively.

If the functionx2f(x){\displaystyle x^{2}f(x)} isRiemann-integrable on every finite interval[a,b]⊂R,{\displaystyle [a,b]\subset \mathbb {R} ,} then

Var⁡(X)=∫−∞+∞x2f(x)dx−μ2,{\displaystyle \operatorname {Var} (X)=\int _{-\infty }^{+\infty }x^{2}f(x)\,dx-\mu ^{2},}

where the integral is animproper Riemann integral.

Examples

[edit]

Exponential distribution

[edit]

Theexponential distribution with parameterλ > 0 is a continuous distribution whoseprobability density function is given byf(x)=λe−λx{\displaystyle f(x)=\lambda e^{-\lambda x}}on the interval[0, ∞). Its mean can be shown to beE⁡[X]=∫0∞xλe−λxdx=1λ.{\displaystyle \operatorname {E} [X]=\int _{0}^{\infty }x\lambda e^{-\lambda x}\,dx={\frac {1}{\lambda }}.}

Usingintegration by parts and making use of the expected value already calculated, we have:E⁡[X2]=∫0∞x2λe−λxdx=[−x2e−λx]0∞+∫0∞2xe−λxdx=0+2λE⁡[X]=2λ2.{\displaystyle {\begin{aligned}\operatorname {E} \left[X^{2}\right]&=\int _{0}^{\infty }x^{2}\lambda e^{-\lambda x}\,dx\\&={\left[-x^{2}e^{-\lambda x}\right]}_{0}^{\infty }+\int _{0}^{\infty }2xe^{-\lambda x}\,dx\\&=0+{\frac {2}{\lambda }}\operatorname {E} [X]\\&={\frac {2}{\lambda ^{2}}}.\end{aligned}}}

Thus, the variance ofX is given byVar⁡(X)=E⁡[X2]−E⁡[X]2=2λ2−(1λ)2=1λ2.{\displaystyle \operatorname {Var} (X)=\operatorname {E} \left[X^{2}\right]-\operatorname {E} [X]^{2}={\frac {2}{\lambda ^{2}}}-\left({\frac {1}{\lambda }}\right)^{2}={\frac {1}{\lambda ^{2}}}.}

Fair die

[edit]

A fairsix-sided die can be modeled as a discrete random variable,X, with outcomes 1 through 6, each with equal probability 1/6. The expected value ofX is(1+2+3+4+5+6)/6=7/2.{\displaystyle (1+2+3+4+5+6)/6=7/2.} Therefore, the variance ofX isVar⁡(X)=∑i=1616(i−72)2=16((−5/2)2+(−3/2)2+(−1/2)2+(1/2)2+(3/2)2+(5/2)2)=3512≈2.92.{\displaystyle {\begin{aligned}\operatorname {Var} (X)&=\sum _{i=1}^{6}{\frac {1}{6}}\left(i-{\frac {7}{2}}\right)^{2}\\[5pt]&={\frac {1}{6}}\left((-5/2)^{2}+(-3/2)^{2}+(-1/2)^{2}+(1/2)^{2}+(3/2)^{2}+(5/2)^{2}\right)\\[5pt]&={\frac {35}{12}}\approx 2.92.\end{aligned}}}

The general formula for the variance of the outcome,X, of ann-sided die isVar⁡(X)=E⁡(X2)−(E⁡(X))2=1n∑i=1ni2−(1n∑i=1ni)2=(n+1)(2n+1)6−(n+12)2=n2−112.{\displaystyle {\begin{aligned}\operatorname {Var} (X)&=\operatorname {E} \left(X^{2}\right)-(\operatorname {E} (X))^{2}\\[5pt]&={\frac {1}{n}}\sum _{i=1}^{n}i^{2}-\left({\frac {1}{n}}\sum _{i=1}^{n}i\right)^{2}\\[5pt]&={\frac {(n+1)(2n+1)}{6}}-\left({\frac {n+1}{2}}\right)^{2}\\[4pt]&={\frac {n^{2}-1}{12}}.\end{aligned}}}

Commonly used probability distributions

[edit]

The following table lists the variance for some commonly used probability distributions.

Name of the probability distributionProbability distribution functionMeanVariance
Binomial distributionPr(X=k)=(nk)pk(1−p)n−k{\displaystyle \Pr \,(X=k)={\binom {n}{k}}p^{k}(1-p)^{n-k}}np{\displaystyle np}np(1−p){\displaystyle np(1-p)}
Geometric distributionPr(X=k)=(1−p)k−1p{\displaystyle \Pr \,(X=k)=(1-p)^{k-1}p}1p{\displaystyle {\frac {1}{p}}}(1−p)p2{\displaystyle {\frac {(1-p)}{p^{2}}}}
Normal distributionf(x∣μ,σ2)=12πσ2e−12(x−μσ)2{\displaystyle f\left(x\mid \mu ,\sigma ^{2}\right)={\frac {1}{\sqrt {2\pi \sigma ^{2}}}}e^{-{\frac {1}{2}}{\left({\frac {x-\mu }{\sigma }}\right)}^{2}}}μ{\displaystyle \mu }σ2{\displaystyle \sigma ^{2}}
Uniform distribution (continuous)f(x∣a,b)={1b−afor a≤x≤b,0for x<a or x>b{\displaystyle f(x\mid a,b)={\begin{cases}{\frac {1}{b-a}}&{\text{for }}a\leq x\leq b,\\[3pt]0&{\text{for }}x<a{\text{ or }}x>b\end{cases}}}a+b2{\displaystyle {\frac {a+b}{2}}}(b−a)212{\displaystyle {\frac {(b-a)^{2}}{12}}}
Exponential distributionf(x∣λ)=λe−λx{\displaystyle f(x\mid \lambda )=\lambda e^{-\lambda x}}1λ{\displaystyle {\frac {1}{\lambda }}}1λ2{\displaystyle {\frac {1}{\lambda ^{2}}}}
Poisson distributionf(k∣λ)=e−λλkk!{\displaystyle f(k\mid \lambda )={\frac {e^{-\lambda }\lambda ^{k}}{k!}}}λ{\displaystyle \lambda }λ{\displaystyle \lambda }

Properties

[edit]

Basic properties

[edit]

Variance is non-negative because the squares are positive or zero:Var⁡(X)≥0.{\displaystyle \operatorname {Var} (X)\geq 0.}

The variance of a constant is zero.Var⁡(a)=0.{\displaystyle \operatorname {Var} (a)=0.}

Conversely, if the variance of a random variable is 0, then it isalmost surely a constant. That is, it always has the same value:Var⁡(X)=0⟺∃a:P(X=a)=1.{\displaystyle \operatorname {Var} (X)=0\iff \exists a:P(X=a)=1.}

Issues of finiteness

[edit]

If a distribution does not have a finite expected value, as is the case for theCauchy distribution, then the variance cannot be finite either. However, some distributions may not have a finite variance, despite their expected value being finite. An example is aPareto distribution whoseindexk{\displaystyle k} satisfies1<k≤2.{\displaystyle 1<k\leq 2.}

Decomposition

[edit]

The general formula for variance decomposition or thelaw of total variance is: IfX{\displaystyle X} andY{\displaystyle Y} are two random variables, and the variance ofX{\displaystyle X} exists, then

Var⁡[X]=E⁡(Var⁡[X∣Y])+Var⁡(E⁡[X∣Y]).{\displaystyle \operatorname {Var} [X]=\operatorname {E} (\operatorname {Var} [X\mid Y])+\operatorname {Var} (\operatorname {E} [X\mid Y]).}

Theconditional expectationE⁡(X∣Y){\displaystyle \operatorname {E} (X\mid Y)} ofX{\displaystyle X} givenY{\displaystyle Y}, and theconditional varianceVar⁡(X∣Y){\displaystyle \operatorname {Var} (X\mid Y)} may be understood as follows. Given any particular valuey of the random variable Y, there is a conditional expectationE⁡(X∣Y=y){\displaystyle \operatorname {E} (X\mid Y=y)} given the event Y = y. This quantity depends on the particular value y; it is a functiong(y)=E⁡(X∣Y=y){\displaystyle g(y)=\operatorname {E} (X\mid Y=y)}. That same function evaluated at the random variableY is the conditional expectationE⁡(X∣Y)=g(Y).{\displaystyle \operatorname {E} (X\mid Y)=g(Y).}

In particular, ifY{\displaystyle Y} is a discrete random variable assuming possible valuesy1,y2,y3…{\displaystyle y_{1},y_{2},y_{3}\ldots } with corresponding probabilitiesp1,p2,p3…,{\displaystyle p_{1},p_{2},p_{3}\ldots ,}, then in the formula for total variance, the first term on the right-hand side becomes

E⁡(Var⁡[X∣Y])=∑ipiσi2,{\displaystyle \operatorname {E} (\operatorname {Var} [X\mid Y])=\sum _{i}p_{i}\sigma _{i}^{2},}

whereσi2=Var⁡[X∣Y=yi]{\displaystyle \sigma _{i}^{2}=\operatorname {Var} [X\mid Y=y_{i}]}. Similarly, the second term on the right-hand side becomes

Var⁡(E⁡[X∣Y])=∑ipiμi2−(∑ipiμi)2=∑ipiμi2−μ2,{\displaystyle \operatorname {Var} (\operatorname {E} [X\mid Y])=\sum _{i}p_{i}\mu _{i}^{2}-\left(\sum _{i}p_{i}\mu _{i}\right)^{2}=\sum _{i}p_{i}\mu _{i}^{2}-\mu ^{2},}

whereμi=E⁡[X∣Y=yi]{\displaystyle \mu _{i}=\operatorname {E} [X\mid Y=y_{i}]} andμ=∑ipiμi{\textstyle \mu =\sum _{i}p_{i}\mu _{i}}. Thus the total variance is given by

Var⁡[X]=∑ipiσi2+(∑ipiμi2−μ2).{\displaystyle \operatorname {Var} [X]=\sum _{i}p_{i}\sigma _{i}^{2}+\left(\sum _{i}p_{i}\mu _{i}^{2}-\mu ^{2}\right).}

A similar formula is applied inanalysis of variance, where the corresponding formula is

MStotal=MSbetween+MSwithin;{\displaystyle {\mathit {MS}}_{\text{total}}={\mathit {MS}}_{\text{between}}+{\mathit {MS}}_{\text{within}};}

hereMS{\displaystyle {\mathit {MS}}} refers to the Mean of the Squares. Inlinear regression analysis the corresponding formula is

MStotal=MSregression+MSresidual.{\displaystyle {\mathit {MS}}_{\text{total}}={\mathit {MS}}_{\text{regression}}+{\mathit {MS}}_{\text{residual}}.}

This can also be derived from the additivity of variances, since the total (observed) score is the sum of the predicted score and the error score, where the latter two are uncorrelated.

Similar decompositions are possible for the sum of squared deviations (sum of squares,SS{\displaystyle {\mathit {SS}}}):SStotal=SSbetween+SSwithin,{\displaystyle {\mathit {SS}}_{\text{total}}={\mathit {SS}}_{\text{between}}+{\mathit {SS}}_{\text{within}},}SStotal=SSregression+SSresidual.{\displaystyle {\mathit {SS}}_{\text{total}}={\mathit {SS}}_{\text{regression}}+{\mathit {SS}}_{\text{residual}}.}

Calculation from the CDF

[edit]

The population variance for a non-negative random variable can be expressed in terms of thecumulative distribution functionF using

2∫0∞u(1−F(u))du−[∫0∞(1−F(u))du]2.{\displaystyle 2\int _{0}^{\infty }u(1-F(u))\,du-{\left[\int _{0}^{\infty }(1-F(u))\,du\right]}^{2}.}

This expression can be used to calculate the variance in situations where the CDF, but not thedensity, can be conveniently expressed.

Characteristic property

[edit]

The secondmoment of a random variable attains the minimum value when taken around the first moment (i.e., mean) of the random variable, i.e.argminmE((X−m)2)=E(X){\displaystyle \mathrm {argmin} _{m}\,\mathrm {E} \left(\left(X-m\right)^{2}\right)=\mathrm {E} (X)}. Conversely, if a continuous functionφ{\displaystyle \varphi } satisfiesargminmE(φ(X−m))=E(X){\displaystyle \mathrm {argmin} _{m}\,\mathrm {E} (\varphi (X-m))=\mathrm {E} (X)} for all random variablesX, then it is necessarily of the formφ(x)=ax2+b{\displaystyle \varphi (x)=ax^{2}+b}, wherea > 0. This also holds in the multidimensional case.[3]

Units of measurement

[edit]

Unlike theexpected absolute deviation, the variance of a variable has units that are the square of the units of the variable itself. For example, a variable measured in meters will have a variance measured in meters squared. For this reason, describing data sets via theirstandard deviation orroot mean square deviation is often preferred over using the variance. In the dice example the standard deviation is√2.9 ≈ 1.7, slightly larger than the expected absolute deviation of 1.5.

The standard deviation and the expected absolute deviation can both be used as an indicator of the "spread" of a distribution. The standard deviation is more amenable to algebraic manipulation than the expected absolute deviation, and, together with variance and its generalizationcovariance, is used frequently in theoretical statistics; however the expected absolute deviation tends to be morerobust as it is less sensitive tooutliers arising frommeasurement anomalies or an undulyheavy-tailed distribution.

Propagation

[edit]

Addition and multiplication by a constant

[edit]

Variance isinvariant with respect to changes in alocation parameter. That is, if a constant is added to all values of the variable, the variance is unchanged:Var⁡(X+a)=Var⁡(X).{\displaystyle \operatorname {Var} (X+a)=\operatorname {Var} (X).}

If all values are scaled by a constant, the variance isscaled by the square of that constant:Var⁡(aX)=a2Var⁡(X).{\displaystyle \operatorname {Var} (aX)=a^{2}\operatorname {Var} (X).}

The variance of a sum of two random variables is given byVar⁡(aX+bY)=a2Var⁡(X)+b2Var⁡(Y)+2abCov⁡(X,Y)Var⁡(aX−bY)=a2Var⁡(X)+b2Var⁡(Y)−2abCov⁡(X,Y){\displaystyle {\begin{aligned}\operatorname {Var} (aX+bY)&=a^{2}\operatorname {Var} (X)+b^{2}\operatorname {Var} (Y)+2ab\,\operatorname {Cov} (X,Y)\\[1ex]\operatorname {Var} (aX-bY)&=a^{2}\operatorname {Var} (X)+b^{2}\operatorname {Var} (Y)-2ab\,\operatorname {Cov} (X,Y)\end{aligned}}}

whereCov⁡(X,Y){\displaystyle \operatorname {Cov} (X,Y)} is thecovariance.

Linear combinations

[edit]

In general, for the sum ofN{\displaystyle N} random variables{X1,…,XN}{\displaystyle \{X_{1},\dots ,X_{N}\}}, the variance becomes:Var⁡(∑i=1NXi)=∑i,j=1NCov⁡(Xi,Xj)=∑i=1NVar⁡(Xi)+∑i,j=1,i≠jNCov⁡(Xi,Xj),{\displaystyle \operatorname {Var} \left(\sum _{i=1}^{N}X_{i}\right)=\sum _{i,j=1}^{N}\operatorname {Cov} (X_{i},X_{j})=\sum _{i=1}^{N}\operatorname {Var} (X_{i})+\sum _{i,j=1,i\neq j}^{N}\operatorname {Cov} (X_{i},X_{j}),} see also generalBienaymé's identity.

These results lead to the variance of alinear combination as:

Var⁡(∑i=1NaiXi)=∑i,j=1NaiajCov⁡(Xi,Xj)=∑i=1Nai2Var⁡(Xi)+∑i≠jaiajCov⁡(Xi,Xj)=∑i=1Nai2Var⁡(Xi)+2∑1≤i<j≤NaiajCov⁡(Xi,Xj).{\displaystyle {\begin{aligned}\operatorname {Var} \left(\sum _{i=1}^{N}a_{i}X_{i}\right)&=\sum _{i,j=1}^{N}a_{i}a_{j}\operatorname {Cov} (X_{i},X_{j})\\&=\sum _{i=1}^{N}a_{i}^{2}\operatorname {Var} (X_{i})+\sum _{i\neq j}a_{i}a_{j}\operatorname {Cov} (X_{i},X_{j})\\&=\sum _{i=1}^{N}a_{i}^{2}\operatorname {Var} (X_{i})+2\sum _{1\leq i<j\leq N}a_{i}a_{j}\operatorname {Cov} (X_{i},X_{j}).\end{aligned}}}

If the random variablesX1,…,XN{\displaystyle X_{1},\dots ,X_{N}} are such thatCov⁡(Xi,Xj)=0 , ∀ (i≠j),{\displaystyle \operatorname {Cov} (X_{i},X_{j})=0\ ,\ \forall \ (i\neq j),}then they are said to beuncorrelated. It follows immediately from the expression given earlier that if the random variablesX1,…,XN{\displaystyle X_{1},\dots ,X_{N}} are uncorrelated, then the variance of their sum is equal to the sum of their variances, or, expressed symbolically:

Var⁡(∑i=1NXi)=∑i=1NVar⁡(Xi).{\displaystyle \operatorname {Var} \left(\sum _{i=1}^{N}X_{i}\right)=\sum _{i=1}^{N}\operatorname {Var} (X_{i}).}

Since independent random variables are always uncorrelated (seeCovariance § Uncorrelatedness and independence), the equation above holds in particular when the random variablesX1,…,Xn{\displaystyle X_{1},\dots ,X_{n}} are independent. Thus, independence is sufficient but not necessary for the variance of the sum to equal the sum of the variances.

Matrix notation for the variance of a linear combination

[edit]

DefineX{\displaystyle X} as a column vector ofn{\displaystyle n} random variablesX1,…,Xn{\displaystyle X_{1},\ldots ,X_{n}}, andc{\displaystyle c} as a column vector ofn{\displaystyle n} scalarsc1,…,cn{\displaystyle c_{1},\ldots ,c_{n}}. Therefore,cTX{\displaystyle c^{\mathsf {T}}X} is alinear combination of these random variables, wherecT{\displaystyle c^{\mathsf {T}}} denotes thetranspose ofc{\displaystyle c}. Also letΣ{\displaystyle \Sigma } be thecovariance matrix ofX{\displaystyle X}. The variance ofcTX{\displaystyle c^{\mathsf {T}}X} is then given by:[4]

Var⁡(cTX)=cTΣc.{\displaystyle \operatorname {Var} \left(c^{\mathsf {T}}X\right)=c^{\mathsf {T}}\Sigma c.}

This implies that the variance of the mean can be written as (with a column vector of ones)

Var⁡(x¯)=Var⁡(1n1′X)=1n21′Σ1.{\displaystyle \operatorname {Var} \left({\bar {x}}\right)=\operatorname {Var} \left({\frac {1}{n}}1'X\right)={\frac {1}{n^{2}}}1'\Sigma 1.}

Sum of variables

[edit]

Sum of uncorrelated variables

[edit]
Main article:Bienaymé's identity
See also:Sum of normally distributed random variables

One reason for the use of the variance in preference to other measures of dispersion is that the variance of the sum (or the difference) ofuncorrelated random variables is the sum of their variances:

Var⁡(∑i=1nXi)=∑i=1nVar⁡(Xi).{\displaystyle \operatorname {Var} \left(\sum _{i=1}^{n}X_{i}\right)=\sum _{i=1}^{n}\operatorname {Var} (X_{i}).}

This statement is called theBienaymé formula[5] and was discovered in 1853.[6][7] It is often made with the stronger condition that the variables areindependent, but being uncorrelated suffices. So if all the variables have the same variance σ2, then, since division byn is a linear transformation, this formula immediately implies that the variance of their mean is

Var⁡(X¯)=Var⁡(1n∑i=1nXi)=1n2∑i=1nVar⁡(Xi)=1n2nσ2=σ2n.{\displaystyle \operatorname {Var} \left({\overline {X}}\right)=\operatorname {Var} \left({\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right)={\frac {1}{n^{2}}}\sum _{i=1}^{n}\operatorname {Var} \left(X_{i}\right)={\frac {1}{n^{2}}}n\sigma ^{2}={\frac {\sigma ^{2}}{n}}.}

That is, the variance of the mean decreases whenn increases. This formula for the variance of the mean is used in the definition of thestandard error of the sample mean, which is used in thecentral limit theorem.

To prove the initial statement, it suffices to show that

Var⁡(X+Y)=Var⁡(X)+Var⁡(Y).{\displaystyle \operatorname {Var} (X+Y)=\operatorname {Var} (X)+\operatorname {Var} (Y).}

The general result then follows by induction. Starting with the definition,

Var⁡(X+Y)=E⁡[(X+Y)2]−(E⁡[X+Y])2=E⁡[X2+2XY+Y2]−(E⁡[X]+E⁡[Y])2.{\displaystyle {\begin{aligned}\operatorname {Var} (X+Y)&=\operatorname {E} \left[(X+Y)^{2}\right]-(\operatorname {E} [X+Y])^{2}\\[5pt]&=\operatorname {E} \left[X^{2}+2XY+Y^{2}\right]-(\operatorname {E} [X]+\operatorname {E} [Y])^{2}.\end{aligned}}}

Using the linearity of theexpectation operator and the assumption of independence (or uncorrelatedness) ofX andY, this further simplifies as follows:

Var⁡(X+Y)=E⁡[X2]+2E⁡[XY]+E⁡[Y2]−(E⁡[X]2+2E⁡[X]E⁡[Y]+E⁡[Y]2)=E⁡[X2]+E⁡[Y2]−E⁡[X]2−E⁡[Y]2=Var⁡(X)+Var⁡(Y).{\displaystyle {\begin{aligned}\operatorname {Var} (X+Y)&=\operatorname {E} {\left[X^{2}\right]}+2\operatorname {E} [XY]+\operatorname {E} {\left[Y^{2}\right]}-\left(\operatorname {E} [X]^{2}+2\operatorname {E} [X]\operatorname {E} [Y]+\operatorname {E} [Y]^{2}\right)\\[5pt]&=\operatorname {E} \left[X^{2}\right]+\operatorname {E} \left[Y^{2}\right]-\operatorname {E} [X]^{2}-\operatorname {E} [Y]^{2}\\[5pt]&=\operatorname {Var} (X)+\operatorname {Var} (Y).\end{aligned}}}

Sum of correlated variables

[edit]
Sum of correlated variables with fixed sample size
[edit]
Main article:Bienaymé's identity

In general, the variance of the sum ofn variables is the sum of theircovariances:

Var⁡(∑i=1nXi)=∑i=1n∑j=1nCov⁡(Xi,Xj)=∑i=1nVar⁡(Xi)+2∑1≤i<j≤nCov⁡(Xi,Xj).{\displaystyle \operatorname {Var} \left(\sum _{i=1}^{n}X_{i}\right)=\sum _{i=1}^{n}\sum _{j=1}^{n}\operatorname {Cov} \left(X_{i},X_{j}\right)=\sum _{i=1}^{n}\operatorname {Var} \left(X_{i}\right)+2\sum _{1\leq i<j\leq n}\operatorname {Cov} \left(X_{i},X_{j}\right).}

(Note: The second equality comes from the fact thatCov(Xi,Xi) = Var(Xi).)

Here,Cov⁡(⋅,⋅){\displaystyle \operatorname {Cov} (\cdot ,\cdot )} is thecovariance, which is zero for independent random variables (if it exists). The formula states that the variance of a sum is equal to the sum of all elements in the covariance matrix of the components. The next expression states equivalently that the variance of the sum is the sum of the diagonal of covariance matrix plus two times the sum of its upper triangular elements (or its lower triangular elements); this emphasizes that the covariance matrix is symmetric. This formula is used in the theory ofCronbach's alpha inclassical test theory.

So, if the variables have equal varianceσ2 and the averagecorrelation of distinct variables isρ, then the variance of their mean is

Var⁡(X¯)=σ2n+n−1nρσ2.{\displaystyle \operatorname {Var} \left({\overline {X}}\right)={\frac {\sigma ^{2}}{n}}+{\frac {n-1}{n}}\rho \sigma ^{2}.}

This implies that the variance of the mean increases with the average of the correlations. In other words, additional correlated observations are not as effective as additional independent observations at reducing theuncertainty of the mean. Moreover, if the variables have unit variance, for example if they are standardized, then this simplifies to

Var⁡(X¯)=1n+n−1nρ.{\displaystyle \operatorname {Var} \left({\overline {X}}\right)={\frac {1}{n}}+{\frac {n-1}{n}}\rho .}

This formula is used in theSpearman–Brown prediction formula of classical test theory. This converges toρ ifn goes to infinity, provided that the average correlation remains constant or converges too. So for the variance of the mean of standardized variables with equal correlations or converging average correlation we have

limn→∞Var⁡(X¯)=ρ.{\displaystyle \lim _{n\to \infty }\operatorname {Var} \left({\overline {X}}\right)=\rho .}

Therefore, the variance of the mean of a large number of standardized variables is approximately equal to their average correlation. This makes clear that the sample mean of correlated variables does not generally converge to the population mean, even though thelaw of large numbers states that the sample mean will converge for independent variables.

Sum of uncorrelated variables with random sample size
[edit]

There are cases when a sample is taken without knowing, in advance, how many observations will be acceptable according to some criterion. In such cases, the sample sizeN is a random variable whose variation adds to the variation ofX, such that,[8]Var⁡(∑i=1NXi)=E⁡[N]Var⁡(X)+Var⁡(N)(E⁡[X])2{\displaystyle \operatorname {Var} \left(\sum _{i=1}^{N}X_{i}\right)=\operatorname {E} \left[N\right]\operatorname {Var} (X)+\operatorname {Var} (N)(\operatorname {E} \left[X\right])^{2}}which follows from thelaw of total variance.

IfN has aPoisson distribution, thenE⁡[N]=Var⁡(N){\displaystyle \operatorname {E} [N]=\operatorname {Var} (N)} with estimatorn =N. So, the estimator ofVar⁡(∑i=1nXi){\displaystyle \operatorname {Var} \left(\sum _{i=1}^{n}X_{i}\right)} becomesnSx2+nX¯2{\displaystyle n{S_{x}}^{2}+n{\bar {X}}^{2}}, givingSE⁡(X¯)=Sx2+X¯2n{\displaystyle \operatorname {SE} ({\bar {X}})={\sqrt {\frac {{S_{x}}^{2}+{\bar {X}}^{2}}{n}}}}(seestandard error of the sample mean).

Weighted sum of variables

[edit]
See also:Variance of a weighted arithmetic mean
Not to be confused withWeighted variance.

The scaling property and the Bienaymé formula, along with the property of thecovarianceCov(aX, bY) =ab Cov(X, Y) jointly imply that

Var⁡(aX±bY)=a2Var⁡(X)+b2Var⁡(Y)±2abCov⁡(X,Y).{\displaystyle \operatorname {Var} (aX\pm bY)=a^{2}\operatorname {Var} (X)+b^{2}\operatorname {Var} (Y)\pm 2ab\,\operatorname {Cov} (X,Y).}

This implies that in a weighted sum of variables, the variable with the largest weight will have a disproportionally large weight in the variance of the total. For example, ifX andY are uncorrelated and the weight ofX is two times the weight ofY, then the weight of the variance ofX will be four times the weight of the variance ofY.

The expression above can be extended to a weighted sum of multiple variables:

Var⁡(∑inaiXi)=∑i=1nai2Var⁡(Xi)+2∑1≤i∑<j≤naiajCov⁡(Xi,Xj){\displaystyle \operatorname {Var} \left(\sum _{i}^{n}a_{i}X_{i}\right)=\sum _{i=1}^{n}a_{i}^{2}\operatorname {Var} (X_{i})+2\sum _{1\leq i}\sum _{<j\leq n}a_{i}a_{j}\operatorname {Cov} (X_{i},X_{j})}

Product of variables

[edit]

Product of independent variables

[edit]

If two variables X and Y areindependent, the variance of their product is given by[9]Var⁡(XY)=[E⁡(X)]2Var⁡(Y)+[E⁡(Y)]2Var⁡(X)+Var⁡(X)Var⁡(Y).{\displaystyle \operatorname {Var} (XY)=[\operatorname {E} (X)]^{2}\operatorname {Var} (Y)+[\operatorname {E} (Y)]^{2}\operatorname {Var} (X)+\operatorname {Var} (X)\operatorname {Var} (Y).}

Equivalently, using the basic properties of expectation, it is given by

Var⁡(XY)=E⁡(X2)E⁡(Y2)−[E⁡(X)]2[E⁡(Y)]2.{\displaystyle \operatorname {Var} (XY)=\operatorname {E} \left(X^{2}\right)\operatorname {E} \left(Y^{2}\right)-[\operatorname {E} (X)]^{2}[\operatorname {E} (Y)]^{2}.}

Product of statistically dependent variables

[edit]

In general, if two variables are statistically dependent, then the variance of their product is given by:Var⁡(XY)=E⁡[X2Y2]−[E⁡(XY)]2=Cov⁡(X2,Y2)+E⁡(X2)E⁡(Y2)−[E⁡(XY)]2=Cov⁡(X2,Y2)+(Var⁡(X)+[E⁡(X)]2)(Var⁡(Y)+[E⁡(Y)]2)−[Cov⁡(X,Y)+E⁡(X)E⁡(Y)]2{\displaystyle {\begin{aligned}\operatorname {Var} (XY)={}&\operatorname {E} \left[X^{2}Y^{2}\right]-[\operatorname {E} (XY)]^{2}\\[5pt]={}&\operatorname {Cov} \left(X^{2},Y^{2}\right)+\operatorname {E} (X^{2})\operatorname {E} \left(Y^{2}\right)-[\operatorname {E} (XY)]^{2}\\[5pt]={}&\operatorname {Cov} \left(X^{2},Y^{2}\right)+\left(\operatorname {Var} (X)+[\operatorname {E} (X)]^{2}\right)\left(\operatorname {Var} (Y)+[\operatorname {E} (Y)]^{2}\right)\\[5pt]&-[\operatorname {Cov} (X,Y)+\operatorname {E} (X)\operatorname {E} (Y)]^{2}\end{aligned}}}

Arbitrary functions

[edit]
Main article:Uncertainty propagation

Thedelta method uses second-orderTaylor expansions to approximate the variance of a function of one or more random variables: seeTaylor expansions for the moments of functions of random variables. For example, the approximate variance of a function of one variable is given by

Var⁡[f(X)]≈(f′(E⁡[X]))2Var⁡[X]{\displaystyle \operatorname {Var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {Var} \left[X\right]}

provided thatf is twice differentiable and that the mean and variance ofX are finite.

Population variance and sample variance

[edit]

See also:Unbiased estimation of standard deviation

Real-world observations such as the measurements of yesterday's rain throughout the day typically cannot be complete sets of all possible observations that could be made. As such, the variance calculated from the finite set will in general not match the variance that would have been calculated from the full population of possible observations. This means that oneestimates the mean and variance from a limited set of observations by using anestimator equation. The estimator is a function of thesample ofnobservations drawn without observational bias from the wholepopulation of potential observations. In this example, the sample would be the set of actual measurements of yesterday's rainfall from available rain gauges within the geography of interest.

The simplest estimators for population mean and population variance are simply the mean and variance of the sample, thesample mean and(uncorrected) sample variance – these areconsistent estimators (they converge to the value of the whole population as the number of samples increases) but can be improved. Most simply, the sample variance is computed as the sum ofsquared deviations about the (sample) mean, divided byn as the number of samples. However, using values other thann improves the estimator in various ways. Four common values for the denominator aren,n − 1,n + 1, andn − 1.5:n is the simplest (the variance of the sample),n − 1 eliminates bias,[10]n + 1 minimizesmean squared error for the normal distribution,[11] andn − 1.5 mostly eliminates bias inunbiased estimation of standard deviation for the normal distribution.[12]

Firstly, if the true population mean is unknown, then the sample variance (which uses the sample mean in place of the true mean) is abiased estimator: it underestimates the variance by a factor of (n − 1) /n; correcting this factor, resulting in the sum of squared deviations about the sample mean divided byn -1 instead ofn, is calledBessel's correction.[10] The resulting estimator is unbiased and is called the(corrected) sample variance orunbiased sample variance. If the mean is determined in some other way than from the same samples used to estimate the variance, then this bias does not arise, and the variance can safely be estimated as that of the samples about the (independently known) mean.

Secondly, the sample variance does not generally minimizemean squared error between sample variance and population variance. Correcting for bias often makes this worse: one can always choose a scale factor that performs better than the corrected sample variance, though the optimal scale factor depends on theexcess kurtosis of the population (seemean squared error: variance) and introduces bias. This always consists of scaling down the unbiased estimator (dividing by a number larger thann − 1) and is a simple example of ashrinkage estimator: one "shrinks" the unbiased estimator towards zero. For the normal distribution, dividing byn + 1 (instead ofn − 1 orn) minimizes mean squared error.[11] The resulting estimator is biased, however, and is known as thebiased sample variation.

Population variance

[edit]

In general, thepopulation variance of afinitepopulation of sizeN with valuesxi is given byσ2=1N∑i=1N(xi−μ)2=1N∑i=1N(xi2−2μxi+μ2)=(1N∑i=1Nxi2)−2μ(1N∑i=1Nxi)+μ2=E⁡[xi2]−μ2{\displaystyle {\begin{aligned}\sigma ^{2}&={\frac {1}{N}}\sum _{i=1}^{N}{\left(x_{i}-\mu \right)}^{2}={\frac {1}{N}}\sum _{i=1}^{N}\left(x_{i}^{2}-2\mu x_{i}+\mu ^{2}\right)\\[5pt]&=\left({\frac {1}{N}}\sum _{i=1}^{N}x_{i}^{2}\right)-2\mu \left({\frac {1}{N}}\sum _{i=1}^{N}x_{i}\right)+\mu ^{2}\\[5pt]&=\operatorname {E} [x_{i}^{2}]-\mu ^{2}\end{aligned}}}

where the population mean isμ=E⁡[xi]=1N∑i=1Nxi{\textstyle \mu =\operatorname {E} [x_{i}]={\frac {1}{N}}\sum _{i=1}^{N}x_{i}} andE⁡[xi2]=(1N∑i=1Nxi2){\textstyle \operatorname {E} [x_{i}^{2}]=\left({\frac {1}{N}}\sum _{i=1}^{N}x_{i}^{2}\right)}, whereE{\textstyle \operatorname {E} } is theexpectation value operator.

The population variance can also be computed using[13]

σ2=1N2∑i<j(xi−xj)2=12N2∑i,j=1N(xi−xj)2.{\displaystyle \sigma ^{2}={\frac {1}{N^{2}}}\sum _{i<j}\left(x_{i}-x_{j}\right)^{2}={\frac {1}{2N^{2}}}\sum _{i,j=1}^{N}\left(x_{i}-x_{j}\right)^{2}.}

(The right side has duplicate terms in the sum while the middle side has only unique terms to sum.) This is true because12N2∑i,j=1N(xi−xj)2=12N2∑i,j=1N(xi2−2xixj+xj2)=12N∑j=1N(1N∑i=1Nxi2)−(1N∑i=1Nxi)(1N∑j=1Nxj)+12N∑i=1N(1N∑j=1Nxj2)=12(σ2+μ2)−μ2+12(σ2+μ2)=σ2.{\displaystyle {\begin{aligned}&{\frac {1}{2N^{2}}}\sum _{i,j=1}^{N}{\left(x_{i}-x_{j}\right)}^{2}\\[5pt]={}&{\frac {1}{2N^{2}}}\sum _{i,j=1}^{N}\left(x_{i}^{2}-2x_{i}x_{j}+x_{j}^{2}\right)\\[5pt]={}&{\frac {1}{2N}}\sum _{j=1}^{N}\left({\frac {1}{N}}\sum _{i=1}^{N}x_{i}^{2}\right)-\left({\frac {1}{N}}\sum _{i=1}^{N}x_{i}\right)\left({\frac {1}{N}}\sum _{j=1}^{N}x_{j}\right)+{\frac {1}{2N}}\sum _{i=1}^{N}\left({\frac {1}{N}}\sum _{j=1}^{N}x_{j}^{2}\right)\\[5pt]={}&{\frac {1}{2}}\left(\sigma ^{2}+\mu ^{2}\right)-\mu ^{2}+{\frac {1}{2}}\left(\sigma ^{2}+\mu ^{2}\right)\\[5pt]={}&\sigma ^{2}.\end{aligned}}}

The population variance matches the variance of the generating probability distribution. In this sense, the concept of population can be extended to continuous random variables with infinite populations.

Sample variance

[edit]
See also:Sample standard deviation

Biased sample variance

[edit]

In many practical situations, the true variance of a population is not knowna priori and must be computed somehow. When dealing with extremely large populations, it is not possible to count every object in the population, so the computation must be performed on asample of the population.[14] This is generally referred to assample variance orempirical variance. Sample variance can also be applied to the estimation of the variance of a continuous distribution from a sample of that distribution.

We take asample with replacement ofn valuesY1, ...,Yn from the population of sizeN, wheren <N, and estimate the variance on the basis of this sample.[15] Directly taking the variance of the sample data gives the average of thesquared deviations:[16]

S~Y2=1n∑i=1n(Yi−Y¯)2=(1n∑i=1nYi2)−Y¯2=1n2∑i,j:i<j(Yi−Yj)2.{\displaystyle {\tilde {S}}_{Y}^{2}={\frac {1}{n}}\sum _{i=1}^{n}\left(Y_{i}-{\overline {Y}}\right)^{2}=\left({\frac {1}{n}}\sum _{i=1}^{n}Y_{i}^{2}\right)-{\overline {Y}}^{2}={\frac {1}{n^{2}}}\sum _{i,j\,:\,i<j}\left(Y_{i}-Y_{j}\right)^{2}.}

(See the sectionPopulation variance for the derivation of this formula.) Here,Y¯{\displaystyle {\overline {Y}}} denotes thesample mean:Y¯=1n∑i=1nYi.{\displaystyle {\overline {Y}}={\frac {1}{n}}\sum _{i=1}^{n}Y_{i}.}

Since theYi are selected randomly, bothY¯{\displaystyle {\overline {Y}}} andS~Y2{\displaystyle {\tilde {S}}_{Y}^{2}} arerandom variables. Their expected values can be evaluated by averaging over the ensemble of all possible samples{Yi} of sizen from the population. ForS~Y2{\displaystyle {\tilde {S}}_{Y}^{2}} this gives:E⁡[S~Y2]=E⁡[1n∑i=1n(Yi−1n∑j=1nYj)2]=1n∑i=1nE⁡[Yi2−2nYi∑j=1nYj+1n2∑j=1nYj∑k=1nYk]=1n∑i=1n(E⁡[Yi2]−2n(∑j≠iE⁡[YiYj]+E⁡[Yi2])+1n2∑j=1n∑k≠jnE⁡[YjYk]+1n2∑j=1nE⁡[Yj2])=1n∑i=1n(n−2nE⁡[Yi2]−2n∑j≠iE⁡[YiYj]+1n2∑j=1n∑k≠jnE⁡[YjYk]+1n2∑j=1nE⁡[Yj2])=1n∑i=1n[n−2n(σ2+μ2)−2n(n−1)μ2+1n2n(n−1)μ2+1n(σ2+μ2)]=n−1nσ2.{\displaystyle {\begin{aligned}\operatorname {E} [{\tilde {S}}_{Y}^{2}]&=\operatorname {E} \left[{\frac {1}{n}}\sum _{i=1}^{n}{\left(Y_{i}-{\frac {1}{n}}\sum _{j=1}^{n}Y_{j}\right)}^{2}\right]\\[5pt]&={\frac {1}{n}}\sum _{i=1}^{n}\operatorname {E} \left[Y_{i}^{2}-{\frac {2}{n}}Y_{i}\sum _{j=1}^{n}Y_{j}+{\frac {1}{n^{2}}}\sum _{j=1}^{n}Y_{j}\sum _{k=1}^{n}Y_{k}\right]\\[5pt]&={\frac {1}{n}}\sum _{i=1}^{n}\left(\operatorname {E} \left[Y_{i}^{2}\right]-{\frac {2}{n}}\left(\sum _{j\neq i}\operatorname {E} \left[Y_{i}Y_{j}\right]+\operatorname {E} \left[Y_{i}^{2}\right]\right)+{\frac {1}{n^{2}}}\sum _{j=1}^{n}\sum _{k\neq j}^{n}\operatorname {E} \left[Y_{j}Y_{k}\right]+{\frac {1}{n^{2}}}\sum _{j=1}^{n}\operatorname {E} \left[Y_{j}^{2}\right]\right)\\[5pt]&={\frac {1}{n}}\sum _{i=1}^{n}\left({\frac {n-2}{n}}\operatorname {E} \left[Y_{i}^{2}\right]-{\frac {2}{n}}\sum _{j\neq i}\operatorname {E} \left[Y_{i}Y_{j}\right]+{\frac {1}{n^{2}}}\sum _{j=1}^{n}\sum _{k\neq j}^{n}\operatorname {E} \left[Y_{j}Y_{k}\right]+{\frac {1}{n^{2}}}\sum _{j=1}^{n}\operatorname {E} \left[Y_{j}^{2}\right]\right)\\[5pt]&={\frac {1}{n}}\sum _{i=1}^{n}\left[{\frac {n-2}{n}}\left(\sigma ^{2}+\mu ^{2}\right)-{\frac {2}{n}}(n-1)\mu ^{2}+{\frac {1}{n^{2}}}n(n-1)\mu ^{2}+{\frac {1}{n}}\left(\sigma ^{2}+\mu ^{2}\right)\right]\\[5pt]&={\frac {n-1}{n}}\sigma ^{2}.\end{aligned}}}

Hereσ2=E⁡[Yi2]−μ2{\textstyle \sigma ^{2}=\operatorname {E} [Y_{i}^{2}]-\mu ^{2}} derived in the section ispopulation variance andE⁡[YiYj]=E⁡[Yi]E⁡[Yj]=μ2{\textstyle \operatorname {E} [Y_{i}Y_{j}]=\operatorname {E} [Y_{i}]\operatorname {E} [Y_{j}]=\mu ^{2}} due to independency ofYi{\textstyle Y_{i}} andYj{\textstyle Y_{j}}.

HenceS~Y2{\textstyle {\tilde {S}}_{Y}^{2}} gives an estimate of the population varianceσ2{\textstyle \sigma ^{2}} that is biased by a factor ofn−1n{\textstyle {\frac {n-1}{n}}} because the expectation value ofS~Y2{\textstyle {\tilde {S}}_{Y}^{2}} is smaller than the population variance (true variance) by that factor. For this reason,S~Y2{\textstyle {\tilde {S}}_{Y}^{2}} is referred to as thebiased sample variance.

Unbiased sample variance

[edit]

Correcting for this bias yields theunbiased sample variance, denotedS2{\displaystyle S^{2}}:

S2=nn−1S~Y2=nn−1[1n∑i=1n(Yi−Y¯)2]=1n−1∑i=1n(Yi−Y¯)2{\displaystyle S^{2}={\frac {n}{n-1}}{\tilde {S}}_{Y}^{2}={\frac {n}{n-1}}\left[{\frac {1}{n}}\sum _{i=1}^{n}\left(Y_{i}-{\overline {Y}}\right)^{2}\right]={\frac {1}{n-1}}\sum _{i=1}^{n}\left(Y_{i}-{\overline {Y}}\right)^{2}}

Either estimator may be simply referred to as thesample variance when the version can be determined by context. The same proof is also applicable for samples taken from a continuous probability distribution.

The use of the termn − 1 is calledBessel's correction, and it is also used insample covariance and thesample standard deviation (the square root of variance). The square root is aconcave function and thus introduces negative bias (byJensen's inequality), which depends on the distribution, and thus the corrected sample standard deviation (using Bessel's correction) is biased. Theunbiased estimation of standard deviation is a technically involved problem, though for the normal distribution using the termn − 1.5 yields an almost unbiased estimator.

The unbiased sample variance is aU-statistic for the functionf(y1,y2) = (y1 −y2)2/2, meaning that it is obtained by averaging a 2-sample statistic over 2-element subsets of the population.

Example
[edit]

For a set of numbers {10, 15, 30, 45, 57, 52, 63, 72, 81, 93, 102, 105}, if this set is the whole data population for some measurement, then variance is the population variance 932.743 as the sum of the squared deviations about the mean of this set, divided by 12 as the number of the set members. If the set is a sample from the whole population, then the unbiased sample variance can be calculated as 1017.538 that is the sum of the squared deviations about the mean of the sample, divided by 11 instead of 12. A function VAR.S inMicrosoft Excel gives the unbiased sample variance while VAR.P is for population variance.

Distribution of the sample variance

[edit]
Distribution and cumulative distribution ofS2/σ2, for various values ofν =n − 1, when theyi are independent normally distributed.

Being a function ofrandom variables, the sample variance is itself a random variable, and it is natural to study its distribution. In the case thatYi are independent observations from anormal distribution,Cochran's theorem shows that theunbiased sample varianceS2 follows a scaledchi-squared distribution (see also:asymptotic properties and anelementary proof):[17](n−1)S2σ2∼χn−12{\displaystyle (n-1){\frac {S^{2}}{\sigma ^{2}}}\sim \chi _{n-1}^{2}}

whereσ2 is thepopulation variance. As a direct consequence, it follows thatE⁡(S2)=E⁡(σ2n−1χn−12)=σ2,{\displaystyle \operatorname {E} \left(S^{2}\right)=\operatorname {E} \left({\frac {\sigma ^{2}}{n-1}}\chi _{n-1}^{2}\right)=\sigma ^{2},}

and[18]

Var⁡[S2]=Var⁡(σ2n−1χn−12)=σ4(n−1)2Var⁡(χn−12)=2σ4n−1.{\displaystyle \operatorname {Var} \left[S^{2}\right]=\operatorname {Var} \left({\frac {\sigma ^{2}}{n-1}}\chi _{n-1}^{2}\right)={\frac {\sigma ^{4}}{{\left(n-1\right)}^{2}}}\operatorname {Var} \left(\chi _{n-1}^{2}\right)={\frac {2\sigma ^{4}}{n-1}}.}

IfYi are independent and identically distributed, but not necessarily normally distributed, then[19]

E⁡[S2]=σ2,Var⁡[S2]=σ4n(κ−1+2n−1)=1n(μ4−n−3n−1σ4),{\displaystyle \operatorname {E} \left[S^{2}\right]=\sigma ^{2},\quad \operatorname {Var} \left[S^{2}\right]={\frac {\sigma ^{4}}{n}}\left(\kappa -1+{\frac {2}{n-1}}\right)={\frac {1}{n}}\left(\mu _{4}-{\frac {n-3}{n-1}}\sigma ^{4}\right),}

whereκ is thekurtosis of the distribution andμ4 is the fourthcentral moment.

If the conditions of thelaw of large numbers hold for the squared observations,S2 is aconsistent estimator of σ2. One can see indeed that the variance of the estimator tends asymptotically to zero. An asymptotically equivalent formula was given in Kenney and Keeping (1951:164), Rose and Smith (2002:264), and Weisstein (n.d.).[20][21][22]

Samuelson's inequality

[edit]

Samuelson's inequality is a result that states bounds on the values that individual observations in a sample can take, given that the sample mean and (biased) variance have been calculated.[23] Values must lie within the limitsy¯±σY(n−1)1/2.{\displaystyle {\bar {y}}\pm \sigma _{Y}(n-1)^{1/2}.}

Relations with the harmonic and arithmetic means

[edit]

It has been shown[24] that for a sample {yi} of positive real numbers,

σy2≤2ymax(A−H),{\displaystyle \sigma _{y}^{2}\leq 2y_{\max }(A-H),}

whereymax is the maximum of the sample,A is the arithmetic mean,H is theharmonic mean of the sample andσy2{\displaystyle \sigma _{y}^{2}} is the (biased) variance of the sample.

This bound has been improved, and it is known that variance is bounded by

σy2≤ymax(A−H)(ymax−A)ymax−H,σy2≥ymin(A−H)(A−ymin)H−ymin,{\displaystyle {\begin{aligned}\sigma _{y}^{2}&\leq {\frac {y_{\max }(A-H)(y_{\max }-A)}{y_{\max }-H}},\\[1ex]\sigma _{y}^{2}&\geq {\frac {y_{\min }(A-H)(A-y_{\min })}{H-y_{\min }}},\end{aligned}}}

whereymin is the minimum of the sample.[25]

Tests of equality of variances

[edit]

TheF-test of equality of variances and thechi square tests are adequate when the sample is normally distributed. Non-normality makes testing for the equality of two or more variances more difficult.

Several non parametric tests have been proposed: these include the Barton–David–Ansari–Freund–Siegel–Tukey test, theCapon test,Mood test, theKlotz test and theSukhatme test. The Sukhatme test applies to two variances and requires that bothmedians be known and equal to zero. The Mood, Klotz, Capon and Barton–David–Ansari–Freund–Siegel–Tukey tests also apply to two variances. They allow the median to be unknown but do require that the two medians are equal.

TheLehmann test is a parametric test of two variances. Of this test there are several variants known. Other tests of the equality of variances include theBox test, theBox–Anderson test and theMoses test.

Resampling methods, which include thebootstrap and thejackknife, may be used to test the equality of variances.

Moment of inertia

[edit]
See also:Moment (physics) § Examples

The variance of a probability distribution is analogous to themoment of inertia inclassical mechanics of a corresponding mass distribution along a line, with respect to rotation about its center of mass.[26] It is because of this analogy that such things as the variance are calledmoments ofprobability distributions.[26] The covariance matrix is related to themoment of inertia tensor for multivariate distributions. The moment of inertia of a cloud ofn points with a covariance matrix ofΣ{\displaystyle \Sigma } is given by[citation needed]I=n(13×3tr⁡(Σ)−Σ).{\displaystyle I=n\left(\mathbf {1} _{3\times 3}\operatorname {tr} (\Sigma )-\Sigma \right).}

This difference between moment of inertia in physics and in statistics is clear for points that are gathered along a line. Suppose many points are close to thex axis and distributed along it. The covariance matrix might look likeΣ=[100000.10000.1].{\displaystyle \Sigma ={\begin{bmatrix}10&0&0\\0&0.1&0\\0&0&0.1\end{bmatrix}}.}

That is, there is the most variance in thex direction. Physicists would consider this to have a low momentabout thex axis so the moment-of-inertia tensor isI=n[0.200010.100010.1].{\displaystyle I=n{\begin{bmatrix}0.2&0&0\\0&10.1&0\\0&0&10.1\end{bmatrix}}.}

Semivariance

[edit]

Thesemivariance is calculated in the same manner as the variance but only those observations that fall below the mean are included in the calculation:Semivariance=1n∑i:xi<μ(xi−μ)2{\displaystyle {\text{Semivariance}}={\frac {1}{n}}\sum _{i:x_{i}<\mu }{\left(x_{i}-\mu \right)}^{2}}It is also described as a specific measure in different fields of application. For skewed distributions, the semivariance can provide additional information that a variance does not.[27]

For inequalities associated with the semivariance, seeChebyshev's inequality § Semivariances.

Etymology

[edit]

The termvariance was first introduced byRonald Fisher in his 1918 paperThe Correlation Between Relatives on the Supposition of Mendelian Inheritance:[28]

The great body of available statistics show us that the deviations of ahuman measurement from its mean follow very closely theNormal Law of Errors, and, therefore, that the variability may be uniformly measured by thestandard deviation corresponding to thesquare root of themean square error. When there are two independent causes of variability capable of producing in an otherwise uniform population distributions with standard deviationsσ1{\displaystyle \sigma _{1}} andσ2{\displaystyle \sigma _{2}}, it is found that the distribution, when both causes act together, has a standard deviationσ12+σ22{\displaystyle {\sqrt {\sigma _{1}^{2}+\sigma _{2}^{2}}}}. It is therefore desirable in analysing the causes of variability to deal with the square of the standard deviation as the measure of variability. We shall term this quantity the Variance...

Generalizations

[edit]

For complex variables

[edit]

Ifx{\displaystyle x} is a scalarcomplex-valued random variable, with values inC,{\displaystyle \mathbb {C} ,} then its variance isE⁡[(x−μ)(x−μ)∗],{\displaystyle \operatorname {E} \left[(x-\mu )(x-\mu )^{*}\right],} wherex∗{\displaystyle x^{*}} is thecomplex conjugate ofx.{\displaystyle x.} This variance is a real scalar.

For vector-valued random variables

[edit]

As a matrix

[edit]

IfX{\displaystyle X} is avector-valued random variable, with values inRn,{\displaystyle \mathbb {R} ^{n},} and thought of as a column vector, then a natural generalization of variance isE⁡[(X−μ)(X−μ)T],{\displaystyle \operatorname {E} \left[(X-\mu ){(X-\mu )}^{\mathsf {T}}\right],} whereμ=E⁡(X){\displaystyle \mu =\operatorname {E} (X)} andXT{\displaystyle X^{\mathsf {T}}} is the transpose ofX, and so is a row vector. The result is apositive semi-definite square matrix, commonly referred to as thevariance-covariance matrix (or simply as thecovariance matrix).

IfX{\displaystyle X} is a vector- and complex-valued random variable, with values inCn,{\displaystyle \mathbb {C} ^{n},} then thecovariance matrix isE⁡[(X−μ)(X−μ)†],{\displaystyle \operatorname {E} \left[(X-\mu ){(X-\mu )}^{\dagger }\right],} whereX†{\displaystyle X^{\dagger }} is theconjugate transpose ofX.{\displaystyle X.}[citation needed] This matrix is also positive semi-definite and square.

As a scalar

[edit]

Another generalization of variance for vector-valued random variablesX{\displaystyle X}, which results in a scalar value rather than in a matrix, is thegeneralized variancedet(C){\displaystyle \det(C)}, thedeterminant of the covariance matrix. The generalized variance can be shown to be related to the multidimensional scatter of points around their mean.[29]

A different generalization is obtained by considering the equation for the scalar variance,Var⁡(X)=E⁡[(X−μ)2]{\displaystyle \operatorname {Var} (X)=\operatorname {E} \left[(X-\mu )^{2}\right]}, and reinterpreting(X−μ)2{\displaystyle (X-\mu )^{2}} as the squaredEuclidean distance between the random variable and its mean, or, simply as the scalar product of the vectorX−μ{\displaystyle X-\mu } with itself. This results inE⁡[(X−μ)T(X−μ)]=tr⁡(C),{\displaystyle \operatorname {E} \left[(X-\mu )^{\mathsf {T}}(X-\mu )\right]=\operatorname {tr} (C),} which is thetrace of the covariance matrix.

See also

[edit]
Look upvariance in Wiktionary, the free dictionary.

Types of variance

[edit]

References

[edit]
  1. ^Wasserman, Larry (2005).All of Statistics: a concise course in statistical inference. Springer texts in statistics. p. 51.ISBN 978-1-4419-2322-6.
  2. ^Yuli Zhang; Huaiyu Wu; Lei Cheng (June 2012).Some new deformation formulas about variance and covariance. Proceedings of 4th International Conference on Modelling, Identification and Control(ICMIC2012). pp. 987–992.
  3. ^Kagan, A.; Shepp, L. A. (1998). "Why the variance?".Statistics & Probability Letters.38 (4):329–333.doi:10.1016/S0167-7152(98)00041-8.
  4. ^Johnson, Richard; Wichern, Dean (2001).Applied Multivariate Statistical Analysis. Prentice Hall. p. 76.ISBN 0-13-187715-1.
  5. ^Loève, M. (1977) "Probability Theory",Graduate Texts in Mathematics, Volume 45, 4th edition, Springer-Verlag, p. 12.
  6. ^Bienaymé, I.-J. (1853) "Considérations à l'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés",Comptes rendus de l'Académie des sciences Paris, 37, p. 309–317; digital copy available[1]Archived 2018-06-23 at theWayback Machine
  7. ^Bienaymé, I.-J. (1867) "Considérations à l'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés",Journal de Mathématiques Pures et Appliquées, Série 2, Tome 12, p. 158–167; digital copy available[2][3]
  8. ^Cornell, J R, and Benjamin, C A,Probability, Statistics, and Decisions for Civil Engineers, McGraw-Hill, NY, 1970, pp.178-9.
  9. ^Goodman, Leo A. (December 1960). "On the Exact Variance of Products".Journal of the American Statistical Association.55 (292):708–713.doi:10.2307/2281592.JSTOR 2281592.
  10. ^abReichmann, W. J. (1961). "Appendix 8".Use and Abuse of Statistics (Reprinted 1964–1970 by Pelican ed.). London: Methuen.
  11. ^abKourouklis, Stavros (2012)."A New Estimator of the Variance Based on Minimizing Mean Squared Error".The American Statistician.66 (4):234–236.doi:10.1080/00031305.2012.735209.ISSN 0003-1305.JSTOR 23339501.
  12. ^Brugger, R. M. (1969). "A Note on Unbiased Estimation of the Standard Deviation".The American Statistician.23 (4): 32.doi:10.1080/00031305.1969.10481865.
  13. ^Yuli Zhang; Huaiyu Wu; Lei Cheng (June 2012).Some new deformation formulas about variance and covariance. Proceedings of 4th International Conference on Modelling, Identification and Control(ICMIC2012). pp. 987–992.
  14. ^Navidi, William (2006).Statistics for Engineers and Scientists. McGraw-Hill. p. 14.
  15. ^Montgomery, D. C. and Runger, G. C. (1994)Applied statistics and probability for engineers, page 201. John Wiley & Sons New York
  16. ^Yuli Zhang; Huaiyu Wu; Lei Cheng (June 2012).Some new deformation formulas about variance and covariance. Proceedings of 4th International Conference on Modelling, Identification and Control(ICMIC2012). pp. 987–992.
  17. ^Knight, K. (2000).Mathematical Statistics. New York: Chapman and Hall. proposition 2.11.
  18. ^Casella, George; Berger, Roger L. (2002).Statistical Inference (2nd ed.). Example 7.3.3, p. 331.ISBN 0-534-24312-6.
  19. ^Mood, A. M., Graybill, F. A., and Boes, D.C. (1974)Introduction to the Theory of Statistics, 3rd Edition, McGraw-Hill, New York, p. 229
  20. ^Kenney, John F.; Keeping, E.S. (1951).Mathematics of Statistics. Part Two(PDF) (2nd ed.). Princeton, New Jersey: D. Van Nostrand Company, Inc. Archived fromthe original(PDF) on Nov 17, 2018 – via KrishiKosh.
  21. ^Rose, Colin; Smith, Murray D. (2002). "Mathematical Statistics with Mathematica". Springer-Verlag, New York.
  22. ^Weisstein, Eric W. "Sample Variance Distribution". MathWorld Wolfram.
  23. ^Samuelson, Paul (1968). "How Deviant Can You Be?".Journal of the American Statistical Association.63 (324):1522–1525.doi:10.1080/01621459.1968.10480944.JSTOR 2285901.
  24. ^Mercer, A. McD. (2000)."Bounds for A–G, A–H, G–H, and a family of inequalities of Ky Fan's type, using a general method".J. Math. Anal. Appl.243 (1):163–173.doi:10.1006/jmaa.1999.6688.
  25. ^Sharma, R. (2008). "Some more inequalities for arithmetic mean, harmonic mean and variance".Journal of Mathematical Inequalities.2 (1):109–114.CiteSeerX 10.1.1.551.9397.doi:10.7153/jmi-02-11.
  26. ^abMagnello, M. Eileen."Karl Pearson and the Origins of Modern Statistics: An Elastician becomes a Statistician".The Rutherford Journal.
  27. ^Fama, Eugene F.; French, Kenneth R. (2010-04-21)."Q&A: Semi-Variance: A Better Risk Measure?".Fama/French Forum.
  28. ^Ronald Fisher (1918)The correlation between relatives on the supposition of Mendelian Inheritance
  29. ^Kocherlakota, S.; Kocherlakota, K. (2004). "Generalized Variance".Encyclopedia of Statistical Sciences. Wiley Online Library.doi:10.1002/0471667196.ess0869.ISBN 0-471-66719-6.
Continuous data
Center
Dispersion
Shape
Count data
Summary tables
Dependence
Graphics
Study design
Survey methodology
Controlled experiments
Adaptive designs
Observational studies
Statistical theory
Frequentist inference
Point estimation
Interval estimation
Testing hypotheses
Parametric tests
Specific tests
Goodness of fit
Rank statistics
Bayesian inference
Correlation
Regression analysis (see alsoTemplate:Least squares and regression analysis
Linear regression
Non-standard predictors
Generalized linear model
Partition of variance
Categorical
Multivariate
Time-series
General
Specific tests
Time domain
Frequency domain
Survival
Survival function
Hazard function
Test
Biostatistics
Engineering statistics
Social statistics
Spatial statistics
International
National
Retrieved from "https://en.wikipedia.org/w/index.php?title=Variance&oldid=1324241789"
Categories:
Hidden categories:
[8]ページ先頭
©2009-2025 Movatter.jp