This articlemay be too technical for most readers to understand. Pleasehelp improve it tomake it understandable to non-experts, without removing the technical details.(June 2014) (Learn how and when to remove this message)

Thesample mean (sample average) orempirical mean (empirical average), and thesample covariance orempirical covariance arestatistics computed from asample of data on one or morerandom variables.

The sample mean is theaverage value (ormean value) of asample of numbers taken from a largerpopulation of numbers, where "population" indicates not number of people but the entirety of relevant data, whether collected or not. A sample of 40 companies' sales from theFortune 500 might be used for convenience instead of looking at the population, all 500 companies' sales. The sample mean is used as anestimator for the population mean, the average value in the entire population, where the estimate is more likely to be close to the population mean if the sample is large and representative. The reliability of the sample mean is estimated using thestandard error, which in turn is calculated using thevariance of the sample. If the sample is random, the standard error falls with the size of the sample and the sample mean's distribution approaches the normal distribution as the sample size increases.

The term "sample mean" can also be used to refer to avector of average values when the statistician is looking at the values of several variables in the sample, e.g. the sales, profits, and employees of a sample of Fortune 500 companies. In this case, there is not just a sample variance for each variable but a samplevariance-covariance matrix (or simplycovariance matrix) showing also the relationship between each pair of variables. This would be a 3×3 matrix when 3 variables are being considered. The sample covariance is useful in judging the reliability of the sample means as estimators and is also useful as an estimate of the population covariance matrix.

Due to their ease of calculation and other desirable characteristics, the sample mean and sample covariance are widely used in statistics to represent thelocation anddispersion of thedistribution of values in the sample, and to estimate the values for the population.

The sample mean is the average of the values of a variable in a sample, which is the sum of those values divided by the number of values. Usingmathematical notation, if a sample ofN observations on variableX is taken from the population, the sample mean is:

${\displaystyle \overline{X}=\frac{1}{N}\sum _{i=1}^N{X}_i.}$

Under this definition, if the sample (1, 4, 1) is taken from the population (1,1,3,4,0,2,1,0), then the sample mean is${\displaystyle \overline{x}=(1+4+1)/3=2}$, as compared to the population mean of${\displaystyle \mu =(1+1+3+4+0+2+1+0)/8=12/8=1.5}$. Even if a sample is random, it is rarely perfectly representative, and other samples would have other sample means even if the samples were all from the same population. The sample (2, 1, 0), for example, would have a sample mean of 1.

If the statistician is interested inK variables rather than one, each observation having a value for each of thoseK variables, the overall sample mean consists ofK sample means for individual variables. Let${\displaystyle x_{ij}}$ be thei^th independently drawn observation (i=1,...,N) on thej^th random variable (j=1,...,K). These observations can be arranged intoNcolumn vectors, each withK entries, with theK×1 column vector giving thei-th observations of all variables being denoted${\displaystyle {\mathbf{x}}_i}$ (i=1,...,N).

Thesample mean vector${\displaystyle \overline{\mathbf{x}}}$ is a column vector whosej-th element${\displaystyle {\overline{x}}_j}$ is the average value of theN observations of thej^th variable:

${\displaystyle {\overline{x}}_j=\frac{1}{N}\sum _{i=1}^N{x}_{ij},j=1,\dots ,K.}$

Thus, the sample mean vector contains the average of the observations for each variable, and is written

${\displaystyle \overline{\mathbf{x}}=\frac{1}{N}\sum _{i=1}^N{\mathbf{x}}_i=\left[\begin{array}{c}{\overline{x}}_1\\ ⋮\\ {\overline{x}}_j\\ ⋮\\ {\overline{x}}_K\end{array}\right]}$

Thesample covariance matrix is aK-by-Kmatrix${\displaystyle \mathbf{Q}=\left[q_{jk}\right]}$ with entries

${\displaystyle q_{jk}=\frac{1}{N-1}\sum _{i=1}^N\left(x_{ij}-{\overline{x}}_j\right)\left(x_{ik}-{\overline{x}}_k\right),}$

where${\displaystyle q_{jk}}$ is an estimate of thecovariance between thej^thvariable and thek^th variable of the population underlying the data.In terms of the observation vectors, the sample covariance is

${\displaystyle \mathbf{Q}=\frac{1}{N-1}\sum _{i=1}^N({\mathbf{x}}_i.-\overline{\mathbf{x}})({\mathbf{x}}_i.-\overline{\mathbf{x}}{)}^{\mathrm{T}},}$

Alternatively, arranging the observation vectors as the columns of a matrix, so that

,

which is a matrix ofK rows andN columns.Here, the sample covariance matrix can be computed as

${\displaystyle \mathbf{Q}=\frac{1}{N-1}(\mathbf{F}-\overline{\mathbf{x}}{\mathbf{1}}_N^{\mathrm{T}})(\mathbf{F}-\overline{\mathbf{x}}{\mathbf{1}}_N^{\mathrm{T}}{)}^{\mathrm{T}}}$,

where${\displaystyle {\mathbf{1}}_N}$ is anN by1 vector of ones. If the observations are arranged as rows instead of columns, so${\displaystyle \overline{\mathbf{x}}}$ is now a 1×K row vector and${\displaystyle \mathbf{M}={\mathbf{F}}^{\mathrm{T}}}$ is anN×K matrix whose columnj is the vector ofN observations on variablej, then applying transposes in the appropriate places yields

${\displaystyle \mathbf{Q}=\frac{1}{N-1}(\mathbf{M}-{\mathbf{1}}_N\overline{\mathbf{x}}{)}^{\mathrm{T}}(\mathbf{M}-{\mathbf{1}}_N\overline{\mathbf{x}}).}$

Like covariance matrices forrandom vector, sample covariance matrices arepositive semi-definite. To prove it, note that for any matrix${\displaystyle \mathbf{A}}$ the matrix${\displaystyle {\mathbf{A}}^T\mathbf{A}}$ is positive semi-definite. Furthermore, a covariance matrix is positive definiteif and only if the rank of the${\displaystyle {\mathbf{x}}_i.-\overline{\mathbf{x}}}$ vectors is K.

The sample mean and the sample covariance matrix areunbiased estimates of themean and thecovariance matrix of therandom vector${\displaystyle \mathbf{X}}$, a row vector whosej^th element (j = 1, ..., K) is one of the random variables.^[1] The sample covariance matrix has${\displaystyle N-1}$ in the denominator rather than${\displaystyle N}$ due to a variant ofBessel's correction: In short, the sample covariance relies on the difference between each observation and the sample mean, but the sample mean is slightly correlated with each observation since it is defined in terms of all observations. If the population mean${\displaystyle \mathrm{E}(\mathbf{X})}$ is known, the analogous unbiased estimate

${\displaystyle q_{jk}=\frac{1}{N}\sum _{i=1}^N\left(x_{ij}-\mathrm{E}(X_j)\right)\left(x_{ik}-\mathrm{E}(X_k)\right),}$

using the population mean, has${\displaystyle N}$ in the denominator. This is an example of why in probability and statistics it is essential to distinguish betweenrandom variables (upper case letters) andrealizations of the random variables (lower case letters).

Themaximum likelihoodestimate of the covariance

${\displaystyle q_{jk}=\frac{1}{N}\sum _{i=1}^N\left(x_{ij}-{\overline{x}}_j\right)\left(x_{ik}-{\overline{x}}_k\right)}$

for theGaussian distribution case hasN in the denominator as well. The ratio of 1/N to 1/(N − 1) approaches 1 for large N, so the maximum likelihood estimate approximately equals the unbiased estimate when the sample is large.

For each random variable, the sample mean is a goodestimator of the population mean, where a "good" estimator is defined as beingefficient and unbiased. Of course the estimator will likely not be the true value of thepopulation mean since different samples drawn from the same distribution will give different sample means and hence different estimates of the true mean. Thus the sample mean is arandom variable, not a constant, and consequently has its own distribution.

Denoting withμ the population mean and with${\displaystyle {\sigma }^2}$ the population variance, for a random sample ofnindependent observations drawn from the population, the expected value of the sample mean is

${\displaystyle \mathrm{E}(\overline{x})=\mu }$

and thevariance of the sample mean is

${\displaystyle \mathrm{var}(\overline{x})=\frac{{\sigma }^2}{n}.}$

If the samples are not independent, butcorrelated, then special care has to be taken in order to avoid the problem ofpseudoreplication.

If the population isnormally distributed, then the sample mean is normally distributed as follows:

${\displaystyle \overline{x}\sim N\left\{\mu ,\frac{{\sigma }^2}{n}\right\}.}$

If the population is not normally distributed, the sample mean is nonetheless approximately normally distributed ifn is large and σ²/n < +∞. This is a consequence of thecentral limit theorem.

In a weighted sample, each vector${\displaystyle {\mathbf{\text{x}}}_i}$ (each set of single observations on each of theK random variables) is assigned a weight${\displaystyle w_i\ge 0}$.Without loss of generality, assume that the weights arenormalized:

${\displaystyle \sum _{i=1}^N{w}_i=1.}$

(If they are not, divide the weights by their sum).Then theweighted mean vector${\displaystyle \overline{\mathbf{x}}}$ is given by

${\displaystyle \overline{\mathbf{x}}=\sum _{i=1}^N{w}_i{\mathbf{x}}_i.}$

and the elements${\displaystyle q_{jk}}$ of the weighted covariance matrix${\displaystyle \mathbf{Q}}$ are^[2]

${\displaystyle q_{jk}=\frac{1}{1-\sum _{i=1}^N{w}_i^2}\sum _{i=1}^N{w}_i\left(x_{ij}-{\overline{x}}_j\right)\left(x_{ik}-{\overline{x}}_k\right).}$

If all weights are the same,${\displaystyle w_i=1/N}$, the weighted mean and covariance reduce to the (biased) sample mean and covariance mentioned above.

The sample mean and sample covariance are notrobust statistics, meaning that they are sensitive tooutliers. As robustness is often a desired trait, particularly in real-world applications, robust alternatives may prove desirable, notablyquantile-based statistics such as thesample median for location,^[3] andinterquartile range (IQR) for dispersion. Other alternatives includetrimming andWinsorising, as in thetrimmed mean and theWinsorized mean.

• Estimation of covariance matrices
• Scatter matrix
• Unbiased estimation of standard deviation

• Covariance and correlation
• Estimation methods
• Summary statistics
• Matrices (mathematics)
• U-statistics

• Webarchive template wayback links
• Articles with short description
• Short description matches Wikidata
• Wikipedia articles that are too technical from June 2014
• All articles that are too technical