In statistics,efficiency is a measure of quality of anestimator, of an experimental design,^[1] or of ahypothesis testing procedure.^[2] Essentially, a more efficient estimator needs fewer input data or observations than a less efficient one to achieve theCramér–Rao bound. Anefficient estimator is characterized by having the smallest possiblevariance, indicating that there is a smalldeviance between the estimated value and the "true" value in theL2 norm sense.^[1]

Therelative efficiency of two procedures is the ratio of their efficiencies, although often this concept is used where the comparison is made between a given procedure and a notional "best possible" procedure. The efficiencies and the relative efficiency of two procedures theoretically depend on the sample size available for the given procedure, but it is often possible to use theasymptotic relative efficiency (defined as the limit of the relative efficiencies as the sample size grows) as the principal comparison measure.

The efficiency of anunbiasedestimator,T, of aparameterθ is defined as^[3]

${\displaystyle e(T)=\frac{1/\mathcal{I}(\theta )}{\mathrm{var}(T)}}$

where${\displaystyle \mathcal{I}(\theta )}$ is theFisher information of the sample. Thuse(T) is the minimum possible variance for an unbiased estimator divided by its actual variance. TheCramér–Rao bound can be used to prove thate(T) ≤ 1.

Anefficient estimator is anestimator that estimates the quantity of interest in some “best possible” manner. The notion of “best possible” relies upon the choice of a particularloss function — the function which quantifies the relative degree of undesirability of estimation errors of different magnitudes. The most common choice of the loss function isquadratic, resulting in themean squared error criterion of optimality.^[4]

In general, the spread of an estimator around the parameter θ is a measure of estimator efficiency and performance. This performance can be calculated by finding the mean squared error. More formally, letT be an estimator for the parameterθ. The mean squared error ofT is the value${\displaystyle \mathrm{MSE}(T)=E[(T-\theta {)}^2]}$, which can be decomposed as a sum of its variance and bias:

An estimatorT₁ performs better than an estimatorT₂ if.^[5] For a more specific case, ifT₁ andT₂are two unbiased estimators for the same parameter θ, then the variance can be compared to determine performance. In this case,T₂ ismore efficient thanT₁ if the variance ofT₂ issmaller than the variance ofT₁, i.e.${\displaystyle \mathrm{var}(T_1)>\mathrm{var}(T_2)}$ for all values ofθ. This relationship can be determined by simplifying the more general case above for mean squared error; since the expected value of an unbiased estimator is equal to the parameter value,${\displaystyle \mathrm{E}[T]=\theta }$. Therefore, for an unbiased estimator,${\displaystyle \mathrm{MSE}(T)=\mathrm{var}(T)}$, as the${\displaystyle (\mathrm{E}[T]-\theta {)}^2}$ term drops out for being equal to 0.^[5]

If anunbiasedestimator of a parameterθ attains${\displaystyle e(T)=1}$ for all values of the parameter, then the estimator is called efficient.^[3]

Equivalently, the estimator achieves equality in theCramér–Rao inequality for allθ. TheCramér–Rao lower bound is a lower bound of the variance of an unbiased estimator, representing the "best" an unbiased estimator can be.

An efficient estimator is also theminimum variance unbiased estimator (MVUE). This is because an efficient estimator maintains equality on the Cramér–Rao inequality for all parameter values, which means it attains the minimum variance for all parameters (the definition of the MVUE). The MVUE estimator, even if it exists, is not necessarily efficient, because "minimum" does not mean equality holds on the Cramér–Rao inequality.

Thus an efficient estimator need not exist, but if it does, it is the MVUE.

Suppose{P_θ |θ ∈ Θ} is aparametric model andX = (X₁, …,X_n) are the data sampled from this model. LetT =T(X) be anestimator for the parameterθ. If this estimator isunbiased (that is,E[ T ] =θ), then theCramér–Rao inequality states thevariance of this estimator is bounded from below:

${\displaystyle \mathrm{var}[T]\ge {\mathcal{I}}_{\theta }^{-1},}$

where${\displaystyle {\mathcal{I}}_{\theta }}$ is theFisher information matrix of the model at pointθ. Generally, the variance measures the degree of dispersion of a random variable around its mean. Thus estimators with small variances are more concentrated, they estimate the parameters more precisely. We say that the estimator is afinite-sample efficient estimator (in the class of unbiased estimators) if it reaches the lower bound in the Cramér–Rao inequality above, for allθ ∈ Θ. Efficient estimators are alwaysminimum variance unbiased estimators. However the converse is false: There exist point-estimation problems for which the minimum-variance mean-unbiased estimator is inefficient.^[6]

Historically, finite-sample efficiency was an early optimality criterion. However this criterion has some limitations:

• Finite-sample efficient estimators are extremely rare. In fact, it was proved that efficient estimation is possible only in anexponential family, and only for the natural parameters of that family.^[7]
• This notion of efficiency is sometimes restricted to the class ofunbiased estimators. (Often it is not.^[8]) Since there are no good theoretical reasons to require that estimators are unbiased, this restriction is inconvenient. In fact, if we usemean squared error as a selection criterion, many biased estimators will slightly outperform the “best” unbiased ones. For example, inmultivariate statistics for dimension three or more, the mean-unbiased estimator,sample mean, isinadmissible: Regardless of the outcome, its performance is worse than for example theJames–Stein estimator.^{[citation needed]}
• Finite-sample efficiency is based on the variance, as a criterion according to which the estimators are judged. A more general approach is to useloss functions other than quadratic ones, in which case the finite-sample efficiency can no longer be formulated.^{[citation needed][dubious –discuss]}

As an example, among the models encountered in practice, efficient estimators exist for: the meanμ of thenormal distribution (but not the varianceσ²), parameterλ of thePoisson distribution, the probabilityp in thebinomial ormultinomial distribution.

Consider the model of anormal distribution with unknown mean but known variance:{P_θ =N(θ,σ²) |θ ∈R }. The data consists ofnindependent and identically distributed observations from this model:X = (x₁, …,x_n). We estimate the parameterθ using thesample mean of all observations:

${\displaystyle T(X)=\frac{1}{n}\sum _{i=1}^n{x}_i.}$

This estimator has meanθ and variance ofσ² / n, which is equal to the reciprocal of theFisher information from the sample. Thus, the sample mean is a finite-sample efficient estimator for the mean of the normal distribution.

Asymptotic efficiency requiresConsistency (statistics), asymptotically normal distribution of the estimator, and an asymptotic variance-covariance matrix no worse than that of any other estimator.^[9]

Consider a sample of size${\displaystyle N}$ drawn from anormal distribution of mean${\displaystyle \mu }$ and unitvariance, i.e.,${\displaystyle X_n\sim \mathcal{N}(\mu ,1).}$

Thesample mean,${\displaystyle \overline{X}}$, of the sample${\displaystyle X_1,X_2,\dots ,X_N}$, defined as

${\displaystyle \overline{X}=\frac{1}{N}\sum _{n=1}^N{X}_n\sim \mathcal{N}\left(\mu ,\frac{1}{N}\right).}$

The variance of the mean, 1/N (the square of thestandard error) is equal to the reciprocal of theFisher information from the sample and thus, by theCramér–Rao inequality, the sample mean is efficient in the sense that its efficiency is unity (100%).

Now consider thesample median,${\displaystyle \tilde{X}}$. This is anunbiased andconsistent estimator for${\displaystyle \mu }$. For large${\displaystyle N}$ the sample median is approximatelynormally distributed with mean${\displaystyle \mu }$ and variance${\displaystyle \pi /2N,}$^[10]

${\displaystyle \tilde{X}\sim \mathcal{N}\left(\mu ,\frac{\pi }{2N}\right).}$

The efficiency of the median for large${\displaystyle N}$ is thus

${\displaystyle e\left(\tilde{X}\right)=\left(\frac{1}{N}\right){\left(\frac{\pi }{2N}\right)}^{-1}=2/\pi \approx \mathrm{0.64.}}$

In other words, the relative variance of the median will be${\displaystyle \pi /2\approx 1.57}$, or 57% greater than the variance of the mean – the standard error of the median will be 25% greater than that of the mean.^[11]

Note that this is theasymptotic efficiency — that is, the efficiency in the limit as sample size${\displaystyle N}$ tends to infinity. For finite values of${\displaystyle N,}$ the efficiency is higher than this (for example, a sample size of 3 gives an efficiency of about 74%).^{[citation needed]}

The sample mean is thus more efficient than the sample median in this example. However, there may be measures by which the median performs better. For example, the median is far more robust tooutliers, so that if the Gaussian model is questionable or approximate, there may advantages to using the median (seeRobust statistics).

If${\displaystyle T_1}$ and${\displaystyle T_2}$ are estimators for the parameter${\displaystyle \theta }$, then${\displaystyle T_1}$ is said todominate${\displaystyle T_2}$ if:

1. itsmean squared error (MSE) is smaller for at least some value of${\displaystyle \theta }$
2. the MSE does not exceed that of${\displaystyle T_2}$ for any value of θ.

Formally,${\displaystyle T_1}$ dominates${\displaystyle T_2}$ if

${\displaystyle \mathrm{E}[(T_1-\theta {)}^2]\le \mathrm{E}[(T_2-\theta {)}^2]}$

holds for all${\displaystyle \theta }$, with strict inequality holding somewhere.

The relative efficiency of two unbiased estimators is defined as^[12]

${\displaystyle e(T_1,T_2)=\frac{\mathrm{E}[(T_2-\theta {)}^2]}{\mathrm{E}[(T_1-\theta {)}^2]}=\frac{\mathrm{var}(T_2)}{\mathrm{var}(T_1)}}$

Although${\displaystyle e}$ is in general a function of${\displaystyle \theta }$, in many cases the dependence drops out; if this is so,${\displaystyle e}$ being greater than one would indicate that${\displaystyle T_1}$ is preferable, regardless of the true value of${\displaystyle \theta }$.

An alternative to relative efficiency for comparing estimators, is thePitman closeness criterion. This replaces the comparison of mean-squared-errors with comparing how often one estimator produces estimates closer to the true value than another estimator.

In estimating the mean of uncorrelated, identically distributed variables we can take advantage of the fact thatthe variance of the sum is the sum of the variances. In this case efficiency can be defined as the square of thecoefficient of variation, i.e.,^[13]

${\displaystyle e\equiv {\left(\frac{\sigma }{\mu }\right)}^2}$

Relative efficiency of two such estimators can thus be interpreted as the relative sample size of one required to achieve the certainty of the other. Proof:

${\displaystyle \frac{e_1}{e_2}=\frac{s_1^2}{s_2^2}.}$

Now because${\displaystyle s_1^2=n_1{\sigma }^2,s_2^2=n_2{\sigma }^2}$ we have${\displaystyle \frac{e_1}{e_2}=\frac{n_1}{n_2}}$, so the relative efficiency expresses the relative sample size of the first estimator needed to match the variance of the second.

Efficiency of an estimator may change significantly if the distribution changes, often dropping. This is one of the motivations ofrobust statistics – an estimator such as the sample mean is an efficient estimator of the population mean of a normal distribution, for example, but can be an inefficient estimator of amixture distribution of two normal distributions with the same mean and different variances. For example, if a distribution is a combination of 98%N(μ,σ) and 2%N(μ, 10σ), the presence of extreme values from the latter distribution (often "contaminating outliers") significantly reduces the efficiency of the sample mean as an estimator ofμ. By contrast, thetrimmed mean is less efficient for a normal distribution, but is more robust (i.e., less affected) by changes in the distribution, and thus may be more efficient for a mixture distribution. Similarly, theshape of a distribution, such asskewness orheavy tails, can significantly reduce the efficiency of estimators that assume a symmetric distribution or thin tails.

Efficiency in statistics is important because it allows the performance of various estimators to be compared. Although an unbiased estimator is usually favored over a biased one, a more efficient biased estimator can sometimes be more valuable than a less efficient unbiased estimator. For example, this can occur when the values of the biased estimator gathers around a number closer to the true value. Thus, estimator performance can be predicted easily by comparing their mean squared errors or variances.

While efficiency is a desirable quality of an estimator, it must be weighed against other considerations, and an estimator that is efficient for certain distributions may well be inefficient for other distributions. Most significantly, estimators that are efficient for clean data from a simple distribution, such as the normal distribution (which is symmetric, unimodal, and has thin tails) may not be robust to contamination by outliers, and may be inefficient for more complicated distributions. Inrobust statistics, more importance is placed on robustness and applicability to a wide variety of distributions, rather than efficiency on a single distribution.M-estimators are a general class of estimators motivated by these concerns. They can be designed to yield both robustness and high relative efficiency, though possibly lower efficiency than traditional estimators for some cases. They can be very computationally complicated, however.

A more traditional alternative areL-estimators, which are very simple statistics that are easy to compute and interpret, in many cases robust, and often sufficiently efficient for initial estimates. Seeapplications of L-estimators for further discussion. Inefficient statistics in this sense are discussed in detail inThe Atomic Nucleus by R. D. Evans, written before the advent of computers, when efficiently estimating even the arithmetic mean of a sorted series of measurements was laborious.^[14]

For comparingsignificance tests, a meaningful measure of efficiency can be defined based on the sample size required for the test to achieve a given taskpower.^[15]

Pitman efficiency^[16] andBahadur efficiency (orHodges–Lehmann efficiency)^[17][18][19] relate to the comparison of the performance ofstatistical hypothesis testing procedures.

For experimental designs, efficiency relates to the ability of a design to achieve the objective of the study with minimal expenditure of resources such as time and money. In simple cases, the relative efficiency of designs can be expressed as the ratio of the sample sizes required to achieve a given objective.^[20]

• Bayes estimator
• Consistent estimator
• Hodges' estimator
• Optimal instruments

• Lehmann, E.L.; Casella, G. (1998).Theory of Point Estimation (2nd ed.). Springer.ISBN 0-387-98502-6.
• Pfanzagl, Johann; with the assistance of R. Hamböker (1994).Parametric Statistical Theory. Berlin: Walter de Gruyter.ISBN 3-11-013863-8.MR 1291393.

• Estimation theory

• CS1 maint: location missing publisher
• Articles with short description
• Short description matches Wikidata
• All articles with unsourced statements
• Articles with unsourced statements from December 2011
• Articles with unsourced statements from February 2012
• All accuracy disputes
• Articles with disputed statements from February 2012
• Articles with unsourced statements from April 2013