This article includes a list ofgeneral references, butit lacks sufficient correspondinginline citations. Please help toimprove this article byintroducing more precise citations.(December 2010) (Learn how and when to remove this message)

Instatistics, thekthorder statistic of astatistical sample is equal to itskth-smallest value.^[1] Together withrank statistics, order statistics are among the most fundamental tools innon-parametric statistics andinference.

Important special cases of the order statistics are theminimum andmaximum value of a sample, and (with some qualifications discussed below) thesample median and othersample quantiles.

When usingprobability theory to analyze order statistics ofrandom samples from acontinuous distribution, thecumulative distribution function is used to reduce the analysis to the case of order statistics of theuniform distribution.

For example, suppose that four numbers are observed or recorded, resulting in a sample of size 4. If the sample values are

6, 9, 3, 7,

the order statistics would be denoted

where the subscript(i) enclosed in parentheses indicates theith order statistic of the sample.

Thefirst order statistic (orsmallest order statistic) is always theminimum of the sample, that is,

$${\displaystyle X_{(1)}=min\{X_1,\dots ,X_n\}}$$

where, following a common convention, we use upper-case letters to refer to random variables, and lower-case letters (as above) to refer to their actual observed values.

Similarly, for a sample of sizen, thenth order statistic (orlargest order statistic) is themaximum, that is,

$${\displaystyle X_{(n)}=max\{X_1,\dots ,X_n\}.}$$

Thesample range is the difference between the maximum and minimum. It is a function of the order statistics:

$${\displaystyle \mathrm{R}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\{X_1,\dots ,X_n\}=X_{(n)}-X_{(1)}.}$$

A similar important statistic inexploratory data analysis that is simply related to the order statistics is the sampleinterquartile range.

The sample median may or may not be an order statistic, since there is a single middle value only when the numbern of observations isodd. More precisely, ifn = 2m+1 for some integerm, then the sample median is${\displaystyle X_{(m+1)}}$ and so is an order statistic. On the other hand, whenn iseven,n = 2m and there are two middle values,${\displaystyle X_{(m)}}$ and${\displaystyle X_{(m+1)}}$, and the sample median is some function of the two (usually the average) and hence not an order statistic. Similar remarks apply to all sample quantiles.

Given any random variablesX₁,X₂, ...,X_n, the order statistics X₍₁₎, X₍₂₎, ..., X_(n) are also random variables, defined by sorting the values (realizations) ofX₁, ...,X_n in increasing order.

When the random variablesX₁,X₂, ...,X_n form asample they areindependent and identically distributed. This is the case treated below. In general, the random variablesX₁, ...,X_n can arise by sampling from more than one population. Then they areindependent, but not necessarily identically distributed, and theirjoint probability distribution is given by theBapat–Beg theorem.

From now on, we will assume that the random variables under consideration arecontinuous and, where convenient, we will also assume that they have aprobability density function (PDF), that is, they areabsolutely continuous. The peculiarities of the analysis of distributions assigning mass to points (in particular,discrete distributions) are discussed at the end.

For a random sample as above, with cumulative distribution${\displaystyle F_X(x)}$, the order statistics for that sample have cumulative distributions as follows^[2](wherer specifies which order statistic):$${\displaystyle F_{X_{(r)}}(x)=\sum _{j=r}^n(\genfrac{}{}{0ex}{}{n}{j}){\left[F_X(x)\right]}^j{\left[1-F_X(x)\right]}^{n-j}}$$The proof of this formula is purecombinatorics: for the${\displaystyle r}$th order statistic to be${\displaystyle \le x}$, the number of samples that are${\displaystyle >x}$ has to be between${\displaystyle 0}$ and${\displaystyle n-r}$. In the case that${\displaystyle X_{(j)}}$ is the largest order statistic${\displaystyle \le x}$, there has to be${\displaystyle j}$ samples${\displaystyle \le x}$ (each with an independent probability of${\displaystyle F_X(x)}$) and${\displaystyle n-j}$ samples${\displaystyle >x}$ (each with an independent probability of${\displaystyle 1-F_X(x)}$). Finally there are$(\genfrac{}{}{0ex}{}{n}{j})$ different ways of choosing which of the${\displaystyle n}$ samples are of the${\displaystyle \le x}$ kind.

The corresponding probability density function may be derived from this result, and is found to be

$${\displaystyle f_{X_{(r)}}(x)=\frac{n!}{(r-1)!(n-r)!}{f}_X(x){\left[F_X(x)\right]}^{r-1}{\left[1-F_X(x)\right]}^{n-r}.}$$

Moreover, there are two special cases, which have CDFs that are easy to compute.

$${\displaystyle F_{X_{(n)}}(x)=Pr(max\{X_1,\dots ,X_n\}\le x)=[F_X(x){]}^n}$$

$${\displaystyle F_{X_{(1)}}(x)=Pr(min\{X_1,\dots ,X_n\}\le x)=1-[1-F_X(x){]}^n}$$

Which can be derived by careful consideration of probabilities.

In this section we show that the order statistics of theuniform distribution on theunit interval havemarginal distributions belonging to thebeta distribution family. We also give a simple method to derive the joint distribution of any number of order statistics, and finally translate these results to arbitrary continuous distributions using thecdf.

We assume throughout this section that${\displaystyle X_1,X_2,\dots ,X_n}$ is arandom sample drawn from a continuous distribution with cdf${\displaystyle F_X}$. Denoting${\displaystyle U_i=F_X(X_i)}$ we obtain the corresponding random sample${\displaystyle U_1,\dots ,U_n}$ from the standarduniform distribution. Note that the order statistics also satisfy${\displaystyle U_{(i)}=F_X(X_{(i)})}$.

The probability density function of the order statistic${\displaystyle U_{(k)}}$ is equal to^[3]

$${\displaystyle f_{U_{(k)}}(u)=\frac{n!}{(k-1)!(n-k)!}{u}^{k-1}(1-u{)}^{n-k}}$$

that is, thekth order statistic of the uniform distribution is abeta-distributed random variable.^[3][4]

$${\displaystyle U_{(k)}\sim \mathrm{Beta}(k,n+1\mathbf{-}k).}$$

The proof of these statements is as follows. For${\displaystyle U_{(k)}}$ to be betweenu andu + du, it is necessary that exactlyk − 1 elements of the sample are smaller thanu, and that at least one is betweenu andu + du. The probability that more than one is in this latter interval is already${\displaystyle O(d{u}^2)}$, so we have to calculate the probability that exactlyk − 1, 1 andn − k observations fall in the intervals${\displaystyle (0,u)}$,${\displaystyle (u,u+du)}$ and${\displaystyle (u+du,1)}$ respectively. This equals (refer tomultinomial distribution for details)

$${\displaystyle \frac{n!}{(k-1)!(n-k)!}{u}^{k-1}\cdot du\cdot (1-u-du{)}^{n-k}}$$

and the result follows.

The mean of this distribution isk / (n + 1).

Similarly, fori < j, thejoint probability density function of the two order statisticsU_(i) < U_(j) can be shown to be

$${\displaystyle f_{U_{(i)},U_{(j)}}(u,v)=n!\frac{u^{i-1}}{(i-1)!}\frac{(v-u{)}^{j-i-1}}{(j-i-1)!}\frac{(1-v{)}^{n-j}}{(n-j)!}}$$

which is (up to terms of higher order than${\displaystyle O(dudv)}$) the probability thati − 1, 1,j − 1 − i, 1 andn − j sample elements fall in the intervals${\displaystyle (0,u)}$,${\displaystyle (u,u+du)}$,${\displaystyle (u+du,v)}$,${\displaystyle (v,v+dv)}$,${\displaystyle (v+dv,1)}$ respectively.

One reasons in an entirely analogous way to derive the higher-order joint distributions. Perhaps surprisingly, the joint density of then order statistics turns out to beconstant:

$${\displaystyle f_{U_{(1)},U_{(2)},\dots ,U_{(n)}}(u_1,u_2,\dots ,u_n)=n!.}$$

One way to understand this is that the unordered sample does have constant density equal to 1, and that there aren! different permutations of the sample corresponding to the same sequence of order statistics. This is related to the fact that 1/n! is the volume of the region. It is also related with another particularity of order statistics of uniform random variables: It follows from theBRS-inequality that the maximum expected number of uniform U(0,1] random variables one can choose from a sample of size n with a sum up not exceeding is bounded above by${\displaystyle \sqrt{2sn}}$, which is thus invariant on the set of all${\displaystyle s,n}$ with constant product${\displaystyle sn}$.

Using the above formulas, one can derive the distribution of the range of the order statistics, that is the distribution of${\displaystyle U_{(n)}-U_{(1)}}$, i.e. maximum minus the minimum. More generally, for${\displaystyle n\ge k>j\ge 1}$,${\displaystyle U_{(k)}-U_{(j)}}$ also has a beta distribution:$${\displaystyle U_{(k)}-U_{(j)}\sim \mathrm{Beta}(k-j,n-(k-j)+1)}$$From these formulas we can derive the covariance between two order statistics:$${\displaystyle \mathrm{Cov}(U_{(k)},U_{(j)})=\frac{j(n-k+1)}{(n+1{)}^2(n+2)}}$$The formula follows from noting thatand comparing that with$${\displaystyle \mathrm{Var}(U)=\frac{(k-j)(n-(k-j)+1)}{(n+1{)}^2(n+2)}}$$where${\displaystyle U\sim \mathrm{Beta}(k-j,n-(k-j)+1)}$, which is the actual distribution of the difference.

For${\displaystyle X_1,X_2,..,X_n}$ a random sample of sizen from anexponential distribution with parameterλ, the order statisticsX_(i) fori = 1,2,3, ...,n each have distribution

$${\displaystyle X_{(i)}\stackrel{d}{=}\frac{1}{\lambda }\left(\sum _{j=1}^i\frac{Z_j}{n-j+1}\right)}$$

where theZ_j are i.i.d. standard exponential random variables (i.e. with rate parameter 1). This result was first published byAlfréd Rényi.^[5][6]

TheLaplace transform of order statistics may be sampled from anErlang distribution via a path counting method^{[clarification needed]}.^[7]

IfF_X isabsolutely continuous, it has a density such that${\displaystyle d{F}_X(x)=f_X(x)dx}$, and we can use the substitutions

$${\displaystyle u=F_X(x)}$$

and

$${\displaystyle du=f_X(x)dx}$$

to derive the following probability density functions for the order statistics of a sample of sizen drawn from the distribution ofX:

$${\displaystyle f_{X_{(k)}}(x)=\frac{n!}{(k-1)!(n-k)!}[F_X(x){]}^{k-1}[1-F_X(x){]}^{n-k}{f}_X(x)}$$

$${\displaystyle f_{X_{(j)},X_{(k)}}(x,y)=\frac{n!}{(j-1)!(k-j-1)!(n-k)!}[F_X(x){]}^{j-1}[F_X(y)-F_X(x){]}^{k-1-j}[1-F_X(y){]}^{n-k}{f}_X(x)f_X(y)}$$ where${\displaystyle x\le y}$

$${\displaystyle f_{X_{(1)},\dots ,X_{(n)}}(x_1,\dots ,x_n)=n!f_X(x_1)\cdots f_X(x_n)}$$ where${\displaystyle x_1\le x_2\le \cdots \le x_n.}$

An interesting question is how well the order statistics perform as estimators of thequantiles of the underlying distribution.

The simplest case to consider is how well the sample median estimates the population median.

As an example, consider a random sample of size 6. In that case, the sample median is usually defined as the midpoint of the interval delimited by the 3rd and 4th order statistics. However, we know from the preceding discussion that the probability that this interval actually contains the population median is^{[clarification needed]}

$${\displaystyle (\genfrac{}{}{0ex}{}{6}{3})(1/2{)}^6=\frac{5}{16}\approx 31\mathrm{\%}.}$$

Although the sample median is probably among the best distribution-independentpoint estimates of the population median, what this example illustrates is that it is not a particularly good one in absolute terms. In this particular case, a better confidence interval for the median is the one delimited by the 2nd and 5th order statistics, which contains the population median with probability

$${\displaystyle \left[(\genfrac{}{}{0ex}{}{6}{2})+(\genfrac{}{}{0ex}{}{6}{3})+(\genfrac{}{}{0ex}{}{6}{4})\right](1/2{)}^6=\frac{25}{32}\approx 78\mathrm{\%}.}$$

With such a small sample size, if one wants at least 95% confidence, one is reduced to saying that the median is between the minimum and the maximum of the 6 observations with probability 31/32 or approximately 97%. Size 6 is, in fact, the smallest sample size such that the interval determined by the minimum and the maximum is at least a 95% confidence interval for the population median.

For the uniform distribution, asn tends to infinity, thep^th sample quantile is asymptoticallynormally distributed, since it is approximated by

$${\displaystyle U_{(\lceil np\rceil )}\sim AN\left(p,\frac{p(1-p)}{n}\right).}$$

For a general distributionF with a continuous non-zero density atF ⁻¹(p), a similar asymptotic normality applies:

$${\displaystyle X_{(\lceil np\rceil )}\sim AN\left(F^{-1}(p),\frac{p(1-p)}{n[f(F^{-1}(p)){]}^2}\right)}$$

wheref is thedensity function, andF ⁻¹ is thequantile function associated withF. One of the first people to mention and prove this result wasFrederick Mosteller in his seminal paper in 1946.^[8] Further research led in the 1960s to theBahadur representation which provides information about the errorbounds. The convergence to normal distribution also holds in a stronger sense, such as convergence inrelative entropy or KL divergence.^[9]

An interesting observation can be made in the case where the distribution is symmetric, and the population median equals the population mean. In this case, thesample mean, by thecentral limit theorem, is also asymptotically normally distributed, but with variance σ²/n instead. This asymptotic analysis suggests that the mean outperforms the median in cases of lowkurtosis, and vice versa. For example, the median achieves better confidence intervals for theLaplace distribution, while the mean performs better forX that are normally distributed.

It can be shown that

$${\displaystyle B(k,n+1-k)\stackrel{\mathrm{d}}{=}\frac{X}{X+Y},}$$

where

$${\displaystyle X=\sum _{i=1}^k{Z}_i,Y=\sum _{i=k+1}^{n+1}{Z}_i,}$$

withZ_i being independent identically distributedexponential random variables with rate 1. SinceX/n andY/n are asymptotically normally distributed by the CLT, our results follow by application of thedelta method.

Themutual information andf-divergence between order statistics have also been considered.^[10] For example, if the parent distribution is continuous, then for all${\displaystyle 1\le r,m\le n}$$${\displaystyle I(X_{(r)};X_{(m)})=I(U_{(r)};U_{(m)}),}$$In other words, mutual information is independent of the parent distribution. For discrete random variables, the equality need not to hold and we only have$${\displaystyle I(X_{(r)};X_{(m)})\le I(U_{(r)};U_{(m)}),}$$

The mutual information between uniform order statistics is given by$${\displaystyle I(U_{(r)};U_{(m)})=T_{m-1}+T_{n-r}-T_{m-r+1}-T_n}$$where$${\displaystyle T_k=\log (k!)-k{H}_k}$$where${\displaystyle H_k}$ is the${\displaystyle k}$-th harmonic number.

Moments of the distribution for the first order statistic can be used to develop a non-parametric density estimator.^[11] Suppose, we want to estimate the density${\displaystyle f_X}$ at the point${\displaystyle x^{\ast }}$. Consider the random variables${\displaystyle Y_i=|X_i-x^{\ast }|}$, which are i.i.d with distribution function${\displaystyle g_Y(y)=f_X(y+x^{\ast })+f_X(x^{\ast }-y)}$. In particular,${\displaystyle f_X(x^{\ast })=\frac{g_Y(0)}{2}}$.

The expected value of the first order statistic${\displaystyle Y_{(1)}}$ given a sample of${\displaystyle N}$ total observations yields,

$${\displaystyle E(Y_{(1)})=\frac{1}{(N+1)g(0)}+\frac{1}{(N+1)(N+2)}{\int }_0^1{Q}^{″}(z){\delta }_{N+1}(z)dz}$$

where${\displaystyle Q}$ is the quantile function associated with the distribution${\displaystyle g_Y}$, and${\displaystyle {\delta }_N(z)=(N+1)(1-z{)}^N}$. This equation in combination with ajackknifing technique becomes the basis for the following density estimation algorithm,

  Input: A sample of${\displaystyle N}$ observations.${\displaystyle \{x_{\ell }{\}}_{\ell =1}^M}$ points of density evaluation. Tuning parameter${\displaystyle a\in (0,1)}$ (usually 1/3).  Output:${\displaystyle \{{\hat{f}}_{\ell }{\}}_{\ell =1}^M}$ estimated density at the points of evaluation.

  1: Set${\displaystyle m_N=\mathrm{round}(N^{1-a})}$  2: Set${\displaystyle s_N=\frac{N}{m_N}}$  3: Create an${\displaystyle s_N\times m_N}$ matrix${\displaystyle M_{ij}}$ which holds${\displaystyle m_N}$ subsets with${\displaystyle s_N}$ observations each.  4: Create a vector${\displaystyle \hat{f}}$ to hold the density evaluations.  5:for${\displaystyle \ell =1\to M}$do  6:for${\displaystyle k=1\to m_N}$do  7:         Find the nearest distance${\displaystyle d_{\ell k}}$ to the current point${\displaystyle x_{\ell }}$ within the${\displaystyle k}$th subset  8:end for  9:     Compute the subset average of distances to${\displaystyle x_{\ell }:d_{\ell }=\sum _{k=1}^{m_N}\frac{d_{\ell k}}{m_N}}$ 10:     Compute the density estimate at${\displaystyle x_{\ell }:{\hat{f}}_{\ell }=\frac{1}{2(1+s_N)d_{\ell }}}$ 11:end for 12:return${\displaystyle \hat{f}}$

In contrast to the bandwidth/length based tuning parameters forhistogram andkernel based approaches, the tuning parameter for the order statistic based density estimator is the size of sample subsets. Such an estimator is more robust than histogram and kernel based approaches, for example densities like the Cauchy distribution (which lack finite moments) can be inferred without the need for specialized modifications such asIQR based bandwidths. This is because the first moment of the order statistic always exists if the expected value of the underlying distribution does, but the converse is not necessarily true.^[12]

Suppose${\displaystyle X_1,X_2,\dots ,X_n}$ are i.i.d. random variables from a discrete distribution with cumulative distribution function${\displaystyle F(x)}$ andprobability mass function${\displaystyle f(x)}$. To find the probabilities of the${\displaystyle k^{\text{th}}}$ order statistics, three values are first needed, namely

The cumulative distribution function of the${\displaystyle k^{\text{th}}}$ order statistic can be computed by noting that

Similarly, is given by

Note that the probability mass function of${\displaystyle X_{(k)}}$ is just the difference of these values, that is to say

The problem of computing thekth smallest (or largest) element of a list is called the selection problem and is solved by a selection algorithm. Although this problem is difficult for very large lists, sophisticated selection algorithms have been created that can solve this problem in time proportional to the number of elements in the list, even if the list is totally unordered. If the data is stored in certain specialized data structures, this time can be brought down to O(logn). In many applications all order statistics are required, in which case asorting algorithm can be used and the time taken is O(n logn).

Order statistics have a lot of applications in areas as reliability theory, financial mathematics, survival analysis, epidemiology, sports, quality control, actuarial risk, etc. There is an extensive literature devoted to studies on applications of order statistics in these fields.

For example, a recent application in actuarial risk can be found in,^[13] where some weighted premium principles in terms of record claims and kth record claims are provided.

• Rankit
• Box plot
• BRS-inequality
• Concomitant (statistics)
• Fisher–Tippett distribution
• Bapat–Beg theorem for the order statistics of independent but not necessarily identically distributed random variables
• Bernstein polynomial
• L-estimator – linear combinations of order statistics
• Rank-size distribution
• Selection algorithm

• Sample maximum and minimum
• Quantile
• Percentile
• Decile
• Quartile
• Median
• Mean
• Sample mean and covariance

• Order statistics atPlanetMath. Retrieved Feb 02, 2005
• Weisstein, Eric W."Order Statistic".MathWorld. Retrieved Feb 02, 2005
• C++ sourceDynamic Order Statistics

• Nonparametric statistics
• Summary statistics
• Permutations

• Articles with short description
• Short description matches Wikidata
• Articles lacking in-text citations from December 2010
• All articles lacking in-text citations
• Wikipedia articles needing clarification from February 2019
• Wikipedia articles needing clarification from September 2021