This article has multiple issues. Please helpimprove it or discuss these issues on thetalk page.(Learn how and when to remove these messages) This article includes a list ofgeneral references, butit lacks sufficient correspondinginline citations. Please help toimprove this article byintroducing more precise citations.(February 2012) (Learn how and when to remove this message) This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Statistical distance" – news ·newspapers ·books ·scholar ·JSTOR(December 2020) (Learn how and when to remove this message) (Learn how and when to remove this message)

Instatistics,probability theory, andinformation theory, astatistical distance quantifies thedistance between two statistical objects, which can be tworandom variables, or twoprobability distributions orsamples, or the distance can be between an individual sample point and a population or a wider sample of points.

A distance between populations can be interpreted as measuring the distance between twoprobability distributions and hence they are essentially measures of distances betweenprobability measures. Where statistical distance measures relate to the differences betweenrandom variables, these may havestatistical dependence,^[1] and hence these distances are not directly related to measures of distances between probability measures. Again, a measure of distance between random variables may relate to the extent of dependence between them, rather than to their individual values.

Many statistical distance measures are notmetrics, and some are not symmetric. Some types of distance measures, which generalizesquared distance, are referred to as (statistical)divergences.

Many terms are used to refer to various notions of distance; these are often confusingly similar, and may be used inconsistently between authors and over time, either loosely or with precise technical meaning. In addition to "distance", similar terms includedeviance,deviation,discrepancy, discrimination, anddivergence, as well as others such ascontrast function andmetric. Terms frominformation theory includecross entropy,relative entropy,discrimination information, andinformation gain.

Ametric on a setX is afunction (called thedistance function or simplydistance)d :X ×X →R⁺(whereR⁺ is the set of non-negativereal numbers). For allx,y,z inX, this function is required to satisfy the following conditions:

1. d(x,y) ≥ 0     (non-negativity)
2. d(x,y) = 0   if and only if  x =y     (identity of indiscernibles. Note that condition 1 and 2 together producepositive definiteness)
3. d(x,y) =d(y,x)     (symmetry)
4. d(x,z) ≤d(x,y) +d(y,z)     (subadditivity /triangle inequality).

Many statistical distances are notmetrics, because they lack one or more properties of proper metrics. For example,pseudometrics violate property (2), identity of indiscernibles;quasimetrics violate property (3), symmetry; andsemimetrics violate property (4), the triangle inequality. Statistical distances that satisfy (1) and (2) are referred to asdivergences.

The total variation distance of two distributions${\displaystyle X}$ and${\displaystyle Y}$ over a finite domain${\displaystyle D}$, (often referred to asstatistical difference^[2]orstatistical distance^[3] in cryptography) is defined as

${\displaystyle \mathrm{\Delta }(X,Y)=\frac{1}{2}\sum _{\alpha \in D}|Pr[X=\alpha ]-Pr[Y=\alpha ]|}$.

We say that twoprobability ensembles${\displaystyle \{X_k{\}}_{k\in \mathbb{N}}}$ and${\displaystyle \{Y_k{\}}_{k\in \mathbb{N}}}$ are statistically close if${\displaystyle \mathrm{\Delta }(X_k,Y_k)}$ is anegligible function in${\displaystyle k}$.

• Total variation distance (sometimes just called "the" statistical distance)
• Hellinger distance
• Lévy–Prokhorov metric
• Wasserstein metric: also known as the Kantorovich metric, orearth mover's distance
• Mahalanobis distance
• Integral probability metrics generalize several metrics or pseudometrics on distributions

• Kullback–Leibler divergence
• Rényi divergence
• Jensen–Shannon divergence
• Ball divergence
• Bhattacharyya distance (despite its name it is not a distance, as it violates the triangle inequality)
• f-divergence: generalizes several distances and divergences
• Discriminability index, specifically theBayes discriminability index, is a positive-definite symmetric measure of the overlap of two distributions.

• Probabilistic metric space
• Randomness extractor
• Similarity measure
• Zero-knowledge proof

• Distance and Similarity Measures (Wolfram Alpha)

• Dodge, Y. (2003)Oxford Dictionary of Statistical Terms, OUP.ISBN 0-19-920613-9

• Statistical distance

• Articles with short description
• Short description is different from Wikidata
• Articles lacking in-text citations from February 2012
• All articles lacking in-text citations
• Articles needing additional references from December 2020
• All articles needing additional references
• Articles with multiple maintenance issues