Inmathematics, more specificallymeasure theory, there are various notions of theconvergence of measures. For an intuitive general sense of what is meant byconvergence of measures, consider a sequence of measuresμ_n on a space, sharing a common collection of measurable sets. Such a sequence might represent an attempt to construct 'better and better' approximations to a desired measureμ that is difficult to obtain directly. The meaning of 'better and better' is subject to all the usual caveats for takinglimits; for any error toleranceε > 0 we require there beN sufficiently large forn ≥N to ensure the 'difference' betweenμ_n andμ is smaller thanε. Various notions of convergence specify precisely what the word 'difference' should mean in that description; these notions are not equivalent to one another, and vary in strength.

Three of the most common notions of convergence are described below.

This section attempts to provide a rough intuitive description of three notions of convergence, using terminology developed incalculus courses; this section is necessarily imprecise as well as inexact, and the reader should refer to the formal clarifications in subsequent sections. In particular, the descriptions here do not address the possibility that the measure of some sets could be infinite, or that the underlying space could exhibit pathological behavior, and additional technical assumptions are needed for some of the statements. The statements in this section are however all correct ifμ_n is a sequence of probability measures on aPolish space.

The various notions of convergence formalize the assertion that the 'average value' of each 'sufficiently nice' function should converge:$${\displaystyle \int fd{\mu }_n\to \int fd\mu }$$

To formalize this requires a careful specification of the set of functions under consideration and how uniform the convergence should be.

The notion ofweak convergence requires this convergence to take place for every continuous bounded functionf. This notion treats convergence for different functionsf independently of one another, i.e., different functionsf may require different values ofN ≤n to be approximated equally well (thus, convergence is non-uniform inf).

The notion ofsetwise convergence formalizes the assertion that the measure of each measurable set should converge:$${\displaystyle {\mu }_n(A)\to \mu (A)}$$

Again, no uniformity over the setA is required.Intuitively, considering integrals of 'nice' functions, this notion provides more uniformity than weak convergence. As a matter of fact, when considering sequences of measures with uniformly boundedvariation on aPolish space, setwise convergence implies the convergence$\int fd{\mu }_n\to \int fd\mu$ for any bounded measurable functionf^{[citation needed]}.As before, this convergence is non-uniform inf.

The notion oftotal variation convergence formalizes the assertion that the measure of all measurable sets should convergeuniformly, i.e. for everyε > 0 there existsN such that for everyn >N and for every measurable setA. As before, this implies convergence of integrals against bounded measurable functions, but this time convergence is uniform over all functions bounded by any fixed constant.

This is the strongest notion of convergence shown on this page and is defined as follows. Let${\displaystyle (X,\mathcal{F})}$ be ameasurable space. Thetotal variation distance between two (positive) measuresμ andν is then given by

${\displaystyle {\Vert \mu -\nu \Vert }_{\text{TV}}=\underset{f}{sup}\left\{{\int }_{X}fd\mu -{\int }_{X}fd\nu \right\}.}$

Here the supremum is taken overf ranging over the set of allmeasurable functions fromX to[−1, 1]. This is in contrast, for example, to theWasserstein metric, where the definition isof the same form, but the supremum is taken overf ranging over the set of those measurable functions fromX to[−1, 1] which haveLipschitz constant at most 1; and also in contrast to theRadon metric, where the supremum is taken overf ranging over the set of continuous functions fromX to[−1, 1]. In the case whereX is aPolish space, the total variation metric coincides with the Radon metric.

Ifμ andν are bothprobability measures, then the total variation distance is also given by

${\displaystyle {\Vert \mu -\nu \Vert }_{\text{TV}}=2\cdot \underset{A\in \mathcal{F}}{sup}|\mu (A)-\nu (A)|.}$

The equivalence between these two definitions can be seen as a particular case of theMonge–Kantorovich duality. From the two definitions above, it is clear that the total variation distance between probability measures is always between 0 and 2.

To illustrate the meaning of the total variation distance, consider the following thought experiment. Assume that we are given two probability measuresμ andν, as well as a random variableX. We know thatX has law eitherμ orν but we do not know which one of the two. Assume that these two measures have prior probabilities 0.5 each of being the true law ofX. Assume now that we are givenone single sample distributed according to the law ofX and that we are then asked to guess which one of the two distributions describes that law. The quantity

${\displaystyle \frac{2+\Vert \mu -\nu {\Vert }_{\text{TV}}}{4}}$

then provides a sharp upper bound on the prior probability that our guess will be correct.

Given the above definition of total variation distance, a sequenceμ_n of measures defined on the same measure space is said toconverge to a measureμ in total variation distance if for everyε > 0, there exists anN such that for alln >N, one has that^[1]

For${\displaystyle (X,\mathcal{F})}$ ameasurable space, a sequenceμ_n is said to converge setwise to a limitμ if

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}{\mu }_n(A)=\mu (A)}$

for every set${\displaystyle A\in \mathcal{F}}$.

Typical arrow notations are${\displaystyle {\mu }_n\stackrel{sw}{\to }\mu }$ and${\displaystyle {\mu }_n\stackrel{s}{\to }\mu }$.

For example, as a consequence of theRiemann–Lebesgue lemma, the sequenceμ_n of measures on the interval[−1, 1] given byμ_n(dx) = (1 + sin(nx))dx converges setwise to Lebesgue measure, but it does not converge in total variation.

In a measure theoretical or probabilistic context setwise convergence is often referred to as strong convergence (as opposed to weak convergence). This can lead to some ambiguity because infunctional analysis, strong convergence usually refers to convergence with respect to a norm.

Inmathematics andstatistics,weak convergence is one of many types of convergence relating to the convergence ofmeasures. It depends on a topology on the underlying space and thus is not a purely measure-theoretic notion.

There are several equivalentdefinitions of weak convergence of a sequence of measures, some of which are (apparently) more general than others. The equivalence of these conditions is sometimes known as thePortmanteau theorem.^[2]

Definition. Let${\displaystyle S}$ be ametric space with itsBorel${\displaystyle \sigma }$-algebra${\displaystyle \mathrm{\Sigma }}$. A bounded sequence of positiveprobability measures${\displaystyle P_n(n=1,2,\dots )}$ on${\displaystyle (S,\mathrm{\Sigma })}$ is said toconverge weakly to a probability measure${\displaystyle P}$ (denoted${\displaystyle P_n\Rightarrow P}$) if any of the following equivalent conditions is true (here${\displaystyle {\mathrm{E}}_n}$ denotes expectation or the integral with respect to${\displaystyle P_n}$, while${\displaystyle \mathrm{E}}$ denotes expectation or the integral with respect to${\displaystyle P}$):

• ${\displaystyle {\mathrm{E}}_n[f]\to \mathrm{E}[f]}$ for allbounded,continuous functions${\displaystyle f}$;
• ${\displaystyle {\mathrm{E}}_n[f]\to \mathrm{E}[f]}$ for all bounded andLipschitz functions${\displaystyle f}$;
• ${\displaystyle lim sup{\mathrm{E}}_n[f]\le \mathrm{E}[f]}$ for everyupper semi-continuous function${\displaystyle f}$ bounded from above;
• ${\displaystyle lim inf{\mathrm{E}}_n[f]\ge \mathrm{E}[f]}$ for everylower semi-continuous function${\displaystyle f}$ bounded from below;
• ${\displaystyle lim sup{P}_n(C)\le P(C)}$ for allclosed sets${\displaystyle C}$ of space${\displaystyle S}$;
• ${\displaystyle lim inf{P}_n(U)\ge P(U)}$ for allopen sets${\displaystyle U}$ of space${\displaystyle S}$;
• ${\displaystyle lim{P}_n(A)=P(A)}$ for allcontinuity sets${\displaystyle A}$ of measure${\displaystyle P}$.

In the case${\displaystyle S}$ and${\displaystyle \mathbf{R}}$ (with its usual topology) are homeomorphic , if${\displaystyle F_n}$ and${\displaystyle F}$ denote thecumulative distribution functions of the measures${\displaystyle P_n}$ and${\displaystyle P}$, respectively, then${\displaystyle P_n}$ converges weakly to${\displaystyle P}$ if and only if${\displaystyle \underset{n\to \mathrm{\infty }}{lim}{F}_n(x)=F(x)}$ for all points${\displaystyle x\in \mathbf{R}}$ at which${\displaystyle F}$ is continuous.

For example, the sequence where${\displaystyle P_n}$ is theDirac measure located at${\displaystyle 1/n}$ converges weakly to the Dirac measure located at 0 (if we view these as measures on${\displaystyle \mathbf{R}}$ with the usual topology), but it does not converge setwise. This is intuitively clear: we only know that${\displaystyle 1/n}$ is "close" to${\displaystyle 0}$ because of the topology of${\displaystyle \mathbf{R}}$.

This definition of weak convergence can be extended for${\displaystyle S}$ anymetrizabletopological space. It also defines a weak topology on${\displaystyle \mathcal{P}(S)}$, the set of all probability measures defined on${\displaystyle (S,\mathrm{\Sigma })}$. The weak topology is generated by the following basis of open sets:

${\displaystyle \left\{U_{\phi ,x,\delta }|\phi :S\to \mathbf{R}\text{ is bounded and continuous, }x\in \mathbf{R}\text{ and }\delta >0\right\},}$

where

If${\displaystyle S}$ is alsoseparable, then${\displaystyle \mathcal{P}(S)}$ is metrizable and separable, for example by theLévy–Prokhorov metric. If${\displaystyle S}$ is also compact orPolish, so is${\displaystyle \mathcal{P}(S)}$.

If${\displaystyle S}$ is separable, it naturally embeds into${\displaystyle \mathcal{P}(S)}$ as the (closed) set ofDirac measures, and itsconvex hull isdense.

There are many "arrow notations" for this kind of convergence: the most frequently used are${\displaystyle P_n\Rightarrow P}$,${\displaystyle P_n\rightharpoonup P}$,${\displaystyle P_n\stackrel{w}{\to }P}$ and${\displaystyle P_n\stackrel{\mathcal{D}}{\to }P}$.

Let${\displaystyle (\mathrm{\Omega },\mathcal{F},\mathbb{P})}$ be aprobability space andX be a metric space. IfX_n: Ω →X is a sequence ofrandom variables thenX_n is said toconverge weakly (orin distribution orin law) to the random variableX: Ω →X asn → ∞ if the sequence ofpushforward measures (X_n)_∗(P) converges weakly toX_∗(P) in the sense of weak convergence of measures onX, as defined above.

Let${\displaystyle X}$ be a metric space (for example${\displaystyle \mathbb{R}}$ or${\displaystyle [0,1]}$). The following spaces of test functions are commonly used in the convergence of probability measures.^[3]

• ${\displaystyle C_c(X)}$ the class of continuous functions${\displaystyle f}$ each vanishing outside a compact set.
• ${\displaystyle C_0(X)}$ the class of continuous functions${\displaystyle f}$ such that${\displaystyle \underset{|x|\to \mathrm{\infty }}{lim}f(x)=0}$
• ${\displaystyle C_B(X)}$ the class of continuous bounded functions

We have${\displaystyle C_c\subset C_0\subset C_B\subset C}$. Moreover,${\displaystyle C_0}$ is the closure of${\displaystyle C_c}$ with respect to uniform convergence.^[3]

A sequence of measures${\displaystyle {\left({\mu }_n\right)}_{n\in \mathbb{N}}}$convergesvaguely to a measure${\displaystyle \mu }$ if for all${\displaystyle f\in C_c(X)}$,${\displaystyle {\int }_{X}fd{\mu }_n\to {\int }_{X}fd\mu }$.

A sequence of measures${\displaystyle {\left({\mu }_n\right)}_{n\in \mathbb{N}}}$converges weakly to a measure${\displaystyle \mu }$ if for all${\displaystyle f\in C_B(X)}$,${\displaystyle {\int }_{X}fd{\mu }_n\to {\int }_{X}fd\mu }$.

In general, these two convergence notions are not equivalent.

In a probability setting, vague convergence and weak convergence of probability measures are equivalent assumingtightness. That is, a tight sequence of probability measures${\displaystyle ({\mu }_n{)}_{n\in \mathbb{N}}}$ convergesvaguely to a probability measure${\displaystyle \mu }$ if and only if${\displaystyle ({\mu }_n{)}_{n\in \mathbb{N}}}$ converges weakly to${\displaystyle \mu }$.

The weak limit of a sequence of probability measures, provided it exists, is a probability measure. In general, if tightness is not assumed, a sequence of probability (or sub-probability) measures may not necessarily convergevaguely to a true probability measure, but rather to a sub-probability measure (a measure such that${\displaystyle \mu (X)\le 1}$).^[3] Thus, a sequence of probability measures${\displaystyle ({\mu }_n{)}_{n\in \mathbb{N}}}$ such that${\displaystyle {\mu }_n\stackrel{v}{\to }\mu }$ where${\displaystyle \mu }$ is not specified to be a probability measure is not guaranteed to imply weak convergence.

Despite having the same name asweak convergence in the context of functional analysis, weak convergence of measures is actually an example of weak-* convergence. The definitions of weak and weak-* convergences used in functional analysis are as follows:

Let${\displaystyle V}$ be a topological vector space or Banach space.

1. A sequence${\displaystyle x_n}$ in${\displaystyle V}$converges weakly to${\displaystyle x}$ if${\displaystyle \phi \left(x_n\right)\to \phi (x)}$ as${\displaystyle n\to \mathrm{\infty }}$ for all${\displaystyle \phi \in V^{\ast }}$. One writes${\displaystyle x_n\stackrel{w}{\to }x}$ as${\displaystyle n\to \mathrm{\infty }}$.
2. A sequence of${\displaystyle {\phi }_n\in V^{\ast }}$converges in the weak-* topology to${\displaystyle \phi }$ provided that${\displaystyle {\phi }_n(x)\to \phi (x)}$ for all${\displaystyle x\in V}$. That is, convergence occurs in the point-wise sense. In this case, one writes${\displaystyle {\phi }_n\stackrel{w^{\ast }}{\to }\phi }$ as${\displaystyle n\to \mathrm{\infty }}$.

To illustrate how weak convergence of measures is an example of weak-* convergence, we give an example in terms of vague convergence (see above). Let${\displaystyle X}$ be a locally compact Hausdorff space. By theRiesz-Representation theorem, the space${\displaystyle M(X)}$ of Radon measures is isomorphic to a subspace of the space of continuous linear functionals on${\displaystyle C_0(X)}$. Therefore, for each Radon measure${\displaystyle {\mu }_n\in M(X)}$, there is a linear functional${\displaystyle {\phi }_n\in C_0(X{)}^{\ast }}$ such that${\displaystyle {\phi }_n(f)={\int }_{X}fd{\mu }_n}$ for all${\displaystyle f\in C_0(X)}$. Applying the definition of weak-* convergence in terms of linear functionals, the characterization of vague convergence of measures is obtained. For compact${\displaystyle X}$,${\displaystyle C_0(X)=C_B(X)}$, so in this case weak convergence of measures is a special case of weak-* convergence.

• Convergence of random variables
• Lévy–Prokhorov metric
• Prokhorov's theorem
• Tightness of measures

• Ambrosio, L., Gigli, N. & Savaré, G. (2005).Gradient Flows in Metric Spaces and in the Space of Probability Measures. Basel: ETH Zürich, Birkhäuser Verlag.ISBN 3-7643-2428-7.{{cite book}}: CS1 maint: multiple names: authors list (link)
• Billingsley, Patrick (1995).Probability and Measure. New York, NY: John Wiley & Sons, Inc.ISBN 0-471-00710-2.
• Billingsley, Patrick (1999).Convergence of Probability Measures. New York, NY: John Wiley & Sons, Inc.ISBN 0-471-19745-9.

This article includes a list ofgeneral references, butit lacks sufficient correspondinginline citations. Please help toimprove this article byintroducing more precise citations.(February 2010) (Learn how and when to remove this message)

• Measure theory
• Convergence (mathematics)

• Articles with short description
• Short description matches Wikidata
• All articles with unsourced statements
• Articles with unsourced statements from February 2024
• CS1 maint: multiple names: authors list
• Articles lacking in-text citations from February 2010
• All articles lacking in-text citations