Inprobability theory, there exist several different notions ofconvergence of sequences of random variables, includingconvergence in probability,convergence in distribution, andalmost sure convergence. The different notions of convergence capture different properties about the sequence, with some notions of convergence being stronger than others. For example, convergence in distribution tells us about the limitdistribution of a sequence of random variables. This is a weaker notion than convergence in probability, which tells us about the value a random variable will take, rather than just the distribution.

The concept is important in probability theory, and its applications tostatistics andstochastic processes. The same concepts are known in more generalmathematics asstochastic convergence and they formalize the idea that certain properties of a sequence of essentially random or unpredictable events can sometimes be expected to settle down into a behavior that is essentially unchanging when items far enough into the sequence are studied. The different possible notions of convergence relate to how such a behavior can be characterized: two readily understood behaviors are that the sequence eventually takes a constant value, and that values in the sequence continue to change but can be described by an unchanging probability distribution.

"Stochastic convergence" formalizes the idea that a sequence of essentially random or unpredictable events can sometimes be expected to settle into a pattern. The pattern may for instance be

• Convergence in the classical sense to a fixed value, perhaps itself coming from a random event
• An increasing similarity of outcomes to what a purely deterministic function would produce
• An increasing preference towards a certain outcome
• An increasing "aversion" against straying far away from a certain outcome
• That the probability distribution describing the next outcome may grow increasingly similar to a certain distribution

Some less obvious, more theoretical patterns could be

• That the series formed by calculating theexpected value of the outcome's distance from a particular value may converge to 0
• That the variance of therandom variable describing the next event grows smaller and smaller.

These other types of patterns that may arise are reflected in the different types of stochastic convergence that have been studied.

While the above discussion has related to the convergence of a single series to a limiting value, the notion of the convergence of two series towards each other is also important, but this is easily handled by studying the sequence defined as either the difference or the ratio of the two series.

For example, if the average ofnindependent random variables${\displaystyle Y_i,i=1,\dots ,n}$, all having the same finitemean andvariance, is given by

${\displaystyle X_n=\frac{1}{n}\sum _{i=1}^n{Y}_i,}$

then as${\displaystyle n}$ tends to infinity,${\displaystyle X_n}$ convergesin probability (see below) to the commonmean,${\displaystyle \mu }$, of the random variables${\displaystyle Y_i}$. This result is known as theweak law of large numbers. Other forms of convergence are important in other useful theorems, including thecentral limit theorem.

Throughout the following, we assume that${\displaystyle (X_n)}$ is a sequence of random variables, and${\displaystyle X}$ is a random variable, and all of them are defined on the sameprobability space${\displaystyle (\mathrm{\Omega },\mathcal{F},\mathbb{P})}$.

Examples of convergence in distribution

Dice factory
Suppose a new dice factory has just been built. The first few dice come out quite biased, due to imperfections in the production process. The outcome from tossing any of them will follow a distribution markedly different from the desireduniform distribution. As the factory is improved, the dice become less and less loaded, and the outcomes from tossing a newly produced die will follow the uniform distribution more and more closely.
Tossing coins
LetXn be the fraction of heads after tossing up an unbiased coinn times. ThenX1 has theBernoulli distribution with expected valueμ = 0.5 and varianceσ2 = 0.25. The subsequent random variablesX2,X3, ... will all be distributedbinomially. Asn grows larger, this distribution will gradually start to take shape more and more similar to thebell curve of the normal distribution. If we shift and rescaleXn appropriately, then${\displaystyle Z_n=\frac{\sqrt{n}}{\sigma }(X_n-\mu )}$ will beconverging in distribution to the standard normal, the result that follows from the celebratedcentral limit theorem.
Graphic example
Suppose{Xi} is aniid sequence ofuniformU(−1, 1) random variables. Let${\displaystyle Z_n=\frac{1}{\sqrt{n}}\sum _{i=1}^n{X}_i}$ be their (normalized) sums. Then according to thecentral limit theorem, the distribution ofZn approaches the normalN(0,⁠1/3⁠) distribution. This convergence is shown in the picture: asn grows larger, the shape of the probability density function gets closer and closer to the Gaussian curve.

Loosely, with this mode of convergence, we increasingly expect to see the next outcome in a sequence of random experiments becoming better and better modeled by a givenprobability distribution. More precisely, the distribution of the associated random variable in the sequence becomes arbitrarily close to a specified fixed distribution.

Convergence in distribution is the weakest form of convergence typically discussed, since it is implied by all other types of convergence mentioned in this article. However, convergence in distribution is very frequently used in practice; most often it arises from application of thecentral limit theorem.

A sequence${\displaystyle X_1,X_2,\dots }$ of real-valuedrandom variables, withcumulative distribution functions${\displaystyle F_1,F_2,\dots }$, is said toconverge in distribution, orconverge weakly, orconverge in law to a random variableX withcumulative distribution functionF if

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}{F}_n(x)=F(x),}$

for every number${\displaystyle x\in \mathbb{R}}$ at which${\displaystyle F}$ iscontinuous.

The requirement that only the continuity points of${\displaystyle F}$ should be considered is essential. For example, if${\displaystyle X_n}$ are distributeduniformly on intervals${\displaystyle \left(0,\frac{1}{n}\right)}$, then this sequence converges in distribution to thedegenerate random variable${\displaystyle X=0}$. Indeed,${\displaystyle F_n(x)=0}$for all${\displaystyle n}$ when${\displaystyle x\le 0}$, and${\displaystyle F_n(x)=1}$ for all${\displaystyle x\ge \frac{1}{n}}$when${\displaystyle n>0}$. However, for this limiting random variable${\displaystyle F(0)=1}$, even though${\displaystyle F_n(0)=0}$ for all${\displaystyle n}$. Thus the convergence of cdfs fails at the point${\displaystyle x=0}$ where${\displaystyle F}$ is discontinuous.

Convergence in distribution may be denoted as

1

where${\displaystyle {\mathcal{L}}_X}$ is the law (probability distribution) ofX. For example, ifX is standard normal we can write${\displaystyle X_n\stackrel{d}{\to }\mathcal{N}(0,1)}$.

Forrandom vectors${\displaystyle \left\{X_1,X_2,\dots \right\}\subset {\mathbb{R}}^k}$ the convergence in distribution is defined similarly. We say that this sequenceconverges in distribution to a randomk-vectorX if

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}\mathbb{P}(X_n\in A)=\mathbb{P}(X\in A)}$

for every${\displaystyle A\subset {\mathbb{R}}^k}$ which is acontinuity set ofX.

The definition of convergence in distribution may be extended from random vectors to more generalrandom elements in arbitrarymetric spaces, and even to the “random variables” which are not measurable — a situation which occurs for example in the study ofempirical processes. This is the “weak convergence of laws without laws being defined” — except asymptotically.^[1]

In this case the termweak convergence is preferable (seeweak convergence of measures), and we say that a sequence of random elements{X_n} converges weakly toX (denoted asX_n ⇒X) if

${\displaystyle {\mathbb{E}}^{\ast }h(X_n)\to \mathbb{E}h(X)}$

for all continuous bounded functionsh.^[2] Here E* denotes theouter expectation, that is the expectation of a “smallest measurable functiong that dominatesh(X_n)”.

• Since${\displaystyle F(a)=\mathbb{P}(X\le a)}$, the convergence in distribution means that the probability forX_n to be in a given range is approximately equal to the probability that the value ofX is in that range, providedn issufficiently large.
• In general, convergence in distribution does not imply that the sequence of correspondingprobability density functions will also converge. As an example one may consider random variables with densitiesf_n(x) = (1 + cos(2πnx))1_(0,1). These random variables converge in distribution to a uniformU(0, 1), whereas their densities do not converge at all.^[3]
  • However, according toScheffé’s theorem, convergence of theprobability density functions implies convergence in distribution.^[4]
• Theportmanteau lemma provides several equivalent definitions of convergence in distribution. Although these definitions are less intuitive, they are used to prove a number of statistical theorems. The lemma states that{X_n} converges in distribution toX if and only if any of the following statements are true:^[5]
  • ${\displaystyle \mathbb{P}(X_n\le x)\to \mathbb{P}(X\le x)}$ for all continuity points of${\displaystyle x\mapsto \mathbb{P}(X\le x)}$;
  • ${\displaystyle \mathbb{E}f(X_n)\to \mathbb{E}f(X)}$ for allbounded,continuous functions${\displaystyle f}$ (where${\displaystyle \mathbb{E}}$ denotes theexpected value operator);
  • ${\displaystyle \mathbb{E}f(X_n)\to \mathbb{E}f(X)}$ for all bounded,Lipschitz functions${\displaystyle f}$;
  • ${\displaystyle liminf\mathbb{E}f(X_n)\ge \mathbb{E}f(X)}$ for all nonnegative, continuous functions${\displaystyle f}$;
  • ${\displaystyle liminf\mathbb{P}(X_n\in G)\ge \mathbb{P}(X\in G)}$ for everyopen set${\displaystyle G}$;
  • ${\displaystyle limsup\mathbb{P}(X_n\in F)\le \mathbb{P}(X\in F)}$ for everyclosed set${\displaystyle F}$;
  • ${\displaystyle \mathbb{P}(X_n\in B)\to \mathbb{P}(X\in B)}$ for allcontinuity sets${\displaystyle B}$ of random variable${\displaystyle X}$;
  • ${\displaystyle lim sup\mathbb{E}f(X_n)\le \mathbb{E}f(X)}$ for everyupper semi-continuous function${\displaystyle f}$ bounded above;^{[citation needed]}
  • ${\displaystyle lim inf\mathbb{E}f(X_n)\ge \mathbb{E}f(X)}$ for everylower semi-continuous function${\displaystyle f}$ bounded below.^{[citation needed]}
• Thecontinuous mapping theorem states that for a continuous functiong, if the sequence{X_n} converges in distribution toX, then{g(X_n)} converges in distribution tog(X).
  • Note however that convergence in distribution of{X_n} toX and{Y_n} toY does in generalnot imply convergence in distribution of{X_n +Y_n} toX +Y or of{X_nY_n} toXY.
• Lévy’s continuity theorem: The sequence{X_n} converges in distribution toX if and only if the sequence of correspondingcharacteristic functions{φ_n}converges pointwise to the characteristic functionφ ofX.
• Convergence in distribution ismetrizable by theLévy–Prokhorov metric.
• A natural link to convergence in distribution is theSkorokhod's representation theorem.

The basic idea behind this type of convergence is that the probability of an “unusual” outcome becomes smaller and smaller as the sequence progresses.

The concept of convergence in probability is used very often in statistics. For example, an estimator is calledconsistent if it converges in probability to the quantity being estimated. Convergence in probability is also the type of convergence established by theweak law of large numbers.

A sequence {X_n} of random variablesconverges in probability towards the random variableX if for allε > 0

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}\mathbb{P}(|X_n-X|>\epsilon )=0.}$

More explicitly, letP_n(ε) be the probability thatX_n is outside the ball of radiusε centered at X. ThenX_n is said to converge in probability toX if for anyε > 0 and anyδ > 0 there exists a numberN (which may depend onε andδ) such that for alln ≥ N,P_n(ε) < δ (the definition of limit).

Notice that for the condition to be satisfied, it is not possible that for eachn the random variablesX andX_n are independent (and thus convergence in probability is a condition on the joint cdf's, as opposed to convergence in distribution, which is a condition on the individual cdf's), unlessX is deterministic like for the weak law of large numbers. At the same time, the case of a deterministicX cannot, whenever the deterministic value is a discontinuity point (not isolated), be handled by convergence in distribution, where discontinuity points have to be explicitly excluded.

Convergence in probability is denoted by adding the letterp over an arrow indicating convergence, or using the "plim" probability limit operator:

${\displaystyle X_n\stackrel{p}{\to }X,X_n\stackrel{P}{\to }X,\underset{n\to \mathrm{\infty }}{\mathrm{plim}}{X}_n=X.}$

2

For random elements {X_n} on aseparable metric space(S,d), convergence in probability is defined similarly by^[6]

${\displaystyle \mathrm{\forall }\epsilon >0,\mathbb{P}(d(X_n,X)\ge \epsilon )\to 0.}$

• Convergence in probability implies convergence in distribution.^[proof]
• In the opposite direction, convergence in distribution implies convergence in probability when the limiting random variableX is a constant.^[proof]
• Convergence in probability does not imply almost sure convergence.^[proof]
• Thecontinuous mapping theorem states that for every continuous function${\displaystyle g}$, if$X_n\stackrel{p}{\to }X$, then also $g(X_n)\stackrel{p}{\to }g(X)$.
• Convergence in probability defines atopology on the space of random variables over a fixed probability space. This topology ismetrizable by theKy Fan metric:^[7]$${\displaystyle d(X,Y)=inf\{\epsilon >0:\mathbb{P}(|X-Y|>\epsilon )\le \epsilon \}}$$ or alternately by this metric$${\displaystyle d(X,Y)=\mathbb{E}\left[min(|X-Y|,1)\right].}$$

Not every sequence of random variables which converges to another random variable in distribution also converges in probability to that random variable. As an example, consider a sequence of standard normal random variables${\displaystyle X_n}$ and a second sequence${\displaystyle Y_n=(-1{)}^n{X}_n}$. Notice that the distribution of${\displaystyle Y_n}$ is equal to the distribution of${\displaystyle X_n}$ for all${\displaystyle n}$, but:$${\displaystyle P(|X_n-Y_n|\ge ϵ)=P(|X_n|\cdot |(1-(-1{)}^n)|\ge ϵ)}$$

which does not converge to${\displaystyle 0}$. So we do not have convergence in probability.

This is the type of stochastic convergence that is most similar topointwise convergence known from elementaryreal analysis.

To say that the sequenceX_n convergesalmost surely oralmost everywhere orwith probability 1 orstrongly towardsX means that$${\displaystyle \mathbb{P}\left(\underset{n\to \mathrm{\infty }}{lim}{X}_n=X\right)=1.}$$

This means that the values ofX_n approach the value ofX, in the sense that events for whichX_n does not converge toX have probability 0 (seeAlmost surely). Using the probability space${\displaystyle (\mathrm{\Omega },\mathcal{F},\mathbb{P})}$ and the concept of the random variable as a function from Ω toR, this is equivalent to the statement$${\displaystyle \mathbb{P}(\omega \in \mathrm{\Omega }:\underset{n\to \mathrm{\infty }}{lim}{X}_n(\omega )=X(\omega ))=1.}$$

Using the notion of thelimit superior of a sequence of sets, almost sure convergence can also be defined as follows:$${\displaystyle \mathbb{P}(\underset{n\to \mathrm{\infty }}{lim sup}\{\omega \in \mathrm{\Omega }:|X_n(\omega )-X(\omega )|>\epsilon \})=0\text{for all}\epsilon >0.}$$

Almost sure convergence is often denoted by adding the lettersa.s. over an arrow indicating convergence:

${\displaystyle \stackrel{}{X_n\stackrel{\mathrm{a}.\mathrm{s}.}{\to }X.}}$

3

For genericrandom elements {X_n} on ametric space${\displaystyle (S,d)}$, convergence almost surely is defined similarly:$${\displaystyle \mathbb{P}(\omega \in \mathrm{\Omega }:d(X_n(\omega ),X(\omega ))\underset{n\to \mathrm{\infty }}{⟶}0)=1}$$

• Almost sure convergence implies convergence in probability (byFatou's lemma), and hence implies convergence in distribution. It is the notion of convergence used in the stronglaw of large numbers.
• The concept of almost sure convergence does not come from atopology on the space of random variables. This means there is no topology on the space of random variables such that the almost surely convergent sequences are exactly the converging sequences with respect to that topology. In particular, there is no metric of almost sure convergence.

Consider a sequence${\displaystyle \{X_n\}}$ of independent random variables such that${\displaystyle P(X_n=1)=\frac{1}{n}}$ and${\displaystyle P(X_n=0)=1-\frac{1}{n}}$. For we have${\displaystyle P(|X_n|\ge \epsilon )=\frac{1}{n}}$ which converges to${\displaystyle 0}$ hence${\displaystyle X_n\to 0}$ in probability.

Since${\displaystyle \sum _{n\ge 1}P(X_n=1)\to \mathrm{\infty }}$ and the events${\displaystyle \{X_n=1\}}$ are independent,second Borel Cantelli Lemma ensures that${\displaystyle P(\underset{n}{lim sup}\{X_n=1\})=1}$ hence the sequence${\displaystyle \{X_n\}}$ does not converge to${\displaystyle 0}$ almost everywhere (in fact the set on which this sequence does not converge to${\displaystyle 0}$ has probability${\displaystyle 1}$).

To say that the sequence ofrandom variables (X_n) defined over the sameprobability space (i.e., arandom process) convergessurely oreverywhere orpointwise towardsX means

$${\displaystyle \mathrm{\forall }\omega \in \mathrm{\Omega }:\underset{n\to \mathrm{\infty }}{lim}{X}_n(\omega )=X(\omega ),}$$

where Ω is thesample space of the underlyingprobability space over which the random variables are defined.

This is the notion ofpointwise convergence of a sequence of functions extended to a sequence ofrandom variables. (Note that random variables themselves are functions).

$${\displaystyle \left\{\omega \in \mathrm{\Omega }:\underset{n\to \mathrm{\infty }}{lim}{X}_n(\omega )=X(\omega )\right\}=\mathrm{\Omega }.}$$

Sure convergence of a random variable implies all the other kinds of convergence stated above, but there is no payoff inprobability theory by using sure convergence compared to using almost sure convergence. The difference between the two only exists on sets with probability zero. This is why the concept of sure convergence of random variables is very rarely used.

Given a real numberr ≥ 1, we say that the sequenceX_n convergesin ther-th mean (orin theL^r-norm) towards the random variableX, if ther-thabsolute moments${\displaystyle \mathbb{E}}$(|X_n|^r) and${\displaystyle \mathbb{E}}$(|X|^r) ofX_n andX exist, and

${\displaystyle \underset{n\to \mathrm{\infty }}{lim}\mathbb{E}\left(|X_n-X{|}^r\right)=0,}$

where the operator E denotes theexpected value. Convergence inr-th mean tells us that the expectation of ther-th power of the difference between${\displaystyle X_n}$ and${\displaystyle X}$ converges to zero.

This type of convergence is often denoted by adding the letterL^r over an arrow indicating convergence:

${\displaystyle \stackrel{}{X_n\stackrel{L^r}{\to }X.}}$

4

The most important cases of convergence inr-th mean are:

• WhenX_n converges inr-th mean toX forr = 1, we say thatX_n convergesin mean toX.
• WhenX_n converges inr-th mean toX forr = 2, we say thatX_n convergesin mean square (orin quadratic mean) toX.

Convergence in ther-th mean, forr ≥ 1, implies convergence in probability (byMarkov's inequality). Furthermore, ifr >s ≥ 1, convergence inr-th mean implies convergence ins-th mean. Hence, convergence in mean square implies convergence in mean.

Additionally,

${\displaystyle \stackrel{}{X_n\stackrel{L^r}{\to }X}\Rightarrow \underset{n\to \mathrm{\infty }}{lim}\mathbb{E}[|X_n{|}^r]=\mathbb{E}[|X{|}^r].}$

The converse is not necessarily true, however it is true if${\displaystyle \stackrel{}{X_n\stackrel{p}{\to }X}}$ (by a more general version ofScheffé's lemma).

Provided the probability space iscomplete:

• If${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X}$ and${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }Y}$, then${\displaystyle X=Y}$almost surely.
• If${\displaystyle X_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }X}$ and${\displaystyle X_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }Y}$, then${\displaystyle X=Y}$ almost surely.
• If${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }X}$ and${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }Y}$, then${\displaystyle X=Y}$ almost surely.
• If${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X}$ and${\displaystyle Y_n\stackrel{\stackrel{}{p}}{\to }Y}$, then${\displaystyle a{X}_n+b{Y}_n\stackrel{\stackrel{}{p}}{\to }aX+bY}$ (for any real numbersa andb) and${\displaystyle X_n{Y}_n\stackrel{\stackrel{}{p}}{\to }XY}$.
• If${\displaystyle X_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }X}$ and${\displaystyle Y_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }Y}$, then${\displaystyle a{X}_n+b{Y}_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }aX+bY}$ (for any real numbersa andb) and${\displaystyle X_n{Y}_n\stackrel{\stackrel{}{\text{a.s.}}}{\to }XY}$.
• If${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }X}$ and${\displaystyle Y_n\stackrel{\stackrel{}{L^r}}{\to }Y}$, then${\displaystyle a{X}_n+b{Y}_n\stackrel{\stackrel{}{L^r}}{\to }aX+bY}$ (for any real numbersa andb).
• None of the above statements are true for convergence in distribution.

The chain of implications between the various notions of convergence are noted in their respective sections. They are, using the arrow notation:

These properties, together with a number of other special cases, are summarized in the following list:

• Almost sure convergence implies convergence in probability:^[8][proof]

  ${\displaystyle X_n\stackrel{\text{a.s.}}{\to }X\Rightarrow X_n\stackrel{\stackrel{}{p}}{\to }X}$

• Convergence in probability implies there exists a sub-sequence${\displaystyle (n_k)}$ which almost surely converges:^[9]

  ${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X\Rightarrow X_{n_k}\stackrel{\text{a.s.}}{\to }X}$

• Convergence in probability implies convergence in distribution:^[8][proof]

  ${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X\Rightarrow X_n\stackrel{\stackrel{}{d}}{\to }X}$

• Convergence inr-th order mean implies convergence in probability:

  ${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }X\Rightarrow X_n\stackrel{\stackrel{}{p}}{\to }X}$

• Convergence inr-th order mean implies convergence in lower order mean, assuming that both orders are greater than or equal to one:

  ${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }X\Rightarrow X_n\stackrel{\stackrel{}{L^s}}{\to }X,}$providedr ≥s ≥ 1.

• IfX_n converges in distribution to a constantc, thenX_n converges in probability toc:^[8][proof]

  ${\displaystyle X_n\stackrel{\stackrel{}{d}}{\to }c\Rightarrow X_n\stackrel{\stackrel{}{p}}{\to }c,}$providedc is a constant.

• IfX_n converges in distribution toX and the difference betweenX_n andY_n converges in probability to zero, thenY_n also converges in distribution toX:^[8][proof]

  ${\displaystyle X_n\stackrel{\stackrel{}{d}}{\to }X,|X_n-Y_n|\stackrel{\stackrel{}{p}}{\to }0\Rightarrow Y_n\stackrel{\stackrel{}{d}}{\to }X}$

• IfX_n converges in distribution toX andY_n converges in distribution to a constantc, then the joint vector(X_n, Y_n) converges in distribution to⁠${\displaystyle (X,c)}$⁠:^[8][proof]

  ${\displaystyle X_n\stackrel{\stackrel{}{d}}{\to }X,Y_n\stackrel{\stackrel{}{d}}{\to }c\Rightarrow (X_n,Y_n)\stackrel{\stackrel{}{d}}{\to }(X,c)}$providedc is a constant.

  Note that the condition thatY_n converges to a constant is important, if it were to converge to a random variableY then we wouldn't be able to conclude that(X_n, Y_n) converges to⁠${\displaystyle (X,Y)}$⁠.

• IfX_n converges in probability toX andY_n converges in probability toY, then the joint vector(X_n, Y_n) converges in probability to(X, Y):^[8][proof]

  ${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X,Y_n\stackrel{\stackrel{}{p}}{\to }Y\Rightarrow (X_n,Y_n)\stackrel{\stackrel{}{p}}{\to }(X,Y)}$

• IfX_n converges in probability toX, and ifP(|X_n| ≤b) = 1 for alln and someb, thenX_n converges inrth mean toX for allr ≥ 1. In other words, ifX_n converges in probability toX and all random variablesX_n are almost surely bounded above and below, thenX_n converges toX also in anyrth mean.^[10]
• Almost sure representation. Usually, convergence in distribution does not imply convergence almost surely. However, for a given sequence {X_n} which converges in distribution toX₀ it is always possible to find a new probability space (Ω,F, P) and random variables {Y_n,n = 0, 1, ...} defined on it such thatY_n is equal in distribution toX_n for eachn ≥ 0, andY_n converges toY₀ almost surely.^[11][12]
• If for allε > 0,

  

  then we say thatX_nconverges almost completely, oralmost in probability towardsX. WhenX_n converges almost completely towardsX then it also converges almost surely toX. In other words, ifX_n converges in probability toX sufficiently quickly (i.e. the above sequence of tail probabilities is summable for allε > 0), thenX_n also converges almost surely toX. This is a direct implication from theBorel–Cantelli lemma.

• IfS_n is a sum ofn real independent random variables:

  ${\displaystyle S_n=X_1+\cdots +X_n}$

  thenS_n converges almost surely if and only ifS_n converges in probability. The proof can be found in Page 126 (Theorem 5.3.4) of the book byKai Lai Chung.^[13]

  However, for a sequence of mutually independent random variables, convergence in probability does not imply almost sure convergence.^{[14][circular reference]}

• Thedominated convergence theorem gives sufficient conditions for almost sure convergence to implyL¹-convergence:

5

• A necessary and sufficient condition forL¹ convergence is${\displaystyle X_n\stackrel{\stackrel{}{P}}{\to }X}$ and the sequence (X_n) isuniformly integrable.
• If${\displaystyle X_n\stackrel{\stackrel{}{p}}{\to }X}$, the followings are equivalent^[15]
  • ${\displaystyle X_n\stackrel{\stackrel{}{L^r}}{\to }X}$,
  • ,
  • ${\displaystyle \{|X_n{|}^r\}}$ isuniformly integrable.

The WikibookEconometric Theory has a page on the topic of:Convergence of random variables

• Proofs of convergence of random variables
• Convergence of measures
• Convergence in measure
• Continuous stochastic process: the question of continuity of astochastic process is essentially a question of convergence, and many of the same concepts and relationships used above apply to the continuity question.
• Asymptotic distribution
• Big O in probability notation
• Skorokhod's representation theorem
• The Tweedie convergence theorem
• Slutsky's theorem
• Continuous mapping theorem

This article incorporates material from theCitizendium article "Stochastic convergence", which is licensed under theCreative Commons Attribution-ShareAlike 3.0 Unported License but not under theGFDL.

• Stochastic processes
• Convergence (mathematics)

• CS1 maint: location
• Articles with short description
• Short description matches Wikidata
• All articles with unsourced statements
• Articles with unsourced statements from February 2013
• All articles lacking reliable references
• Articles lacking reliable references from February 2025
• All articles containing circular references
• Wikipedia articles incorporating text from Citizendium