Movatterモバイル変換

[0]ホーム
URL:

Jump to content
WikipediaThe Free Encyclopedia
Search

Characteristic function (probability theory)

From Wikipedia, the free encyclopedia
(Redirected fromCharacteristic function (probability))
Fourier transform of the probability density function
The characteristic function of a uniformU(–1,1) random variable. This function is real-valued because it corresponds to a random variable that is symmetric around the origin; however characteristic functions may generally be complex-valued.

Inprobability theory andstatistics, thecharacteristic function of anyreal-valuedrandom variable completely defines itsprobability distribution. If a random variable admits aprobability density function, then the characteristic function is theFourier transform (with sign reversal) of the probability density function. Thus it provides an alternative route to analytical results compared with working directly withprobability density functions orcumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables.

In addition tounivariate distributions, characteristic functions can be defined for vector- or matrix-valued random variables, and can also be extended to more generic cases.

The characteristic function always exists when treated as a function of a real-valued argument, unlike themoment-generating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function.

Introduction

[edit]

The characteristic function is a way to describe arandom variableX.Thecharacteristic function,

φX(t)=E⁡[eitX],{\displaystyle \varphi _{X}(t)=\operatorname {E} \left[e^{itX}\right],}

a function oft,determines the behavior and properties of the probability distribution ofX.It is equivalent to aprobability density function orcumulative distribution function, since knowing one of these functions allows computation of the others, but they provide different insights into the features of the random variable. In particular cases, one or another of these equivalent functions may be easier to represent in terms of simple standard functions.

If a random variable admits adensity function, then the characteristic function is itsFourier dual, in the sense that each of them is aFourier transform of the other. If a random variable has amoment-generating functionMX(t){\displaystyle M_{X}(t)}, then the domain of the characteristic function can be extended to the complex plane, and

φX(−it)=MX(t).{\displaystyle \varphi _{X}(-it)=M_{X}(t).}[1]

Note however that the characteristic function of a distribution is well defined for allreal values oft, even when themoment-generating function is not well defined for all real values oft.

The characteristic function approach is particularly useful in analysis of linear combinations of independent random variables: a classical proof of theCentral Limit Theorem uses characteristic functions andLévy's continuity theorem. Another important application is to the theory of thedecomposability of random variables.

Definition

[edit]

For a scalar random variableX thecharacteristic function is defined as theexpected value ofeitX, wherei is theimaginary unit, andt ∈R is the argument of the characteristic function:

{φX:R→CφX(t)=E⁡[eitX]=∫ReitxdFX(x)=∫ReitxfX(x)dx=∫01eitQX(p)dp{\displaystyle {\begin{cases}\displaystyle \varphi _{X}\!:\mathbb {R} \to \mathbb {C} \\\displaystyle \varphi _{X}(t)=\operatorname {E} \left[e^{itX}\right]=\int _{\mathbb {R} }e^{itx}\,dF_{X}(x)=\int _{\mathbb {R} }e^{itx}f_{X}(x)\,dx=\int _{0}^{1}e^{itQ_{X}(p)}\,dp\end{cases}}}

HereFX is thecumulative distribution function ofX,fX is the correspondingprobability density function,QX(p) is the corresponding inverse cumulative distribution function also called thequantile function,[2] and the integrals are of theRiemann–Stieltjes kind. If a random variableX has aprobability density function then the characteristic function is itsFourier transform with sign reversal in the complex exponential.[3][4] This convention for the constants appearing in the definition of the characteristic function differs from the usual convention for the Fourier transform.[5] For example, some authors[6] defineφX(t) = E[e−2πitX], which is essentially a change of parameter. Other notation may be encountered in the literature:p^{\displaystyle \scriptstyle {\hat {p}}} as the characteristic function for a probability measurep, orf^{\displaystyle \scriptstyle {\hat {f}}} as the characteristic function corresponding to a densityf.

Generalizations

[edit]

The notion of characteristic functions generalizes to multivariate random variables and more complicatedrandom elements. The argument of the characteristic function will always belong to thecontinuous dual of the space where the random variableX takes its values. For common cases such definitions are listed below:

Examples

[edit]
DistributionCharacteristic functionφ(t){\displaystyle \varphi (t)}
Degenerateδaeita{\displaystyle e^{ita}}
BernoulliBern(p)1−p+peit{\displaystyle 1-p+pe^{it}}
BinomialB(n, p)(1−p+peit)n{\displaystyle (1-p+pe^{it})^{n}}
Negative binomialNB(r, p)(p1−eit+peit)r{\displaystyle \left({\frac {p}{1-e^{it}+pe^{it}}}\right)^{r}}
PoissonPois(λ)eλ(eit−1){\displaystyle e^{\lambda (e^{it}-1)}}
Uniform (continuous)U(a, b)eitb−eitait(b−a){\displaystyle {\frac {e^{itb}-e^{ita}}{it(b-a)}}}
Uniform (discrete)DU(a, b)eita−eit(b+1)(1−eit)(b−a+1){\displaystyle {\frac {e^{ita}-e^{it(b+1)}}{(1-e^{it})(b-a+1)}}}
LaplaceL(μ,b)eitμ1+b2t2{\displaystyle {\frac {e^{it\mu }}{1+b^{2}t^{2}}}}
LogisticLogistic(μ,s)
eiμtπstsinh⁡(πst){\displaystyle e^{i\mu t}{\frac {\pi st}{\sinh(\pi st)}}}
NormalN(μ,σ2)eitμ−12σ2t2{\displaystyle e^{it\mu -{\frac {1}{2}}\sigma ^{2}t^{2}}}
Chi-squaredχ2k(1−2it)−k/2{\displaystyle (1-2it)^{-k/2}}
Noncentral chi-squaredχk′2{\displaystyle {\chi '_{k}}^{2}}eiλt1−2it(1−2it)−k/2{\displaystyle e^{\frac {i\lambda t}{1-2it}}(1-2it)^{-k/2}}
Generalized chi-squaredχ~(w,k,λ,s,m){\displaystyle {\tilde {\chi }}({\boldsymbol {w}},{\boldsymbol {k}},{\boldsymbol {\lambda }},s,m)}exp⁡[it(m+∑jwjλj1−2iwjt)−s2t22]∏j(1−2iwjt)kj/2{\displaystyle {\frac {\exp \left[it\left(m+\sum _{j}{\frac {w_{j}\lambda _{j}}{1-2iw_{j}t}}\right)-{\frac {s^{2}t^{2}}{2}}\right]}{\prod _{j}\left(1-2iw_{j}t\right)^{k_{j}/2}}}}
CauchyC(μ,θ)eitμ−θ|t|{\displaystyle e^{it\mu -\theta |t|}}
GammaΓ(k,θ)(1−itθ)−k{\displaystyle (1-it\theta )^{-k}}
ExponentialExp(λ)(1−itλ−1)−1{\displaystyle (1-it\lambda ^{-1})^{-1}}
GeometricGf(p)
(number of failures)		p1−eit(1−p){\displaystyle {\frac {p}{1-e^{it}(1-p)}}}
GeometricGt(p)
(number of trials)		pe−it−(1−p){\displaystyle {\frac {p}{e^{-it}-(1-p)}}}
Multivariate normalN(μ,Σ)eitTμ−12tTΣt{\displaystyle e^{i{\mathbf {t} ^{\mathrm {T} }{\boldsymbol {\mu }}}-{\frac {1}{2}}\mathbf {t} ^{\mathrm {T} }{\boldsymbol {\Sigma }}\mathbf {t} }}
Multivariate CauchyMultiCauchy(μ,Σ)[10]eitTμ−tTΣt{\displaystyle e^{i\mathbf {t} ^{\mathrm {T} }{\boldsymbol {\mu }}-{\sqrt {\mathbf {t} ^{\mathrm {T} }{\boldsymbol {\Sigma }}\mathbf {t} }}}}

Oberhettinger (1973) provides extensive tables of characteristic functions.

Properties

[edit]

Continuity

[edit]

The bijection stated above between probability distributions and characteristic functions issequentially continuous. That is, whenever a sequence of distribution functionsFj(x) converges (weakly) to some distributionF(x), the corresponding sequence of characteristic functionsφj(t) will also converge, and the limitφ(t) will correspond to the characteristic function of lawF. More formally, this is stated as

Lévy’s continuity theorem: A sequenceXj ofn-variate random variablesconverges in distribution to random variableX if and only if the sequenceφXj converges pointwise to a functionφ which is continuous at the origin. Whereφ is the characteristic function ofX.[14]

This theorem can be used to prove thelaw of large numbers and thecentral limit theorem.

Inversion formula

[edit]

There is aone-to-one correspondence between cumulative distribution functions and characteristic functions, so it is possible to find one of these functions if we know the other. The formula in the definition of characteristic function allows us to computeφ when we know the distribution functionF (or densityf). If, on the other hand, we know the characteristic functionφ and want to find the corresponding distribution function, then one of the followinginversion theorems can be used.

Theorem. If the characteristic functionφX of a random variableX isintegrable, thenFX is absolutely continuous, and thereforeX has aprobability density function. In the univariate case (i.e. whenX is scalar-valued) the density function is given byfX(x)=FX′(x)=12π∫Re−itxφX(t)dt.{\displaystyle f_{X}(x)=F_{X}'(x)={\frac {1}{2\pi }}\int _{\mathbf {R} }e^{-itx}\varphi _{X}(t)\,dt.}

In the multivariate case it isfX(x)=1(2π)n∫Rne−i(t⋅x)φX(t)λ(dt){\displaystyle f_{X}(x)={\frac {1}{(2\pi )^{n}}}\int _{\mathbf {R} ^{n}}e^{-i(t\cdot x)}\varphi _{X}(t)\lambda (dt)}

wheret⋅x{\textstyle t\cdot x} is thedot product.

The density function is theRadon–Nikodym derivative of the distributionμX with respect to theLebesgue measureλ:fX(x)=dμXdλ(x).{\displaystyle f_{X}(x)={\frac {d\mu _{X}}{d\lambda }}(x).}

Theorem (Lévy).[note 1] IfφX is characteristic function of distribution functionFX, two pointsa <b are such that{x |a <x <b} is acontinuity set ofμX (in the univariate case this condition is equivalent to continuity ofFX at pointsa andb), then

Theorem. Ifa is (possibly) an atom ofX (in the univariate case this means a point of discontinuity ofFX) then

Theorem (Gil-Pelaez).[17] For a univariate random variableX, ifx is acontinuity point ofFX then

FX(x)=12−1π∫0∞Im⁡[e−itxφX(t)]tdt{\displaystyle F_{X}(x)={\frac {1}{2}}-{\frac {1}{\pi }}\int _{0}^{\infty }{\frac {\operatorname {Im} [e^{-itx}\varphi _{X}(t)]}{t}}\,dt}

where the imaginary part of a complex numberz{\displaystyle z} is given byIm(z)=(z−z∗)/2i{\displaystyle \mathrm {Im} (z)=(z-z^{*})/2i}.

And its density function is:

fX(x)=1π∫0∞Re⁡[e−itxφX(t)]dt{\displaystyle f_{X}(x)={\frac {1}{\pi }}\int _{0}^{\infty }\operatorname {Re} [e^{-itx}\varphi _{X}(t)]\,dt}

The integral may be notLebesgue-integrable; for example, whenX is thediscrete random variable that is always 0, it becomes theDirichlet integral.

Inversion formulas for multivariate distributions are available.[15][18]

Criteria for characteristic functions

[edit]

The set of all characteristic functions is closed under certain operations:

It is well known that any non-decreasingcàdlàg functionF with limitsF(−∞) = 0,F(+∞) = 1 corresponds to acumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given functionφ could be the characteristic function of some random variable. The central result here isBochner’s theorem, although its usefulness is limited because the main condition of the theorem,non-negative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult.Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólya-type.[19]

Bochner’s theorem. An arbitrary functionφ :Rn →C is the characteristic function of some random variable if and only ifφ ispositive definite, continuous at the origin, and ifφ(0) = 1.

Khinchine’s criterion. A complex-valued, absolutely continuous functionφ, withφ(0) = 1, is a characteristic function if and only if it admits the representation

φ(t)=∫Rg(t+θ)g(θ)¯dθ.{\displaystyle \varphi (t)=\int _{\mathbf {R} }g(t+\theta ){\overline {g(\theta )}}\,d\theta .}

Mathias’ theorem. A real-valued, even, continuous, absolutely integrable functionφ, withφ(0) = 1, is a characteristic function if and only if

(−1)n(∫Rφ(pt)e−t2/2H2n(t)dt)≥0{\displaystyle (-1)^{n}\left(\int _{\mathbf {R} }\varphi (pt)e^{-t^{2}/2}H_{2n}(t)\,dt\right)\geq 0}

forn = 0,1,2,..., and allp > 0. HereH2n denotes theHermite polynomial of degree2n.

Pólya’s theorem can be used to construct an example of two random variables whose characteristic functions coincide over a finite interval but are different elsewhere.

Pólya’s theorem. Ifφ{\displaystyle \varphi } is a real-valued, even, continuous function which satisfies the conditions

thenφ(t) is the characteristic function of an absolutely continuous distribution symmetric about 0.

Uses

[edit]

Because of thecontinuity theorem, characteristic functions are used in the most frequently seen proof of thecentral limit theorem. The main technique involved in making calculations with a characteristic function is recognizing the function as the characteristic function of a particular distribution.

Basic manipulations of distributions

[edit]

Characteristic functions are particularly useful for dealing with linear functions ofindependent random variables. For example, ifX1,X2, ...,Xn is a sequence of independent (and not necessarily identically distributed) random variables, and

Sn=∑i=1naiXi,{\displaystyle S_{n}=\sum _{i=1}^{n}a_{i}X_{i},\,\!}

where theai are constants, then the characteristic function forSn is given by

φSn(t)=φX1(a1t)φX2(a2t)⋯φXn(ant){\displaystyle \varphi _{S_{n}}(t)=\varphi _{X_{1}}(a_{1}t)\varphi _{X_{2}}(a_{2}t)\cdots \varphi _{X_{n}}(a_{n}t)\,\!}

In particular,φX+Y(t) =φX(t)φY(t). To see this, write out the definition of characteristic function:

φX+Y(t)=E⁡[eit(X+Y)]=E⁡[eitXeitY]=E⁡[eitX]E⁡[eitY]=φX(t)φY(t){\displaystyle \varphi _{X+Y}(t)=\operatorname {E} \left[e^{it(X+Y)}\right]=\operatorname {E} \left[e^{itX}e^{itY}\right]=\operatorname {E} \left[e^{itX}\right]\operatorname {E} \left[e^{itY}\right]=\varphi _{X}(t)\varphi _{Y}(t)}

The independence ofX andY is required to establish the equality of the third and fourth expressions.

Another special case of interest for identically distributed random variables is whenai = 1 /n and thenSn is the sample mean. In this case, writingX for the mean,

φX¯(t)=φX(tn)n{\displaystyle \varphi _{\overline {X}}(t)=\varphi _{X}\!\left({\tfrac {t}{n}}\right)^{n}}

Moments

[edit]

Characteristic functions can also be used to findmoments of a random variable. Provided that then-th moment exists, the characteristic function can be differentiatedn times:

E⁡[Xn]=i−n[dndtnφX(t)]t=0=i−nφX(n)(0),{\displaystyle \operatorname {E} \left[X^{n}\right]=i^{-n}\left[{\frac {d^{n}}{dt^{n}}}\varphi _{X}(t)\right]_{t=0}=i^{-n}\varphi _{X}^{(n)}(0),\!}

This can be formally written using the derivatives of theDirac delta function:fX(x)=∑n=0∞(−1)nn!δ(n)(x)E⁡[Xn]{\displaystyle f_{X}(x)=\sum _{n=0}^{\infty }{\frac {(-1)^{n}}{n!}}\delta ^{(n)}(x)\operatorname {E} [X^{n}]}which allows a formal solution to themoment problem.For example, supposeX has a standardCauchy distribution. ThenφX(t) =e−|t|. This is notdifferentiable att = 0, showing that the Cauchy distribution has noexpectation. Also, the characteristic function of the sample meanX ofnindependent observations has characteristic functionφX(t) = (e−|t|/n)n =e−|t|, using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself.

As a further example, supposeX follows aGaussian distribution i.e.X∼N(μ,σ2){\displaystyle X\sim {\mathcal {N}}(\mu ,\sigma ^{2})}. ThenφX(t)=eμit−12σ2t2{\displaystyle \varphi _{X}(t)=e^{\mu it-{\frac {1}{2}}\sigma ^{2}t^{2}}} and

E⁡[X]=i−1[ddtφX(t)]t=0=i−1[(iμ−σ2t)φX(t)]t=0=μ{\displaystyle \operatorname {E} \left[X\right]=i^{-1}\left[{\frac {d}{dt}}\varphi _{X}(t)\right]_{t=0}=i^{-1}\left[(i\mu -\sigma ^{2}t)\varphi _{X}(t)\right]_{t=0}=\mu }

A similar calculation showsE⁡[X2]=μ2+σ2{\displaystyle \operatorname {E} \left[X^{2}\right]=\mu ^{2}+\sigma ^{2}} and is easier to carry out than applying the definition of expectation and using integration by parts to evaluateE⁡[X2]{\displaystyle \operatorname {E} \left[X^{2}\right]}.

The logarithm of a characteristic function is acumulant generating function, which is useful for findingcumulants; some instead define the cumulant generating function as the logarithm of themoment-generating function, and call the logarithm of the characteristic function thesecond cumulant generating function.

Data analysis

[edit]

Characteristic functions can be used as part of procedures for fitting probability distributions to samples of data. Cases where this provides a practicable option compared to other possibilities include fitting thestable distribution since closed form expressions for the density are not available which makes implementation ofmaximum likelihood estimation difficult. Estimation procedures are available which match the theoretical characteristic function to theempirical characteristic function, calculated from the data. Paulson et al. (1975)[20] and Heathcote (1977)[21] provide some theoretical background for such an estimation procedure. In addition, Yu (2004)[22] describes applications of empirical characteristic functions to fittime series models where likelihood procedures are impractical. Empirical characteristic functions have also been used by Ansari et al. (2020)[23] and Li et al. (2020)[24] for traininggenerative adversarial networks.

Example

[edit]

Thegamma distribution with scale parameter θ and a shape parameterk has the characteristic function

(1−θit)−k.{\displaystyle (1-\theta it)^{-k}.}

Now suppose that we have

X ∼Γ(k1,θ) and Y∼Γ(k2,θ){\displaystyle X~\sim \Gamma (k_{1},\theta ){\mbox{ and }}Y\sim \Gamma (k_{2},\theta )}

withX andY independent from each other, and we wish to know what the distribution ofX +Y is. The characteristic functions are

φX(t)=(1−θit)−k1,φY(t)=(1−θit)−k2{\displaystyle \varphi _{X}(t)=(1-\theta it)^{-k_{1}},\,\qquad \varphi _{Y}(t)=(1-\theta it)^{-k_{2}}}

which by independence and the basic properties of characteristic function leads to

φX+Y(t)=φX(t)φY(t)=(1−θit)−k1(1−θit)−k2=(1−θit)−(k1+k2).{\displaystyle \varphi _{X+Y}(t)=\varphi _{X}(t)\varphi _{Y}(t)=(1-\theta it)^{-k_{1}}(1-\theta it)^{-k_{2}}=\left(1-\theta it\right)^{-(k_{1}+k_{2})}.}

This is the characteristic function of the gamma distribution scale parameterθ and shape parameterk1 +k2, and we therefore conclude

X+Y∼Γ(k1+k2,θ){\displaystyle X+Y\sim \Gamma (k_{1}+k_{2},\theta )}

The result can be expanded ton independent gamma distributed random variables with the same scale parameter and we get

∀i∈{1,…,n}:Xi∼Γ(ki,θ)⇒∑i=1nXi∼Γ(∑i=1nki,θ).{\displaystyle \forall i\in \{1,\ldots ,n\}:X_{i}\sim \Gamma (k_{i},\theta )\qquad \Rightarrow \qquad \sum _{i=1}^{n}X_{i}\sim \Gamma \left(\sum _{i=1}^{n}k_{i},\theta \right).}

Entire characteristic functions

[edit]
[icon]
This sectionneeds expansion. You can help byadding to it.(December 2009)

As defined above, the argument of the characteristic function is treated as a real number: however, certain aspects of the theory of characteristic functions are advanced by extending the definition into the complex plane byanalytic continuation, in cases where this is possible.[25]

Related concepts

[edit]

Related concepts include themoment-generating function and theprobability-generating function. The characteristic function exists for all probability distributions. This is not the case for the moment-generating function.

The characteristic function is closely related to theFourier transform: the characteristic function of a probability density functionp(x) is thecomplex conjugate of thecontinuous Fourier transform ofp(x) (according to the usual convention; seecontinuous Fourier transform – other conventions).

φX(t)=⟨eitX⟩=∫Reitxp(x)dx=(∫Re−itxp(x)dx)¯=P(t)¯,{\displaystyle \varphi _{X}(t)=\langle e^{itX}\rangle =\int _{\mathbf {R} }e^{itx}p(x)\,dx={\overline {\left(\int _{\mathbf {R} }e^{-itx}p(x)\,dx\right)}}={\overline {P(t)}},}

whereP(t) denotes thecontinuous Fourier transform of the probability density functionp(x). Likewise,p(x) may be recovered fromφX(t) through the inverse Fourier transform:

p(x)=12π∫ReitxP(t)dt=12π∫ReitxφX(t)¯dt.{\displaystyle p(x)={\frac {1}{2\pi }}\int _{\mathbf {R} }e^{itx}P(t)\,dt={\frac {1}{2\pi }}\int _{\mathbf {R} }e^{itx}{\overline {\varphi _{X}(t)}}\,dt.}

Indeed, even when the random variable does not have a density, the characteristic function may be seen as the Fourier transform of the measure corresponding to the random variable.

Another related concept is the representation of probability distributions as elements of areproducing kernel Hilbert space via thekernel embedding of distributions. This framework may be viewed as a generalization of the characteristic function under specific choices of thekernel function.

See also

[edit]
  • Subindependence, a weaker condition than independence, that is defined in terms of characteristic functions.
  • Cumulant, a term of thecumulant generating functions, which are logs of the characteristic functions.

Notes

[edit]
  1. ^named after the French mathematicianPaul Lévy

References

[edit]

Citations

[edit]
  1. ^Lukacs (1970), p. 196.
  2. ^Shaw, W. T.; McCabe, J. (2009). "Monte Carlo sampling given a Characteristic Function: Quantile Mechanics in Momentum Space".arXiv:0903.1592 [q-fin.CP].
  3. ^Statistical and Adaptive Signal Processing (2005), p. 79
  4. ^Billingsley (1995), p. 345.
  5. ^Pinsky (2002).
  6. ^Bochner (1955).
  7. ^Andersen et al. (1995), Definition 1.10.
  8. ^Andersen et al. (1995), Definition 1.20.
  9. ^Sobczyk (2001), p. 20.
  10. ^Kotz & Nadarajah (2004), p. 37 using 1 as the number of degree of freedom to recover the Cauchy distribution
  11. ^Klenke, Achim (2013).Probability Theory (3rd ed.). Springer Cham. p. 331.ISBN 978-3-030-56401-8.
  12. ^Lukacs (1970), Corollary 1 to Theorem 2.3.1.
  13. ^"Joint characteristic function".www.statlect.com. Retrieved7 April 2018.
  14. ^Cuppens (1975), Theorem 2.6.9.
  15. ^abcShephard (1991a).
  16. ^Cuppens (1975), Theorem 2.3.2.
  17. ^Wendel (1961).
  18. ^Shephard (1991b).
  19. ^Lukacs (1970), p. 84.
  20. ^Paulson, Holcomb & Leitch (1975).
  21. ^Heathcote (1977).
  22. ^Yu (2004).
  23. ^Ansari, Scarlett & Soh (2020).
  24. ^Li et al. (2020).
  25. ^Lukacs (1970), Chapter 7.

Sources

[edit]

External links

[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Characteristic_function_(probability_theory)&oldid=1323195670"
Category:
Hidden categories:
[8]ページ先頭
©2009-2025 Movatter.jp