Movatterモバイル変換

[0]ホーム
URL:

Jump to content
WikipediaThe Free Encyclopedia
Search

Beta distribution

From Wikipedia, the free encyclopedia
Probability distribution
Not to be confused withBeta function.
Beta
Probability density function
Probability density function for the beta distribution
Cumulative distribution function
Cumulative distribution function for the beta distribution
NotationBeta(α,β)
Parametersα > 0shape (real)
β > 0shape (real)
Supportx∈[0,1]{\displaystyle x\in [0,1]\!} orx∈(0,1){\displaystyle x\in (0,1)\!}
PDFxα−1(1−x)β−1B(α,β){\displaystyle {\frac {x^{\alpha -1}(1-x)^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}\!}
whereB(α,β)=Γ(α)Γ(β)Γ(α+β){\displaystyle \mathrm {B} (\alpha ,\beta )={\frac {\Gamma (\alpha )\Gamma (\beta )}{\Gamma (\alpha +\beta )}}} andΓ{\displaystyle \Gamma } is theGamma function.
CDF

Ix(α,β){\displaystyle I_{x}(\alpha ,\beta )\!}

(theregularized incomplete beta function)
Mean

E⁡[X]=αα+β{\displaystyle \operatorname {E} [X]={\frac {\alpha }{\alpha +\beta }}\!}
E⁡[ln⁡X]=ψ(α)−ψ(α+β){\displaystyle \operatorname {E} [\ln X]=\psi (\alpha )-\psi (\alpha +\beta )\!}

E⁡[Xln⁡X]=αα+β[ψ(α+1)−ψ(α+β+1)]{\displaystyle \operatorname {E} [X\,\ln X]={\frac {\alpha }{\alpha +\beta }}\,\left[\psi (\alpha +1)-\psi (\alpha +\beta +1)\right]\!}
(see section:Geometric mean)

whereψ{\displaystyle \psi } is thedigamma function
MedianI12[−1](α,β) (in general) ≈α−13α+β−23 for α,β>1{\displaystyle {\begin{matrix}I_{\frac {1}{2}}^{[-1]}(\alpha ,\beta ){\text{ (in general) }}\\[0.5em]\approx {\frac {\alpha -{\tfrac {1}{3}}}{\alpha +\beta -{\tfrac {2}{3}}}}{\text{ for }}\alpha ,\beta >1\end{matrix}}}
Mode

α−1α+β−2{\displaystyle {\frac {\alpha -1}{\alpha +\beta -2}}\!} forα,β > 1

Any value in the domain forα =β = 1

No mode ifα<1 orβ<1. Density diverges

at 0 forα ≤ 1, and at 1 ifβ ≤ 1
Variancevar⁡[X]=αβ(α+β)2(α+β+1){\displaystyle \operatorname {var} [X]={\frac {\alpha \beta }{(\alpha +\beta )^{2}(\alpha +\beta +1)}}\!}
var⁡[ln⁡X]=ψ1(α)−ψ1(α+β){\displaystyle \operatorname {var} [\ln X]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )\!}
(seetrigamma function and see section:Geometric variance)
Skewness2(β−α)α+β+1(α+β+2)αβ{\displaystyle {\frac {2\,(\beta -\alpha ){\sqrt {\alpha +\beta +1}}}{(\alpha +\beta +2){\sqrt {\alpha \beta }}}}}
Excess kurtosis6[(α−β)2(α+β+1)−αβ(α+β+2)]αβ(α+β+2)(α+β+3){\displaystyle {\frac {6[(\alpha -\beta )^{2}(\alpha +\beta +1)-\alpha \beta (\alpha +\beta +2)]}{\alpha \beta (\alpha +\beta +2)(\alpha +\beta +3)}}}
Entropyln⁡B(α,β)−(α−1)ψ(α)−(β−1)ψ(β)+(α+β−2)ψ(α+β){\displaystyle {\begin{matrix}\ln \mathrm {B} (\alpha ,\beta )-(\alpha -1)\psi (\alpha )-(\beta -1)\psi (\beta )\\[0.5em]{}+(\alpha +\beta -2)\psi (\alpha +\beta )\end{matrix}}}
MGF1+∑k=1∞(∏r=0k−1α+rα+β+r)tkk!{\displaystyle 1+\sum _{k=1}^{\infty }\left(\prod _{r=0}^{k-1}{\frac {\alpha +r}{\alpha +\beta +r}}\right){\frac {t^{k}}{k!}}}
CF1F1(α;α+β;it){\displaystyle {}_{1}F_{1}(\alpha ;\alpha +\beta ;i\,t)\!} (seeConfluent hypergeometric function)
Fisher information[var⁡[ln⁡X]cov⁡[ln⁡X,ln⁡(1−X)]cov⁡[ln⁡X,ln⁡(1−X)]var⁡[ln⁡(1−X)]]{\displaystyle {\begin{bmatrix}\operatorname {var} [\ln X]&\operatorname {cov} [\ln X,\ln(1-X)]\\\operatorname {cov} [\ln X,\ln(1-X)]&\operatorname {var} [\ln(1-X)]\end{bmatrix}}}
see section:Fisher information matrix
Method of momentsα=(E[X](1−E[X])V[X]−1)E[X]{\displaystyle \alpha =\left({\frac {E[X](1-E[X])}{V[X]}}-1\right)E[X]}
β=(E[X](1−E[X])V[X]−1)(1−E[X]){\displaystyle \beta =\left({\frac {E[X](1-E[X])}{V[X]}}-1\right)(1-E[X])}

Inprobability theory andstatistics, thebeta distribution is a family of continuousprobability distributions defined on the interval [0, 1] or (0, 1) in terms of two positiveparameters, denoted byalpha (α) andbeta (β), that appear as exponents of the variable and its complement to 1, respectively, and control theshape of the distribution.

The beta distribution has been applied to model the behavior ofrandom variables limited to intervals of finite length in a wide variety of disciplines. The beta distribution is a suitable model for the random behavior of percentages and proportions.

InBayesian inference, the beta distribution is theconjugate prior probability distribution for theBernoulli,binomial,negative binomial, andgeometric distributions.

The formulation of the beta distribution discussed here is also known as thebeta distribution of the first kind, whereasbeta distribution of the second kind is an alternative name for thebeta prime distribution. The generalization to multiple variables is called aDirichlet distribution.

Definitions

[edit]

Probability density function

[edit]
An animation of the beta distribution for different values of its parameters.

Theprobability density function (PDF) of the beta distribution, for0≤x≤1{\displaystyle 0\leq x\leq 1} or0<x<1{\displaystyle 0<x<1}, and shape parametersα{\displaystyle \alpha },β>0{\displaystyle \beta >0}, is apower function of the variablex{\displaystyle x} and of itsreflection(1−x){\displaystyle (1-x)} as follows:

f(x;α,β)=constant⋅xα−1(1−x)β−1=xα−1(1−x)β−1∫01uα−1(1−u)β−1du=Γ(α+β)Γ(α)Γ(β)xα−1(1−x)β−1=1B(α,β)xα−1(1−x)β−1{\displaystyle {\begin{aligned}f(x;\alpha ,\beta )&=\mathrm {constant} \cdot x^{\alpha -1}(1-x)^{\beta -1}\\[3pt]&={\frac {x^{\alpha -1}(1-x)^{\beta -1}}{\displaystyle \int _{0}^{1}u^{\alpha -1}(1-u)^{\beta -1}\,du}}\\[6pt]&={\frac {\Gamma (\alpha +\beta )}{\Gamma (\alpha )\Gamma (\beta )}}\,x^{\alpha -1}(1-x)^{\beta -1}\\[6pt]&={\frac {1}{\mathrm {B} (\alpha ,\beta )}}x^{\alpha -1}(1-x)^{\beta -1}\end{aligned}}}

whereΓ(z){\displaystyle \Gamma (z)} is thegamma function. Thebeta function,B{\displaystyle \mathrm {B} }, is anormalization constant to ensure that the total probability is 1. In the above equationsx{\displaystyle x} is arealization—an observed value that actually occurred—of arandom variableX{\displaystyle X}.

Several authors, includingN. L. Johnson andS. Kotz,[1] use the symbolsp{\displaystyle p} andq{\displaystyle q} (instead ofα{\displaystyle \alpha } andβ{\displaystyle \beta }) for the shape parameters of the beta distribution, reminiscent of the symbols traditionally used for the parameters of theBernoulli distribution, because the beta distribution approaches the Bernoulli distribution in the limit when both shape parametersα{\displaystyle \alpha } andβ{\displaystyle \beta } approach zero.

In the following, a random variableX{\displaystyle X} beta-distributed with parametersα{\displaystyle \alpha } andβ{\displaystyle \beta } will be denoted by:[2][3]

X∼Beta⁡(α,β){\displaystyle X\sim \operatorname {Beta} (\alpha ,\beta )}

Other notations for beta-distributed random variables used in the statistical literature areX∼Be(α,β){\displaystyle X\sim {\mathcal {B}}e(\alpha ,\beta )}[4] andX∼βα,β{\displaystyle X\sim \beta _{\alpha ,\beta }}.[5]

Cumulative distribution function

[edit]
CDF for symmetric beta distribution vs.x and α = β
CDF for skewed beta distribution vs.x and β = 5α

Thecumulative distribution function is

F(x;α,β)=B(x;α,β)B(α,β)=Ix(α,β){\displaystyle F(x;\alpha ,\beta )={\frac {\mathrm {B} {}(x;\alpha ,\beta )}{\mathrm {B} {}(\alpha ,\beta )}}=I_{x}(\alpha ,\beta )}

whereB(x;α,β){\displaystyle \mathrm {B} (x;\alpha ,\beta )} is theincomplete beta function andIx(α,β){\displaystyle I_{x}(\alpha ,\beta )} is theregularized incomplete beta function.

For positive integersα andβ, the cumulative distribution function of a beta distribution can be expressed in terms of the cumulative distribution function of abinomial distribution with[6]

Fbeta(x;α,β)=Fbinomial(β−1;α+β−1,1−x).{\displaystyle F_{\text{beta}}(x;\alpha ,\beta )=F_{\text{binomial}}(\beta -1;\alpha +\beta -1,1-x).}

Alternative parameterizations

[edit]

Two parameters

[edit]
Mean and sample size
[edit]

The beta distribution may also be reparameterized in terms of its meanμ(0 <μ < 1) and the sum of the two shape parametersν =α +β > 0([3] p. 83). Denoting by αPosterior and βPosterior the shape parameters of the posterior beta distribution resulting from applying Bayes' theorem to a binomial likelihood function and a prior probability, the interpretation of the addition of both shape parameters to be sample size =ν =α·Posterior +β·Posterior is only correct for the Haldane prior probability Beta(0,0). Specifically, for the Bayes (uniform) prior Beta(1,1) the correct interpretation would be sample size =α·Posterior +β Posterior − 2, orν = (sample size) + 2. For sample size much larger than 2, the difference between these two priors becomes negligible. (See sectionBayesian inference for further details.)ν =α +β is referred to as the "sample size" of a beta distribution, but one should remember that it is, strictly speaking, the "sample size" of a binomial likelihood function only when using a Haldane Beta(0,0) prior in Bayes' theorem.

This parametrization may be useful in Bayesian parameter estimation. For example, one may administer a test to a number of individuals. If it is assumed that each person's score (0 ≤θ ≤ 1) is drawn from a population-level beta distribution, then an important statistic is the mean of this population-level distribution. The mean and sample size parameters are related to the shape parametersα andβ via[3]

α =μν,β = (1 −μ)ν

Under thisparametrization, one may place anuninformative prior probability over the mean, and a vague prior probability (such as anexponential orgamma distribution) over the positive reals for the sample size, if they are independent, and prior data and/or beliefs justify it.

Mode and concentration
[edit]

Concave beta distributions, which haveα,β>1{\displaystyle \alpha ,\beta >1}, can be parametrized in terms of mode and "concentration". The mode,ω=α−1α+β−2{\displaystyle \omega ={\frac {\alpha -1}{\alpha +\beta -2}}}, and concentration,κ=α+β{\displaystyle \kappa =\alpha +\beta }, can be used to define the usual shape parameters as follows:[7]α=ω(κ−2)+1β=(1−ω)(κ−2)+1{\displaystyle {\begin{aligned}\alpha &=\omega (\kappa -2)+1\\\beta &=(1-\omega )(\kappa -2)+1\end{aligned}}}For the mode,0<ω<1{\displaystyle 0<\omega <1}, to be well-defined, we needα,β>1{\displaystyle \alpha ,\beta >1}, or equivalentlyκ>2{\displaystyle \kappa >2}. If instead we define the concentration asc=α+β−2{\displaystyle c=\alpha +\beta -2}, the condition simplifies toc>0{\displaystyle c>0} and the beta density atα=1+cω{\displaystyle \alpha =1+c\omega } andβ=1+c(1−ω){\displaystyle \beta =1+c(1-\omega )} can be written as:f(x;ω,c)=xcω(1−x)c(1−ω)B(1+cω,1+c(1−ω)){\displaystyle f(x;\omega ,c)={\frac {x^{c\omega }(1-x)^{c(1-\omega )}}{\mathrm {B} {\bigl (}1+c\omega ,1+c(1-\omega ){\bigr )}}}}wherec{\displaystyle c} directly scales thesufficient statistics,log⁡(x){\displaystyle \log(x)} andlog⁡(1−x){\displaystyle \log(1-x)}. Note also that in the limit,c→0{\displaystyle c\to 0}, the distribution becomes flat.

Mean and variance
[edit]

Solving the system of (coupled) equations given in the above sections as the equations for the mean and the variance of the beta distribution in terms of the original parametersα andβ, one can express theα andβ parameters in terms of the mean (μ) and the variance (var):

ν=α+β=μ(1−μ)var−1, where ν=(α+β)>0, therefore: var<μ(1−μ)α=μν=μ(μ(1−μ)var−1), if var<μ(1−μ)β=(1−μ)ν=(1−μ)(μ(1−μ)var−1), if var<μ(1−μ).{\displaystyle {\begin{aligned}\nu &=\alpha +\beta ={\frac {\mu (1-\mu )}{\mathrm {var} }}-1,{\text{ where }}\nu =(\alpha +\beta )>0,{\text{ therefore: }}{\text{var}}<\mu (1-\mu )\\\alpha &=\mu \nu =\mu \left({\frac {\mu (1-\mu )}{\text{var}}}-1\right),{\text{ if }}{\text{var}}<\mu (1-\mu )\\\beta &=(1-\mu )\nu =(1-\mu )\left({\frac {\mu (1-\mu )}{\text{var}}}-1\right),{\text{ if }}{\text{var}}<\mu (1-\mu ).\end{aligned}}}

Thisparametrization of the beta distribution may lead to a more intuitive understanding than the one based on the original parametersα andβ. For example, by expressing the mode, skewness, excess kurtosis and differential entropy in terms of the mean and the variance:

Four parameters

[edit]

A beta distribution with the two shape parametersα andβ is supported on the range [0,1] or (0,1). It is possible to alter the location and scale of the distribution by introducing two further parameters representing the minimum,a, and maximumc (c >a), values of the distribution,[1] by a linear transformation substituting the non-dimensional variablex in terms of the new variabley (with support [a,c] or (a,c)) and the parametersa andc:

y=x(c−a)+a, therefore x=y−ac−a.{\displaystyle y=x(c-a)+a,{\text{ therefore }}x={\frac {y-a}{c-a}}.}

Theprobability density function of the four parameter beta distribution is equal to the two parameter distribution, scaled by the range (c − a), (so that the total area under the density curve equals a probability of one), and with the "y" variable shifted and scaled as follows:f(y;α,β,a,c)=f(x;α,β)c−a=(y−ac−a)α−1(c−yc−a)β−1(c−a)B(α,β)=(y−a)α−1(c−y)β−1(c−a)α+β−1B(α,β).{\displaystyle {\begin{aligned}f(y;\alpha ,\beta ,a,c)={\frac {f(x;\alpha ,\beta )}{c-a}}&={\frac {\left({\frac {y-a}{c-a}}\right)^{\alpha -1}\left({\frac {c-y}{c-a}}\right)^{\beta -1}}{(c-a)B(\alpha ,\beta )}}\\[1ex]&={\frac {(y-a)^{\alpha -1}(c-y)^{\beta -1}}{(c-a)^{\alpha +\beta -1}B(\alpha ,\beta )}}.\end{aligned}}}

That a random variableY is beta-distributed with four parametersα,β,a, andc will be denoted by:

Y∼Beta⁡(α,β,a,c).{\displaystyle Y\sim \operatorname {Beta} (\alpha ,\beta ,a,c).}

Some measures of central location are scaled (by (c − a)) and shifted (bya), as follows:

μY=μX(c−a)+a=αα+β(c−a)+a=αc+βaα+β{\displaystyle {\begin{aligned}\mu _{Y}&=\mu _{X}(c-a)+a\\[1ex]&={\frac {\alpha }{\alpha +\beta }}\left(c-a\right)+a={\frac {\alpha c+\beta a}{\alpha +\beta }}\end{aligned}}}

mode(Y)=mode(X)(c−a)+a=α−1α+β−2(c−a)+a=(α−1)c+(β−1)aα+β−2 , if α,β>1{\displaystyle {\begin{aligned}{\text{mode}}(Y)&={\text{mode}}(X)(c-a)+a\\[1ex]&={\frac {\alpha -1}{\alpha +\beta -2}}\left(c-a\right)+a\\[1ex]&={\frac {(\alpha -1)c+(\beta -1)a}{\alpha +\beta -2}}\ ,&{\text{ if }}\alpha ,\,\beta >1\end{aligned}}}

median(Y)=median(X)(c−a)+a=I12[−1](α,β)(c−a)+a{\displaystyle {\begin{aligned}{\text{median}}(Y)&={\text{median}}(X)(c-a)+a\\[1ex]&=I_{\frac {1}{2}}^{[-1]}(\alpha ,\beta )\left(c-a\right)+a\end{aligned}}}

Note: the geometric mean and harmonic mean cannot be transformed by a linear transformation in the way that the mean, median and mode can.

The shape parameters ofY can be written in term of its mean and variance as

α=(a−μY)(ac−aμY−cμY+μY2+σY2)σY2(c−a)β=−(c−μY)(ac−aμY−cμY+μY2+σY2)σY2(c−a){\displaystyle {\begin{aligned}\alpha &={\frac {\left(a-\mu _{Y}\right)\left(a\,c-a\,\mu _{Y}-c\,\mu _{Y}+\mu _{Y}^{2}+\sigma _{Y}^{2}\right)}{\sigma _{Y}^{2}(c-a)}}\\\beta &=-{\frac {\left(c-\mu _{Y}\right)\left(a\,c-a\,\mu _{Y}-c\,\mu _{Y}+\mu _{Y}^{2}+\sigma _{Y}^{2}\right)}{\sigma _{Y}^{2}(c-a)}}\end{aligned}}}

The statistical dispersion measures are scaled (they do not need to be shifted because they are already centered on the mean) by the range (c − a), linearly for the mean deviation and nonlinearly for the variance:

(mean deviation around mean)(Y)=((mean deviation around mean)(X))(c−a)=2ααββB(α,β)(α+β)α+β+1(c−a){\displaystyle {\begin{aligned}&{\text{(mean deviation around mean)}}(Y)\\[1ex]&=({\text{(mean deviation around mean)}}(X))(c-a)\\&={\frac {2\alpha ^{\alpha }\beta ^{\beta }}{\mathrm {B} (\alpha ,\beta )(\alpha +\beta )^{\alpha +\beta +1}}}(c-a)\end{aligned}}}var(Y)=var(X)(c−a)2=αβ(c−a)2(α+β)2(α+β+1).{\displaystyle {\text{var}}(Y)={\text{var}}(X)(c-a)^{2}={\frac {\alpha \beta (c-a)^{2}}{(\alpha +\beta )^{2}(\alpha +\beta +1)}}.}

Since theskewness andexcess kurtosis are non-dimensional quantities (asmoments centered on the mean and normalized by thestandard deviation), they are independent of the parametersa andc, and therefore equal to the expressions given above in terms ofX (with support [0,1] or (0,1)):

skewness(Y)=skewness(X)=2(β−α)α+β+1(α+β+2)αβ.{\displaystyle {\text{skewness}}(Y)={\text{skewness}}(X)={\frac {2(\beta -\alpha ){\sqrt {\alpha +\beta +1}}}{(\alpha +\beta +2){\sqrt {\alpha \beta }}}}.}

kurtosis excess(Y)=kurtosis excess(X)=6[(α−β)2(α+β+1)−αβ(α+β+2)]αβ(α+β+2)(α+β+3){\displaystyle {\text{kurtosis excess}}(Y)={\text{kurtosis excess}}(X)={\frac {6\left[(\alpha -\beta )^{2}(\alpha +\beta +1)-\alpha \beta (\alpha +\beta +2)\right]}{\alpha \beta (\alpha +\beta +2)(\alpha +\beta +3)}}}

Properties

[edit]

Measures of central tendency

[edit]

Mode

[edit]

Themode of a beta distributedrandom variableX withα,β > 1 is the most likely value of the distribution (corresponding to the peak in the PDF), and is given by the following expression:[1]

α−1α+β−2.{\displaystyle {\frac {\alpha -1}{\alpha +\beta -2}}.}

When both parameters are less than one (α,β < 1), this is the anti-mode: the lowest point of the probability density curve.[8]

Lettingα =β, the expression for the mode simplifies to 1/2, showing that forα =β > 1 the mode (resp. anti-mode whenα,β < 1), is at the center of the distribution: it is symmetric in those cases. SeeShapes section in this article for a full list of mode cases, for arbitrary values ofα andβ. For several of these cases, the maximum value of the density function occurs at one or both ends. In some cases the (maximum) value of the density function occurring at the end is finite. For example, in the case ofα = 2,β = 1 (orα = 1,β = 2), the density function becomes aright-triangle distribution which is finite at both ends. In several other cases there is asingularity at one end, where the value of the density function approaches infinity. For example, in the caseα =β = 1/2, the beta distribution simplifies to become thearcsine distribution. There is debate among mathematicians about some of these cases and whether the ends (x = 0, andx = 1) can be calledmodes or not.[9][2]

Mode for beta distribution for 1 ≤α ≤ 5 and 1 ≤ β ≤ 5
  • Whether the ends are part of thedomain of the density function
  • Whether asingularity can ever be called amode
  • Whether cases with two maxima should be calledbimodal

Median

[edit]
Median for beta distribution for 0 ≤α ≤ 5 and 0 ≤β ≤ 5
(Mean–median) for beta distribution versus alpha and beta from 0 to 2

The median of the beta distribution is the unique real numberx=I1/2[−1](α,β){\displaystyle x=I_{1/2}^{[-1]}(\alpha ,\beta )} for which theregularized incomplete beta functionIx(α,β)=12{\displaystyle I_{x}(\alpha ,\beta )={\tfrac {1}{2}}}. There is no generalclosed-form expression for themedian of the beta distribution for arbitrary values ofα andβ.Closed-form expressions for particular values of the parametersα andβ follow:[citation needed]

The following are the limits with one parameter finite (non-zero) and the other approaching these limits:[citation needed]

limβ→0median=limα→∞median=1,limα→0median=limβ→∞median=0.{\displaystyle {\begin{aligned}\lim _{\beta \to 0}{\text{median}}=\lim _{\alpha \to \infty }{\text{median}}=1,\\\lim _{\alpha \to 0}{\text{median}}=\lim _{\beta \to \infty }{\text{median}}=0.\end{aligned}}}

A reasonable approximation of the value of the median of the beta distribution, for both α and β greater or equal to one, is given by the formula[10]

median≈α−13α+β−23 for α,β≥1.{\displaystyle {\text{median}}\approx {\frac {\alpha -{\tfrac {1}{3}}}{\alpha +\beta -{\tfrac {2}{3}}}}{\text{ for }}\alpha ,\beta \geq 1.}

Whenα,β ≥ 1, therelative error (theabsolute error divided by the median) in this approximation is less than 4% and for bothα ≥ 2 andβ ≥ 2 it is less than 1%. Theabsolute error divided by the difference between the mean and the mode is similarly small:

Abs[(Median-Appr.)/Median] for beta distribution for 1 ≤ α ≤ 5 and 1 ≤ β ≤ 5Abs[(Median-Appr.)/(Mean-Mode)] for beta distribution for 1 ≤ α ≤ 5 and 1 ≤ β ≤ 5

Mean

[edit]
Mean for beta distribution for0 ≤α ≤ 5 and0 ≤β ≤ 5

Theexpected value (mean) (μ) of a beta distributionrandom variableX with two parametersα andβ is a function of only the ratioβ/α of these parameters:[1]

μ=E⁡[X]=∫01xf(x;α,β)dx=∫01xxα−1(1−x)β−1B(α,β)dx=αα+β=11+βα{\displaystyle {\begin{aligned}\mu =\operatorname {E} [X]&=\int _{0}^{1}xf(x;\alpha ,\beta )\,dx\\&=\int _{0}^{1}x\,{\frac {x^{\alpha -1}(1-x)^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}\,dx\\&={\frac {\alpha }{\alpha +\beta }}\\&={\frac {1}{1+{\frac {\beta }{\alpha }}}}\end{aligned}}}

Lettingα =β in the above expression one obtainsμ = 1/2, showing that forα =β the mean is at the center of the distribution: it is symmetric. Also, the following limits can be obtained from the above expression:

limβα→0μ=1limβα→∞μ=0{\displaystyle {\begin{aligned}\lim _{{\frac {\beta }{\alpha }}\to 0}\mu =1\\\lim _{{\frac {\beta }{\alpha }}\to \infty }\mu =0\end{aligned}}}

Therefore, forβ/α → 0, or forα/β → ∞, the mean is located at the right end,x = 1. For these limit ratios, the beta distribution becomes a one-pointdegenerate distribution with aDirac delta function spike at the right end,x = 1, with probability 1, and zero probability everywhere else. There is 100% probability (absolute certainty) concentrated at the right end,x = 1.

Similarly, forβ/α → ∞, or forα/β → 0, the mean is located at the left end,x = 0. The beta distribution becomes a 1-pointDegenerate distribution with aDirac delta function spike at the left end,x = 0, with probability 1, and zero probability everywhere else. There is 100% probability (absolute certainty) concentrated at the left end,x = 0. Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limβ→0μ=limα→∞μ=1limα→0μ=limβ→∞μ=0{\displaystyle {\begin{aligned}\lim _{\beta \to 0}\mu =\lim _{\alpha \to \infty }\mu =1\\\lim _{\alpha \to 0}\mu =\lim _{\beta \to \infty }\mu =0\end{aligned}}}

While for typical unimodal distributions (with centrally located modes, inflexion points at both sides of the mode, and longer tails) (with Beta(α, β) such thatα,β > 2) it is known that the sample mean (as an estimate of location) is not asrobust as the sample median, the opposite is the case for uniform or "U-shaped" bimodal distributions (with Beta(α, β) such thatα,β ≤ 1), with the modes located at the ends of the distribution. As Mosteller and Tukey remark ([11] p. 207) "the average of the two extreme observations uses all the sample information. This illustrates how, for short-tailed distributions, the extreme observations should get more weight." By contrast, it follows that the median of "U-shaped" bimodal distributions with modes at the edge of the distribution (with Beta(α, β) such thatα,β ≤ 1) is not robust, as the sample median drops the extreme sample observations from consideration. A practical application of this occurs for example forrandom walks, since the probability for the time of the last visit to the origin in a random walk is distributed as thearcsine distribution Beta(1/2, 1/2):[5][12] the mean of a number ofrealizations of a random walk is a much more robust estimator than the median (which is an inappropriate sample measure estimate in this case).

Geometric mean

[edit]
(Mean − GeometricMean) for beta distribution versusα andβ from 0 to 2, showing the asymmetry betweenα andβ for the geometric mean
Geometric means for beta distribution Purple =G(x), Yellow =G(1 − x), smaller valuesα andβ in front
Geometric means for beta distribution. purple =G(x), yellow =G(1 − x), larger valuesα andβ in front

The logarithm of thegeometric meanGX of a distribution withrandom variableX is the arithmetic mean of ln(X), or, equivalently, its expected value:

ln⁡GX=E⁡[ln⁡X]{\displaystyle \ln G_{X}=\operatorname {E} [\ln X]}

For a beta distribution, the expected value integral gives:

E⁡[ln⁡X]=∫01ln⁡xf(x;α,β)dx=∫01ln⁡xxα−1(1−x)β−1B(α,β)dx=1B(α,β)∫01∂xα−1(1−x)β−1∂αdx=1B(α,β)∂∂α∫01xα−1(1−x)β−1dx=1B(α,β)∂B(α,β)∂α=∂ln⁡B(α,β)∂α=∂ln⁡Γ(α)∂α−∂ln⁡Γ(α+β)∂α=ψ(α)−ψ(α+β){\displaystyle {\begin{aligned}\operatorname {E} [\ln X]&=\int _{0}^{1}\ln x\,f(x;\alpha ,\beta )\,dx\\[4pt]&=\int _{0}^{1}\ln x\,{\frac {x^{\alpha -1}(1-x)^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}\,dx\\[4pt]&={\frac {1}{\mathrm {B} (\alpha ,\beta )}}\,\int _{0}^{1}{\frac {\partial x^{\alpha -1}(1-x)^{\beta -1}}{\partial \alpha }}\,dx\\[4pt]&={\frac {1}{\mathrm {B} (\alpha ,\beta )}}{\frac {\partial }{\partial \alpha }}\int _{0}^{1}x^{\alpha -1}(1-x)^{\beta -1}\,dx\\[4pt]&={\frac {1}{\mathrm {B} (\alpha ,\beta )}}{\frac {\partial \mathrm {B} (\alpha ,\beta )}{\partial \alpha }}\\[4pt]&={\frac {\partial \ln \mathrm {B} (\alpha ,\beta )}{\partial \alpha }}\\[4pt]&={\frac {\partial \ln \Gamma (\alpha )}{\partial \alpha }}-{\frac {\partial \ln \Gamma (\alpha +\beta )}{\partial \alpha }}\\[4pt]&=\psi (\alpha )-\psi (\alpha +\beta )\end{aligned}}}

whereψ is thedigamma function.

Therefore, the geometric mean of a beta distribution with shape parametersα andβ is the exponential of the digamma functions ofα andβ as follows:

GX=eE⁡[ln⁡X]=eψ(α)−ψ(α+β){\displaystyle G_{X}=e^{\operatorname {E} [\ln X]}=e^{\psi (\alpha )-\psi (\alpha +\beta )}}

While for a beta distribution with equal shape parametersα =β, it follows that skewness = 0 and mode = mean = median = 1/2, the geometric mean is less than 1/2:0 <GX < 1/2. The reason for this is that the logarithmic transformation strongly weights the values ofX close to zero, as ln(X) strongly tends towards negative infinity asX approaches zero, while ln(X) flattens towards zero asX → 1.

Along a lineα =β, the following limits apply:

limα=β→0GX=0limα=β→∞GX=12{\displaystyle {\begin{aligned}&\lim _{\alpha =\beta \to 0}G_{X}=0\\&\lim _{\alpha =\beta \to \infty }G_{X}={\tfrac {1}{2}}\end{aligned}}}

Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limβ→0GX=limα→∞GX=1limα→0GX=limβ→∞GX=0{\displaystyle {\begin{aligned}\lim _{\beta \to 0}G_{X}=\lim _{\alpha \to \infty }G_{X}=1\\\lim _{\alpha \to 0}G_{X}=\lim _{\beta \to \infty }G_{X}=0\end{aligned}}}

The accompanying plot shows the difference between the mean and the geometric mean for shape parametersα andβ from zero to 2. Besides the fact that the difference between them approaches zero asα andβ approach infinity and that the difference becomes large for values ofα andβ approaching zero, one can observe an evident asymmetry of the geometric mean with respect to the shape parametersα andβ. The difference between the geometric mean and the mean is larger for small values ofα in relation toβ than when exchanging the magnitudes ofβ andα.

N. L.Johnson andS. Kotz[1] suggest the logarithmic approximation to the digamma functionψ(α) ≈ ln(α − 1/2) which results in the following approximation to the geometric mean:

GX≈α−12α+β−12 if α,β>1.{\displaystyle G_{X}\approx {\frac {\alpha \,-{\frac {1}{2}}}{\alpha +\beta -{\frac {1}{2}}}}{\text{ if }}\alpha ,\beta >1.}

Numerical values for therelative error in this approximation follow: [(α =β = 1): 9.39%]; [(α =β = 2): 1.29%]; [(α = 2,β = 3): 1.51%]; [(α = 3,β = 2): 0.44%]; [(α =β = 3): 0.51%]; [(α =β = 4): 0.26%]; [(α = 3,β = 4): 0.55%]; [(α = 4,β = 3): 0.24%].

Similarly, one can calculate the value of shape parameters required for the geometric mean to equal 1/2. Given the value of the parameterβ, what would be the value of the other parameter, α, required for the geometric mean to equal 1/2?. The answer is that (forβ > 1), the value ofα required tends towardsβ + 1/2 asβ → ∞. For example, all these couples have the same geometric mean of 1/2: [β = 1,α = 1.4427], [β = 2,α = 2.46958], [β = 3,α = 3.47943], [β = 4,α = 4.48449], [β = 5,α = 5.48756], [β = 10,α = 10.4938], [β = 100,α = 100.499].

The fundamental property of the geometric mean, which can be proven to be false for any other mean, is

G(XiYi)=G(Xi)G(Yi){\displaystyle G{\left({\frac {X_{i}}{Y_{i}}}\right)}={\frac {G(X_{i})}{G(Y_{i})}}}

This makes the geometric mean the only correct mean when averagingnormalized results, that is results that are presented as ratios to reference values.[13] This is relevant because the beta distribution is a suitable model for the random behavior of percentages and it is particularly suitable to the statistical modelling of proportions. The geometric mean plays a central role in maximum likelihood estimation, see section "Parameter estimation, maximum likelihood." Actually, when performing maximum likelihood estimation, besides thegeometric meanGX based on the random variable X, also another geometric mean appears naturally: thegeometric mean based on the linear transformation ––(1 −X), the mirror-image ofX, denoted byG(1−X):

G1−X=eE⁡[ln⁡(1−X)]=eψ(β)−ψ(α+β){\displaystyle G_{1-X}=e^{\operatorname {E} [\ln(1-X)]}=e^{\psi (\beta )-\psi (\alpha +\beta )}}

Along a lineα =β, the following limits apply:

limα=β→0G1−X=0limα=β→∞G1−X=12{\displaystyle {\begin{aligned}&\lim _{\alpha =\beta \to 0}G_{1-X}=0\\&\lim _{\alpha =\beta \to \infty }G_{1-X}={\tfrac {1}{2}}\end{aligned}}}

Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limβ→0G(1−X)=limα→∞G(1−X)=0limα→0G(1−X)=limβ→∞G(1−X)=1{\displaystyle {\begin{aligned}\lim _{\beta \to 0}G_{(1-X)}=\lim _{\alpha \to \infty }G_{(1-X)}=0\\\lim _{\alpha \to 0}G_{(1-X)}=\lim _{\beta \to \infty }G_{(1-X)}=1\end{aligned}}}

It has the following approximate value:

G(1−X)≈β−12α+β−12 if α,β>1.{\displaystyle G_{(1-X)}\approx {\frac {\beta -{\frac {1}{2}}}{\alpha +\beta -{\frac {1}{2}}}}{\text{ if }}\alpha ,\beta >1.}

Although bothGX andG1−X are asymmetric, in the case that both shape parameters are equalα =β, the geometric means are equal:GX =G(1−X). This equality follows from the following symmetry displayed between both geometric means:

GX(B(α,β))=G1−X(B(β,α)).{\displaystyle G_{X}(\mathrm {B} (\alpha ,\beta ))=G_{1-X}(\mathrm {B} (\beta ,\alpha )).}

Harmonic mean

[edit]
Harmonic mean for beta distribution for 0 < α < 5 and 0 < β < 5
Harmonic mean for beta distribution versusα andβ from 0 to 2
Harmonic means for beta distribution Purple =H(X), Yellow =H(1 − X), smaller valuesα andβ in front
Harmonic means for beta distribution: purple =H(X), yellow =H(1 − X), larger valuesα andβ in front

The inverse of theharmonic mean (HX) of a distribution withrandom variableX is the arithmetic mean of 1/X, or, equivalently, its expected value. Therefore, theharmonic mean (HX) of a beta distribution with shape parametersα andβ is:

HX=1E⁡[1X]=1∫01f(x;α,β)xdx=1∫01xα−1(1−x)β−1xB(α,β)dx=α−1α+β−1 if α>1 and β>0{\displaystyle {\begin{aligned}H_{X}&={\frac {1}{\operatorname {E} \left[{\frac {1}{X}}\right]}}\\&={\frac {1}{\int _{0}^{1}{\frac {f(x;\alpha ,\beta )}{x}}\,dx}}\\&={\frac {1}{\int _{0}^{1}{\frac {x^{\alpha -1}(1-x)^{\beta -1}}{x\mathrm {B} (\alpha ,\beta )}}\,dx}}\\&={\frac {\alpha -1}{\alpha +\beta -1}}{\text{ if }}\alpha >1{\text{ and }}\beta >0\\\end{aligned}}}

Theharmonic mean (HX) of a beta distribution withα < 1 is undefined, because its defining expression is not bounded in [0, 1] for shape parameterα less than unity.

Lettingα =β in the above expression one obtains

HX=α−12α−1,{\displaystyle H_{X}={\frac {\alpha -1}{2\alpha -1}},}

showing that forα =β the harmonic mean ranges from 0, forα =β = 1, to 1/2, forα =β → ∞.

Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limα→0HX is undefinedlimα→1HX=limβ→∞HX=0limβ→0HX=limα→∞HX=1{\displaystyle {\begin{aligned}&\lim _{\alpha \to 0}H_{X}{\text{ is undefined}}\\&\lim _{\alpha \to 1}H_{X}=\lim _{\beta \to \infty }H_{X}=0\\&\lim _{\beta \to 0}H_{X}=\lim _{\alpha \to \infty }H_{X}=1\end{aligned}}}

The harmonic mean plays a role in maximum likelihood estimation for the four parameter case, in addition to the geometric mean. Actually, when performing maximum likelihood estimation for the four parameter case, besides the harmonic meanHX based on the random variableX, also another harmonic mean appears naturally: the harmonic mean based on the linear transformation (1 − X), the mirror-image ofX, denoted byH1 − X:

H1−X=1E⁡[11−X]=β−1α+β−1 if β>1, and α>0.{\displaystyle H_{1-X}={\frac {1}{\operatorname {E} \left[{\frac {1}{1-X}}\right]}}={\frac {\beta -1}{\alpha +\beta -1}}{\text{ if }}\beta >1,{\text{ and }}\alpha >0.}

Theharmonic mean (H(1 − X)) of a beta distribution withβ < 1 is undefined, because its defining expression is not bounded in [0, 1] for shape parameterβ less than unity.

Lettingα =β in the above expression one obtains

H(1−X)=β−12β−1,{\displaystyle H_{(1-X)}={\frac {\beta -1}{2\beta -1}},}

showing that forα =β the harmonic mean ranges from 0, forα =β = 1, to 1/2, forα =β → ∞.

Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limβ→0H1−X is undefinedlimβ→1H1−X=limα→∞H1−X=0limα→0H1−X=limβ→∞H1−X=1{\displaystyle {\begin{aligned}&\lim _{\beta \to 0}H_{1-X}{\text{ is undefined}}\\&\lim _{\beta \to 1}H_{1-X}=\lim _{\alpha \to \infty }H_{1-X}=0\\&\lim _{\alpha \to 0}H_{1-X}=\lim _{\beta \to \infty }H_{1-X}=1\end{aligned}}}

Although bothHX andH1−X are asymmetric, in the case that both shape parameters are equalα =β, the harmonic means are equal:HX =H1−X. This equality follows from the following symmetry displayed between both harmonic means:

HX(B(α,β))=H1−X(B(β,α)) if α,β>1.{\displaystyle H_{X}(\mathrm {B} (\alpha ,\beta ))=H_{1-X}(\mathrm {B} (\beta ,\alpha )){\text{ if }}\alpha ,\beta >1.}

Measures of statistical dispersion

[edit]

Variance

[edit]

Thevariance (the second moment centered on the mean) of a beta distributionrandom variableX with parametersα andβ is:[1][14]

var⁡(X)=E⁡[(X−μ)2]=αβ(α+β)2(α+β+1){\displaystyle \operatorname {var} (X)=\operatorname {E} \left[(X-\mu )^{2}\right]={\frac {\alpha \beta }{\left(\alpha +\beta \right)^{2}\left(\alpha +\beta +1\right)}}}

Lettingα =β in the above expression one obtains

var⁡(X)=14(2β+1),{\displaystyle \operatorname {var} (X)={\frac {1}{4(2\beta +1)}},}

showing that forα =β the variance decreases monotonically asα =β increases. Settingα =β = 0 in this expression, one finds the maximum variance var(X) = 1/4[1] which only occurs approaching the limit, atα =β = 0.

The beta distribution may also beparametrized in terms of its meanμ(0 <μ < 1) and sample sizeν =α +β (ν > 0) (see subsectionMean and sample size):

α=μν, where ν=(α+β)>0,β=(1−μ)ν, where ν=(α+β)>0.{\displaystyle {\begin{aligned}\alpha &=\mu \nu ,&{\text{ where }}\nu =(\alpha +\beta )>0,\\\beta &=(1-\mu )\nu ,&{\text{ where }}\nu =(\alpha +\beta )>0.\end{aligned}}}

Using thisparametrization, one can express the variance in terms of the meanμ and the sample sizeν as follows:

var⁡(X)=μ(1−μ)1+ν{\displaystyle \operatorname {var} (X)={\frac {\mu (1-\mu )}{1+\nu }}}

Sinceν =α +β > 0, it follows thatvar(X) <μ(1 −μ).

For a symmetric distribution, the mean is at the middle of the distribution,μ = 1/2, and therefore:

var⁡(X)=14(1+ν) if μ=12{\displaystyle \operatorname {var} (X)={\frac {1}{4(1+\nu )}}{\text{ if }}\mu ={\tfrac {1}{2}}}

Also, the following limits (with only the noted variable approaching the limit) can be obtained from the above expressions:

limβ→0var⁡(X)=limα→0var⁡(X)=limβ→∞var⁡(X)=limα→∞var⁡(X)=0limν→∞var⁡(X)=limμ→0var⁡(X)=limμ→1var⁡(X)=0limν→0var⁡(X)=μ(1−μ){\displaystyle {\begin{aligned}&\lim _{\beta \to 0}\operatorname {var} (X)=\lim _{\alpha \to 0}\operatorname {var} (X)=\lim _{\beta \to \infty }\operatorname {var} (X)=\lim _{\alpha \to \infty }\operatorname {var} (X)=0\\&\lim _{\nu \to \infty }\operatorname {var} (X)=\lim _{\mu \to 0}\operatorname {var} (X)=\lim _{\mu \to 1}\operatorname {var} (X)=0\\&\lim _{\nu \to 0}\operatorname {var} (X)=\mu (1-\mu )\end{aligned}}}

Geometric variance and covariance

[edit]
log geometric variances vs.α andβ
log geometric variances vs.α andβ

The logarithm of the geometric variance, ln(varGX), of a distribution withrandom variableX is the second moment of the logarithm ofX centered on the geometric mean ofX, ln(GX):

ln⁡varGX=E⁡[(ln⁡X−ln⁡GX)2]=E⁡[(ln⁡X−E⁡[ln⁡X])2]=E⁡[(ln⁡X)2]−(E⁡[ln⁡X])2=var⁡[ln⁡X]{\displaystyle {\begin{aligned}\ln \operatorname {var} _{GX}&=\operatorname {E} \left[\left(\ln X-\ln G_{X}\right)^{2}\right]\\&=\operatorname {E} \left[\left(\ln X-\operatorname {E} \left[\ln X\right]\right)^{2}\right]\\&=\operatorname {E} \left[\left(\ln X\right)^{2}\right]-\left(\operatorname {E} [\ln X]\right)^{2}\\&=\operatorname {var} [\ln X]\end{aligned}}}

and therefore, the geometric variance is:

varGX=evar⁡[ln⁡X]{\displaystyle \operatorname {var} _{GX}=e^{\operatorname {var} [\ln X]}}

In theFisher information matrix, and the curvature of the loglikelihood function, the logarithm of the geometric variance of thereflected variable 1 − X and the logarithm of the geometric covariance betweenX and 1 − X appear:

ln⁡varG(1-X)=E⁡[(ln⁡(1−X)−ln⁡G1−X)2]=E⁡[(ln⁡(1−X)−E⁡[ln⁡(1−X)])2]=E⁡[(ln⁡(1−X))2]−(E⁡[ln⁡(1−X)])2=var⁡[ln⁡(1−X)]varG(1-X)=evar⁡[ln⁡(1−X)]ln⁡covGX,1-X=E⁡[(ln⁡X−ln⁡GX)(ln⁡(1−X)−ln⁡G1−X)]=E⁡[(ln⁡X−E⁡[ln⁡X])(ln⁡(1−X)−E⁡[ln⁡(1−X)])]=E⁡[ln⁡Xln⁡(1−X)]−E⁡[ln⁡X]E⁡[ln⁡(1−X)]=cov⁡[ln⁡X,ln⁡(1−X)]covGX,(1−X)=ecov⁡[ln⁡X,ln⁡(1−X)]{\displaystyle {\begin{aligned}\ln \operatorname {var_{G(1-X)}} &=\operatorname {E} \left[\left(\ln(1-X)-\ln G_{1-X}\right)^{2}\right]\\&=\operatorname {E} \left[\left(\ln(1-X)-\operatorname {E} [\ln(1-X)]\right)^{2}\right]\\&=\operatorname {E} \left[(\ln(1-X))^{2}\right]-\left(\operatorname {E} [\ln(1-X)]\right)^{2}\\&=\operatorname {var} [\ln(1-X)]\\&\\\operatorname {var_{G(1-X)}} &=e^{\operatorname {var} [\ln(1-X)]}\\&\\\ln \operatorname {cov_{G{X,1-X}}} &=\operatorname {E} [(\ln X-\ln G_{X})(\ln(1-X)-\ln G_{1-X})]\\&=\operatorname {E} [(\ln X-\operatorname {E} [\ln X])(\ln(1-X)-\operatorname {E} [\ln(1-X)])]\\&=\operatorname {E} \left[\ln X\ln(1-X)\right]-\operatorname {E} [\ln X]\operatorname {E} [\ln(1-X)]\\&=\operatorname {cov} [\ln X,\ln(1-X)]\\&\\\operatorname {cov} _{G{X,(1-X)}}&=e^{\operatorname {cov} [\ln X,\ln(1-X)]}\end{aligned}}}

For a beta distribution, higher order logarithmic moments can be derived by using the representation of a beta distribution as a proportion of two gamma distributions and differentiating through the integral. They can be expressed in terms of higher order poly-gamma functions. See the section§ Moments of logarithmically transformed random variables. Thevariance of the logarithmic variables andcovariance of ln X and ln(1−X) are:

var⁡[ln⁡X]=ψ1(α)−ψ1(α+β){\displaystyle \operatorname {var} [\ln X]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )}var⁡[ln⁡(1−X)]=ψ1(β)−ψ1(α+β){\displaystyle \operatorname {var} [\ln(1-X)]=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )}cov⁡[ln⁡X,ln⁡(1−X)]=−ψ1(α+β){\displaystyle \operatorname {cov} [\ln X,\ln(1-X)]=-\psi _{1}(\alpha +\beta )}

where thetrigamma function, denotedψ1(α), is the second of thepolygamma functions, and is defined as the derivative of thedigamma function:

ψ1(α)=d2ln⁡Γ(α)dα2=dψ(α)dα.{\displaystyle \psi _{1}(\alpha )={\frac {d^{2}\ln \Gamma (\alpha )}{d\alpha ^{2}}}={\frac {d\psi (\alpha )}{d\alpha }}.}

Therefore,

ln⁡varGX=var⁡[ln⁡X]=ψ1(α)−ψ1(α+β){\displaystyle \ln \operatorname {var} _{GX}=\operatorname {var} [\ln X]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )}ln⁡varG(1−X)=var⁡[ln⁡(1−X)]=ψ1(β)−ψ1(α+β){\displaystyle \ln \operatorname {var} _{G(1-X)}=\operatorname {var} [\ln(1-X)]=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )}ln⁡covGX,1−X=cov⁡[ln⁡X,ln⁡(1−X)]=−ψ1(α+β){\displaystyle \ln \operatorname {cov} _{GX,1-X}=\operatorname {cov} [\ln X,\ln(1-X)]=-\psi _{1}(\alpha +\beta )}

The accompanying plots show the log geometric variances and log geometric covariance versus the shape parametersα andβ. The plots show that the log geometric variances and log geometric covariance are close to zero for shape parametersα andβ greater than 2, and that the log geometric variances rapidly rise in value for shape parameter valuesα andβ less than unity. The log geometric variances are positive for all values of the shape parameters. The log geometric covariance is negative for all values of the shape parameters, and it reaches large negative values forα andβ less than unity.

Following are the limits with one parameter finite (non-zero) and the other approaching these limits:

limα→0ln⁡varGX=limβ→0ln⁡varG(1−X)=∞limβ→0ln⁡varGX=limα→∞ln⁡varGX=limα→0ln⁡varG(1−X)=limβ→∞ln⁡varG(1−X)=0limα→∞ln⁡covGX,(1−X)=limβ→∞ln⁡covGX,(1−X)=0limβ→∞ln⁡varGX=ψ1(α)limα→∞ln⁡varG(1−X)=ψ1(β)limα→0ln⁡covGX,(1−X)=−ψ1(β)limβ→0ln⁡covGX,(1−X)=−ψ1(α){\displaystyle {\begin{aligned}&\lim _{\alpha \to 0}\ln \operatorname {var} _{GX}=\lim _{\beta \to 0}\ln \operatorname {var} _{G(1-X)}=\infty \\&\lim _{\beta \to 0}\ln \operatorname {var} _{GX}=\lim _{\alpha \to \infty }\ln \operatorname {var} _{GX}=\lim _{\alpha \to 0}\ln \operatorname {var} _{G(1-X)}=\lim _{\beta \to \infty }\ln \operatorname {var} _{G(1-X)}=0\\&\lim _{\alpha \to \infty }\ln \operatorname {cov} _{GX,(1-X)}=\lim _{\beta \to \infty }\ln \operatorname {cov} _{GX,(1-X)}=0\\&\lim _{\beta \to \infty }\ln \operatorname {var} _{GX}=\psi _{1}(\alpha )\\&\lim _{\alpha \to \infty }\ln \operatorname {var} _{G(1-X)}=\psi _{1}(\beta )\\&\lim _{\alpha \to 0}\ln \operatorname {cov} _{GX,(1-X)}=-\psi _{1}(\beta )\\&\lim _{\beta \to 0}\ln \operatorname {cov} _{GX,(1-X)}=-\psi _{1}(\alpha )\end{aligned}}}

Limits with two parameters varying:

limα→∞(limβ→∞ln⁡varGX)=limβ→∞(limα→∞ln⁡varG(1−X))=limα→∞(limβ→0ln⁡covGX,(1−X))=limβ→∞(limα→0ln⁡covGX,(1−X))=0limα→∞(limβ→0ln⁡varGX)=limβ→∞(limα→0ln⁡varG(1−X))=∞limα→0(limβ→0ln⁡covGX,(1−X))=limβ→0(limα→0ln⁡covGX,(1−X))=−∞{\displaystyle {\begin{aligned}&\lim _{\alpha \to \infty }(\lim _{\beta \to \infty }\ln \operatorname {var} _{GX})=\lim _{\beta \to \infty }(\lim _{\alpha \to \infty }\ln \operatorname {var} _{G(1-X)})=\lim _{\alpha \to \infty }(\lim _{\beta \to 0}\ln \operatorname {cov} _{GX,(1-X)})=\lim _{\beta \to \infty }(\lim _{\alpha \to 0}\ln \operatorname {cov} _{GX,(1-X)})=0\\&\lim _{\alpha \to \infty }(\lim _{\beta \to 0}\ln \operatorname {var} _{GX})=\lim _{\beta \to \infty }(\lim _{\alpha \to 0}\ln \operatorname {var} _{G(1-X)})=\infty \\&\lim _{\alpha \to 0}(\lim _{\beta \to 0}\ln \operatorname {cov} _{GX,(1-X)})=\lim _{\beta \to 0}(\lim _{\alpha \to 0}\ln \operatorname {cov} _{GX,(1-X)})=-\infty \end{aligned}}}

Although both ln(varGX) and ln(varG(1 − X)) are asymmetric, when the shape parameters are equal,α =β, one has: ln(varGX) = ln(varG(1−X)). This equality follows from the following symmetry displayed between both log geometric variances:

ln⁡varGX⁡(B(α,β))=ln⁡varG(1−X)⁡(B(β,α)).{\displaystyle \ln \operatorname {var} _{GX}(\mathrm {B} (\alpha ,\beta ))=\ln \operatorname {var} _{G(1-X)}(\mathrm {B} (\beta ,\alpha )).}

The log geometric covariance is symmetric:

ln⁡covGX,(1−X)⁡(B(α,β))=ln⁡covGX,(1−X)⁡(B(β,α)){\displaystyle \ln \operatorname {cov} _{GX,(1-X)}(\mathrm {B} (\alpha ,\beta ))=\ln \operatorname {cov} _{GX,(1-X)}(\mathrm {B} (\beta ,\alpha ))}

Mean absolute deviation around the mean

[edit]
Ratio of ,ean abs.dev. to std.dev. for beta distribution with α and β ranging from 0 to 5
Ratio of mean abs.dev. to std.dev. for beta distribution with mean 0 ≤μ ≤ 1 and sample size 0 <ν ≤ 10

Themean absolute deviation around the mean for the beta distribution with shape parametersα andβ is:[9]

E⁡[|X−E[X]|]=2ααββB(α,β)(α+β)α+β+1{\displaystyle \operatorname {E} [|X-E[X]|]={\frac {2\alpha ^{\alpha }\beta ^{\beta }}{\mathrm {B} (\alpha ,\beta )(\alpha +\beta )^{\alpha +\beta +1}}}}

The mean absolute deviation around the mean is a morerobustestimator ofstatistical dispersion than the standard deviation for beta distributions with tails and inflection points at each side of the mode, Beta(α, β) distributions withα,β > 2, as it depends on the linear (absolute) deviations rather than the square deviations from the mean. Therefore, the effect of very large deviations from the mean are not as overly weighted.

UsingStirling's approximation to theGamma function,N.L.Johnson andS.Kotz[1] derived the following approximation for values of the shape parameters greater than unity (the relative error for this approximation is only −3.5% forα =β = 1, and it decreases to zero asα → ∞,β → ∞):

mean abs. dev. from meanstandard deviation=E⁡[|X−E[X]|]var⁡(X)≈2π(1+712(α+β)−112α−112β), if α,β>1.{\displaystyle {\begin{aligned}{\frac {\text{mean abs. dev. from mean}}{\text{standard deviation}}}&={\frac {\operatorname {E} [|X-E[X]|]}{\sqrt {\operatorname {var} (X)}}}\\&\approx {\sqrt {\frac {2}{\pi }}}\left(1+{\frac {7}{12(\alpha +\beta )}}{}-{\frac {1}{12\alpha }}-{\frac {1}{12\beta }}\right),{\text{ if }}\alpha ,\beta >1.\end{aligned}}}

At the limitα → ∞,β → ∞, the ratio of the mean absolute deviation to the standard deviation (for the beta distribution) becomes equal to the ratio of the same measures for the normal distribution:2π{\displaystyle {\sqrt {\frac {2}{\pi }}}}. Forα =β = 1 this ratio equals32{\displaystyle {\frac {\sqrt {3}}{2}}}, so that fromα =β = 1 toα,β → ∞ the ratio decreases by 8.5%. Forα =β = 0 the standard deviation is exactly equal to the mean absolute deviation around the mean. Therefore, this ratio decreases by 15% fromα =β = 0 toα =β = 1, and by 25% fromα =β = 0 toα,β → ∞ . However, for skewed beta distributions such thatα → 0 orβ → 0, the ratio of the standard deviation to the mean absolute deviation approaches infinity (although each of them, individually, approaches zero) because the mean absolute deviation approaches zero faster than the standard deviation.

Using theparametrization in terms of meanμ and sample sizeν =α +β > 0:

α =μν,β = (1 −μ)ν

one can express the meanabsolute deviation around the mean in terms of the meanμ and the sample sizeν as follows:

E⁡[|X−E[X]|]=2μμν(1−μ)(1−μ)ννB(μν,(1−μ)ν){\displaystyle \operatorname {E} [|X-E[X]|]={\frac {2\mu ^{\mu \nu }(1-\mu )^{(1-\mu )\nu }}{\nu \mathrm {B} (\mu \nu ,(1-\mu )\nu )}}}

For a symmetric distribution, the mean is at the middle of the distribution,μ = 1/2, and therefore:

E⁡[|X−E[X]|]=21−ννB(ν2,ν2)=21−νΓ(ν)ν(Γ(ν2))2limν→0(limμ→12E⁡[|X−E[X]|])=12limν→∞(limμ→12E⁡[|X−E[X]|])=0{\displaystyle {\begin{aligned}\operatorname {E} [|X-E[X]|]={\frac {2^{1-\nu }}{\nu \mathrm {B} ({\tfrac {\nu }{2}},{\tfrac {\nu }{2}})}}&={\frac {2^{1-\nu }\Gamma (\nu )}{\nu (\Gamma ({\tfrac {\nu }{2}}))^{2}}}\\\lim _{\nu \to 0}\left(\lim _{\mu \to {\frac {1}{2}}}\operatorname {E} [|X-E[X]|]\right)&={\frac {1}{2}}\\\lim _{\nu \to \infty }\left(\lim _{\mu \to {\frac {1}{2}}}\operatorname {E} [|X-E[X]|]\right)&=0\end{aligned}}}

Also, the following limits (with only the noted variable approaching the limit) can be obtained from the above expressions:

limβ→0E⁡[|X−E[X]|]=limα→0E⁡[|X−E[X]|]=0limβ→∞E⁡[|X−E[X]|]=limα→∞E⁡[|X−E[X]|]=0limμ→0E⁡[|X−E[X]|]=limμ→1E⁡[|X−E[X]|]=0limν→0E⁡[|X−E[X]|]=μ(1−μ)limν→∞E⁡[|X−E[X]|]=0{\displaystyle {\begin{aligned}\lim _{\beta \to 0}\operatorname {E} [|X-E[X]|]&=\lim _{\alpha \to 0}\operatorname {E} [|X-E[X]|]=0\\\lim _{\beta \to \infty }\operatorname {E} [|X-E[X]|]&=\lim _{\alpha \to \infty }\operatorname {E} [|X-E[X]|]=0\\\lim _{\mu \to 0}\operatorname {E} [|X-E[X]|]&=\lim _{\mu \to 1}\operatorname {E} [|X-E[X]|]=0\\\lim _{\nu \to 0}\operatorname {E} [|X-E[X]|]&={\sqrt {\mu (1-\mu )}}\\\lim _{\nu \to \infty }\operatorname {E} [|X-E[X]|]&=0\end{aligned}}}

Mean absolute difference

[edit]

Themean absolute difference for the beta distribution is:

MD=∫01∫01f(x;α,β)f(y;α,β)|x−y|dxdy=4α+βB(α+β,α+β)B(α,α)B(β,β){\displaystyle {\begin{aligned}\mathrm {MD} &=\int _{0}^{1}\int _{0}^{1}f(x;\alpha ,\beta )\,f(y;\alpha ,\beta )\left|x-y\right|dx\,dy\\[1ex]&={\frac {4}{\alpha +\beta }}{\frac {B(\alpha +\beta ,\alpha +\beta )}{B(\alpha ,\alpha )B(\beta ,\beta )}}\end{aligned}}}

TheGini coefficient for the beta distribution is half of the relative mean absolute difference:

G=(2α)B(α+β,α+β)B(α,α)B(β,β){\displaystyle \mathrm {G} =\left({\frac {2}{\alpha }}\right){\frac {B(\alpha +\beta ,\alpha +\beta )}{B(\alpha ,\alpha )B(\beta ,\beta )}}}

Skewness

[edit]
Skewness for beta distribution as a function of variance and mean

Theskewness (the third moment centered on the mean, normalized by the 3/2 power of the variance) of the beta distribution is[1]

γ1=E⁡[(X−μ)3](var⁡(X))3/2=2(β−α)α+β+1(α+β+2)αβ.{\displaystyle \gamma _{1}={\frac {\operatorname {E} \left[\left(X-\mu \right)^{3}\right]}{\left(\operatorname {var} (X)\right)^{3/2}}}={\frac {2\left(\beta -\alpha \right){\sqrt {\alpha +\beta +1}}}{\left(\alpha +\beta +2\right){\sqrt {\alpha \beta }}}}.}

Lettingα =β in the above expression one obtainsγ1 = 0, showing once again that forα =β the distribution is symmetric and hence the skewness is zero. Positive skew (right-tailed) forα <β, negative skew (left-tailed) forα >β.

Using theparametrization in terms of meanμ and sample sizeν =α +β:

α=μν, where ν=(α+β)>0,β=(1−μ)ν, where ν=(α+β)>0.{\displaystyle {\begin{aligned}\alpha &=\mu \nu ,&{\text{ where }}\nu =(\alpha +\beta )>0,\\\beta &=(1-\mu )\nu ,&{\text{ where }}\nu =(\alpha +\beta )>0.\end{aligned}}}

one can express the skewness in terms of the meanμ and the sample size ν as follows:

γ1=E⁡[(X−μ)3](var⁡(X))3/2=2(1−2μ)1+ν(2+ν)μ(1−μ).{\displaystyle \gamma _{1}={\frac {\operatorname {E} [(X-\mu )^{3}]}{\left(\operatorname {var} (X)\right)^{3/2}}}={\frac {2(1-2\mu ){\sqrt {1+\nu }}}{(2+\nu ){\sqrt {\mu (1-\mu )}}}}.}

The skewness can also be expressed just in terms of the variancevar and the meanμ as follows:

γ1=E⁡[(X−μ)3](var⁡(X))3/2=2(1−2μ)varμ(1−μ)+var if var<μ(1−μ){\displaystyle \gamma _{1}={\frac {\operatorname {E} [(X-\mu )^{3}]}{(\operatorname {var} (X))^{3/2}}}={\frac {2(1-2\mu ){\sqrt {\operatorname {var} }}}{\mu (1-\mu )+\operatorname {var} }}{\text{ if }}\operatorname {var} <\mu (1-\mu )}

The accompanying plot of skewness as a function of variance and mean shows that maximum variance (1/4) is coupled with zero skewness and the symmetry condition (μ = 1/2), and that maximum skewness (positive or negative infinity) occurs when the mean is located at one end or the other, so that the "mass" of the probability distribution is concentrated at the ends (minimum variance).

The following expression for the square of the skewness, in terms of the sample sizeν =α +β and the variance var, is useful for the method of moments estimation of four parameters:

(γ1)2=(E⁡[(X−μ)3])2(var⁡(X))3=4(2+ν)2(1var−4(1+ν)){\displaystyle (\gamma _{1})^{2}={\frac {\left(\operatorname {E} [(X-\mu )^{3}]\right)^{2}}{\left(\operatorname {var} (X)\right)^{3}}}={\frac {4}{(2+\nu )^{2}}}\left({\frac {1}{\operatorname {var} }}-4(1+\nu )\right)}

This expression correctly gives a skewness of zero forα =β, since in that case (see§ Variance):var=14(1+ν){\displaystyle \operatorname {var} ={\frac {1}{4(1+\nu )}}}.

For the symmetric case (α =β), skewness = 0 over the whole range, and the following limits apply:

limα=β→0γ1=limα=β→∞γ1=limν→0γ1=limν→∞γ1=limμ→12γ1=0{\displaystyle \lim _{\alpha =\beta \to 0}\gamma _{1}=\lim _{\alpha =\beta \to \infty }\gamma _{1}=\lim _{\nu \to 0}\gamma _{1}=\lim _{\nu \to \infty }\gamma _{1}=\lim _{\mu \to {\frac {1}{2}}}\gamma _{1}=0}

For the asymmetric cases (α ≠β) the following limits (with only the noted variable approaching the limit) can be obtained from the above expressions:

limα→0γ1=limμ→0γ1=∞limβ→0γ1=limμ→1γ1=−∞limα→∞γ1=−2β,limβ→0(limα→∞γ1)=−∞,limβ→∞(limα→∞γ1)=0limβ→∞γ1=2α,limα→0(limβ→∞γ1)=∞,limα→∞(limβ→∞γ1)=0limν→0γ1=1−2μμ(1−μ),limμ→0(limν→0γ1)=∞,limμ→1(limν→0γ1)=−∞{\displaystyle {\begin{aligned}&\lim _{\alpha \to 0}\gamma _{1}=\lim _{\mu \to 0}\gamma _{1}=\infty \\&\lim _{\beta \to 0}\gamma _{1}=\lim _{\mu \to 1}\gamma _{1}=-\infty \\&\lim _{\alpha \to \infty }\gamma _{1}=-{\frac {2}{\sqrt {\beta }}},\quad \lim _{\beta \to 0}(\lim _{\alpha \to \infty }\gamma _{1})=-\infty ,\quad \lim _{\beta \to \infty }(\lim _{\alpha \to \infty }\gamma _{1})=0\\&\lim _{\beta \to \infty }\gamma _{1}={\frac {2}{\sqrt {\alpha }}},\quad \lim _{\alpha \to 0}(\lim _{\beta \to \infty }\gamma _{1})=\infty ,\quad \lim _{\alpha \to \infty }(\lim _{\beta \to \infty }\gamma _{1})=0\\&\lim _{\nu \to 0}\gamma _{1}={\frac {1-2\mu }{\sqrt {\mu (1-\mu )}}},\quad \lim _{\mu \to 0}(\lim _{\nu \to 0}\gamma _{1})=\infty ,\quad \lim _{\mu \to 1}(\lim _{\nu \to 0}\gamma _{1})=-\infty \end{aligned}}}

Kurtosis

[edit]
Excess Kurtosis for Beta Distribution as a function of variance and mean

The beta distribution has been applied in acoustic analysis to assess damage to gears, as the kurtosis of the beta distribution has been reported to be a good indicator of the condition of a gear.[15] Kurtosis has also been used to distinguish the seismic signal generated by a person's footsteps from other signals. As persons or other targets moving on the ground generate continuous signals in the form of seismic waves, one can separate different targets based on the seismic waves they generate. Kurtosis is sensitive to impulsive signals, so it's much more sensitive to the signal generated by human footsteps than other signals generated by vehicles, winds, noise, etc.[16] Unfortunately, the notation for kurtosis has not been standardized. Kenney and Keeping[17] use the symbol γ2 for theexcess kurtosis, butAbramowitz and Stegun[18] use different terminology. To prevent confusion[19] between kurtosis (the fourth moment centered on the mean, normalized by the square of the variance) and excess kurtosis, when using symbols, they will be spelled out as follows:[9][20]

excess kurtosis=kurtosis−3=E⁡[(X−μ)4](var⁡(X))2−3=6[α3−α2(2β−1)+β2(β+1)−2αβ(β+2)]αβ(α+β+2)(α+β+3)=6[(α−β)2(α+β+1)−αβ(α+β+2)]αβ(α+β+2)(α+β+3).{\displaystyle {\begin{aligned}{\text{excess kurtosis}}&={\text{kurtosis}}-3\\&={\frac {\operatorname {E} [(X-\mu )^{4}]}{(\operatorname {var} (X))^{2}}}-3\\&={\frac {6[\alpha ^{3}-\alpha ^{2}(2\beta -1)+\beta ^{2}(\beta +1)-2\alpha \beta (\beta +2)]}{\alpha \beta (\alpha +\beta +2)(\alpha +\beta +3)}}\\&={\frac {6[(\alpha -\beta )^{2}(\alpha +\beta +1)-\alpha \beta (\alpha +\beta +2)]}{\alpha \beta (\alpha +\beta +2)(\alpha +\beta +3)}}.\end{aligned}}}

Lettingα =β in the above expression one obtains

excess kurtosis=−63+2α if α=β.{\displaystyle {\text{excess kurtosis}}=-{\frac {6}{3+2\alpha }}{\text{ if }}\alpha =\beta .}

Therefore, for symmetric beta distributions, the excess kurtosis is negative, increasing from a minimum value of −2 at the limit as {α =β} → 0, and approaching a maximum value of zero as {α =β} → ∞. The value of −2 is the minimum value of excess kurtosis that any distribution (not just beta distributions, but any distribution of any possible kind) can ever achieve. This minimum value is reached when all the probability density is entirely concentrated at each endx = 0 andx = 1, with nothing in between: a 2-pointBernoulli distribution with equal probability 1/2 at each end (a coin toss: see section below "Kurtosis bounded by the square of the skewness" for further discussion). The description ofkurtosis as a measure of the "potential outliers" (or "potential rare, extreme values") of the probability distribution, is correct for all distributions including the beta distribution. When rare, extreme values can occur in the beta distribution, the higher its kurtosis; otherwise, the kurtosis is lower. Forα ≠β, skewed beta distributions, the excess kurtosis can reach unlimited positive values (particularly forα → 0 for finiteβ, or forβ → 0 for finiteα) because the side away from the mode will produce occasional extreme values. Minimum kurtosis takes place when the mass density is concentrated equally at each end (and therefore the mean is at the center), and there is no probability mass density in between the ends.

Using theparametrization in terms of meanμ and sample sizeν =α +β:

α=μν, where ν=(α+β)>0β=(1−μ)ν, where ν=(α+β)>0.{\displaystyle {\begin{aligned}\alpha &{}=\mu \nu ,{\text{ where }}\nu =(\alpha +\beta )>0\\\beta &{}=(1-\mu )\nu ,{\text{ where }}\nu =(\alpha +\beta )>0.\end{aligned}}}

one can express the excess kurtosis in terms of the meanμ and the sample sizeν as follows:

excess kurtosis=63+ν((1−2μ)2(1+ν)μ(1−μ)(2+ν)−1){\displaystyle {\text{excess kurtosis}}={\frac {6}{3+\nu }}{\bigg (}{\frac {(1-2\mu )^{2}(1+\nu )}{\mu (1-\mu )(2+\nu )}}-1{\bigg )}}

The excess kurtosis can also be expressed in terms of just the following two parameters: the variance var, and the sample sizeν as follows:

excess kurtosis=6(3+ν)(2+ν)(1 var −6−5ν) if var<μ(1−μ){\displaystyle {\text{excess kurtosis}}={\frac {6}{(3+\nu )(2+\nu )}}\left({\frac {1}{\text{ var }}}-6-5\nu \right){\text{ if }}{\text{var}}<\mu (1-\mu )}

and, in terms of the variancevar and the meanμ as follows:

excess kurtosis=6 var (1− var −5μ(1−μ))(var +μ(1−μ))(2 var +μ(1−μ)) if var<μ(1−μ){\displaystyle {\text{excess kurtosis}}={\frac {6{\text{ var }}(1-{\text{ var }}-5\mu (1-\mu ))}{({\text{var }}+\mu (1-\mu ))(2{\text{ var }}+\mu (1-\mu ))}}{\text{ if }}{\text{var}}<\mu (1-\mu )}

The plot of excess kurtosis as a function of the variance and the mean shows that the minimum value of the excess kurtosis (−2, which is the minimum possible value for excess kurtosis for any distribution) is intimately coupled with the maximum value of variance (1/4) and the symmetry condition: the mean occurring at the midpoint (μ = 1/2). This occurs for the symmetric case ofα =β = 0, with zero skewness. At the limit, this is the 2 pointBernoulli distribution with equal probability 1/2 at eachDirac delta function endx = 0 andx = 1 and zero probability everywhere else. (A coin toss: one face of the coin beingx = 0 and the other face beingx = 1.) Variance is maximum because the distribution is bimodal with nothing in between the two modes (spikes) at each end. Excess kurtosis is minimum: the probability density "mass" is zero at the mean and it is concentrated at the two peaks at each end. Excess kurtosis reaches the minimum possible value (for any distribution) when the probability density function has two spikes at each end: it is bi-"peaky" with nothing in between them.

On the other hand, the plot shows that for extreme skewed cases, where the mean is located near one or the other end (μ = 0 orμ = 1), the variance is close to zero, and the excess kurtosis rapidly approaches infinity when the mean of the distribution approaches either end.

Alternatively, the excess kurtosis can also be expressed in terms of just the following two parameters: the square of the skewness, and the sample size ν as follows:

excess kurtosis=63+ν((2+ν)4(skewness)2−1) if (skewness)2−2<excess kurtosis<32(skewness)2{\displaystyle {\text{excess kurtosis}}={\frac {6}{3+\nu }}{\bigg (}{\frac {(2+\nu )}{4}}({\text{skewness}})^{2}-1{\bigg )}{\text{ if (skewness)}}^{2}-2<{\text{excess kurtosis}}<{\frac {3}{2}}({\text{skewness}})^{2}}

From this last expression, one can obtain the same limits published over a century ago byKarl Pearson[21] for the beta distribution (see section below titled "Kurtosis bounded by the square of the skewness"). Settingα + β = ν = 0 in the above expression, one obtains Pearson's lower boundary (values for the skewness and excess kurtosis below the boundary (excess kurtosis + 2 − skewness2 = 0) cannot occur for any distribution, and henceKarl Pearson appropriately called the region below this boundary the "impossible region"). The limit ofα + β = ν → ∞ determines Pearson's upper boundary.

limν→0excess kurtosis=(skewness)2−2limν→∞excess kurtosis=32(skewness)2{\displaystyle {\begin{aligned}&\lim _{\nu \to 0}{\text{excess kurtosis}}=({\text{skewness}})^{2}-2\\&\lim _{\nu \to \infty }{\text{excess kurtosis}}={\tfrac {3}{2}}({\text{skewness}})^{2}\end{aligned}}}

therefore:

(skewness)2−2<excess kurtosis<32(skewness)2{\displaystyle ({\text{skewness}})^{2}-2<{\text{excess kurtosis}}<{\tfrac {3}{2}}({\text{skewness}})^{2}}

Values ofν = α + β such thatν ranges from zero to infinity, 0 < ν < ∞, span the whole region of the beta distribution in the plane of excess kurtosis versus squared skewness.

For the symmetric case (α = β), the following limits apply:

limα=β→0excess kurtosis=−2limα=β→∞excess kurtosis=0limμ→12excess kurtosis=−63+ν{\displaystyle {\begin{aligned}&\lim _{\alpha =\beta \to 0}{\text{excess kurtosis}}=-2\\&\lim _{\alpha =\beta \to \infty }{\text{excess kurtosis}}=0\\&\lim _{\mu \to {\frac {1}{2}}}{\text{excess kurtosis}}=-{\frac {6}{3+\nu }}\end{aligned}}}

For the unsymmetric cases (α ≠ β) the following limits (with only the noted variable approaching the limit) can be obtained from the above expressions:

limα→0excess kurtosis=limβ→0excess kurtosis=limμ→0excess kurtosis=limμ→1excess kurtosis=∞limα→∞excess kurtosis=6β, limβ→0(limα→∞excess kurtosis)=∞, limβ→∞(limα→∞excess kurtosis)=0limβ→∞excess kurtosis=6α, limα→0(limβ→∞excess kurtosis)=∞, limα→∞(limβ→∞excess kurtosis)=0limν→0excess kurtosis=−6+1μ(1−μ), limμ→0(limν→0excess kurtosis)=∞, limμ→1(limν→0excess kurtosis)=∞{\displaystyle {\begin{aligned}&\lim _{\alpha \to 0}{\text{excess kurtosis}}=\lim _{\beta \to 0}{\text{excess kurtosis}}=\lim _{\mu \to 0}{\text{excess kurtosis}}=\lim _{\mu \to 1}{\text{excess kurtosis}}=\infty \\&\lim _{\alpha \to \infty }{\text{excess kurtosis}}={\frac {6}{\beta }},{\text{ }}\lim _{\beta \to 0}(\lim _{\alpha \to \infty }{\text{excess kurtosis}})=\infty ,{\text{ }}\lim _{\beta \to \infty }(\lim _{\alpha \to \infty }{\text{excess kurtosis}})=0\\&\lim _{\beta \to \infty }{\text{excess kurtosis}}={\frac {6}{\alpha }},{\text{ }}\lim _{\alpha \to 0}(\lim _{\beta \to \infty }{\text{excess kurtosis}})=\infty ,{\text{ }}\lim _{\alpha \to \infty }(\lim _{\beta \to \infty }{\text{excess kurtosis}})=0\\&\lim _{\nu \to 0}{\text{excess kurtosis}}=-6+{\frac {1}{\mu (1-\mu )}},{\text{ }}\lim _{\mu \to 0}(\lim _{\nu \to 0}{\text{excess kurtosis}})=\infty ,{\text{ }}\lim _{\mu \to 1}(\lim _{\nu \to 0}{\text{excess kurtosis}})=\infty \end{aligned}}}

Characteristic function

[edit]
Re(characteristic function) symmetric caseα = β ranging from 25 to 0
Re(characteristic function) symmetric caseα = β ranging from 0 to 25
Re(characteristic function)β =α + 1/2;α ranging from 25 to 0
Re(characteristic function)α = β + 1/2;β ranging from 25 to 0
Re(characteristic function)α = β + 1/2;β ranging from 0 to 25

Thecharacteristic function is theFourier transform of the probability density function. The characteristic function of the beta distribution isKummer's confluent hypergeometric function (of the first kind):[1][18][22]

φX(α;β;t)=E⁡[eitX]=∫01eitxf(x;α,β)dx=1F1(α;α+β;it)=∑n=0∞αn¯(it)n(α+β)n¯n!=1+∑k=1∞(∏r=0k−1α+rα+β+r)(it)kk!{\displaystyle {\begin{aligned}\varphi _{X}(\alpha ;\beta ;t)&=\operatorname {E} \left[e^{itX}\right]\\&=\int _{0}^{1}e^{itx}f(x;\alpha ,\beta )\,dx\\&={}_{1}F_{1}(\alpha ;\alpha +\beta ;it)\!\\&=\sum _{n=0}^{\infty }{\frac {\alpha ^{\overline {n}}(it)^{n}}{(\alpha +\beta )^{\overline {n}}n!}}\\&=1+\sum _{k=1}^{\infty }\left(\prod _{r=0}^{k-1}{\frac {\alpha +r}{\alpha +\beta +r}}\right){\frac {(it)^{k}}{k!}}\end{aligned}}}

where

xn¯=x(x+1)(x+2)⋯(x+n−1){\displaystyle x^{\overline {n}}=x(x+1)(x+2)\cdots (x+n-1)}

is therising factorial. The value of the characteristic function fort = 0, is one:

φX(α;β;0)=1F1(α;α+β;0)=1.{\displaystyle \varphi _{X}(\alpha ;\beta ;0)={}_{1}F_{1}(\alpha ;\alpha +\beta ;0)=1.}

Also, the real and imaginary parts of the characteristic function enjoy the following symmetries with respect to the origin of variablet:

Re⁡[1F1(α;α+β;it)]=Re⁡[1F1(α;α+β;−it)]{\displaystyle \operatorname {Re} \left[{}_{1}F_{1}(\alpha ;\alpha +\beta ;it)\right]=\operatorname {Re} \left[{}_{1}F_{1}(\alpha ;\alpha +\beta ;-it)\right]}Im⁡[1F1(α;α+β;it)]=−Im⁡[1F1(α;α+β;−it)]{\displaystyle \operatorname {Im} \left[{}_{1}F_{1}(\alpha ;\alpha +\beta ;it)\right]=-\operatorname {Im} \left[{}_{1}F_{1}(\alpha ;\alpha +\beta ;-it)\right]}

The symmetric caseα =β simplifies the characteristic function of the beta distribution to aBessel function, since in the special caseα +β = 2α theconfluent hypergeometric function (of the first kind) reduces to aBessel function (the modified Bessel function of the first kindIα−12{\displaystyle I_{\alpha -{\frac {1}{2}}}} ) usingKummer's second transformation as follows:

1F1(α;2α;it)=eit20F1(;α+12;(it)216)=eit2(it4)12−αΓ(α+12)Iα−12(it2).{\displaystyle {\begin{aligned}{}_{1}F_{1}(\alpha ;2\alpha ;it)&=e^{\frac {it}{2}}{}_{0}F_{1}\left(;\alpha +{\tfrac {1}{2}};{\frac {(it)^{2}}{16}}\right)\\&=e^{\frac {it}{2}}\left({\frac {it}{4}}\right)^{{\frac {1}{2}}-\alpha }\Gamma \left(\alpha +{\tfrac {1}{2}}\right)I_{\alpha -{\frac {1}{2}}}\left({\frac {it}{2}}\right).\end{aligned}}}

In the accompanying plots, thereal part (Re) of thecharacteristic function of the beta distribution is displayed for symmetric (α =β) and skewed (α ≠β) cases.

Other moments

[edit]

Moment generating function

[edit]

It also follows[1][9] that themoment generating function is

MX(α;β;t)=E⁡[etX]=∫01etxf(x;α,β)dx=1F1(α;α+β;t)=∑n=0∞αn¯(α+β)n¯tnn!=1+∑k=1∞(∏r=0k−1α+rα+β+r)tkk!.{\displaystyle {\begin{aligned}M_{X}(\alpha ;\beta ;t)&=\operatorname {E} \left[e^{tX}\right]\\[4pt]&=\int _{0}^{1}e^{tx}f(x;\alpha ,\beta )\,dx\\[4pt]&={}_{1}F_{1}(\alpha ;\alpha +\beta ;t)\\[4pt]&=\sum _{n=0}^{\infty }{\frac {\alpha ^{\overline {n}}}{(\alpha +\beta )^{\overline {n}}}}{\frac {t^{n}}{n!}}\\[4pt]&=1+\sum _{k=1}^{\infty }\left(\prod _{r=0}^{k-1}{\frac {\alpha +r}{\alpha +\beta +r}}\right){\frac {t^{k}}{k!}}.\end{aligned}}}

In particularMX(α;β; 0) = 1.

Higher moments

[edit]

Using themoment generating function, thek-thraw moment is given by[1] the factor

∏r=0k−1α+rα+β+r{\displaystyle \prod _{r=0}^{k-1}{\frac {\alpha +r}{\alpha +\beta +r}}}

multiplying the (exponential series) term(tkk!){\displaystyle \left({\frac {t^{k}}{k!}}\right)} in the series of themoment generating function

E⁡[Xk]=αk¯(α+β)k¯=∏r=0k−1α+rα+β+r{\displaystyle \operatorname {E} [X^{k}]={\frac {\alpha ^{\overline {k}}}{(\alpha +\beta )^{\overline {k}}}}=\prod _{r=0}^{k-1}{\frac {\alpha +r}{\alpha +\beta +r}}}

where (x)(k) is aPochhammer symbol representing rising factorial. It can also be written in a recursive form as

E⁡[Xk]=α+k−1α+β+k−1E⁡[Xk−1].{\displaystyle \operatorname {E} [X^{k}]={\frac {\alpha +k-1}{\alpha +\beta +k-1}}\operatorname {E} [X^{k-1}].}

Since the moment generating functionMX(α;β;⋅){\displaystyle M_{X}(\alpha ;\beta ;\cdot )} has a positive radius of convergence,[citation needed] the beta distribution isdetermined by its moments.[23]

Moments of transformed random variables

[edit]
Moments of linearly transformed, product and inverted random variables
[edit]

One can also show the following expectations for a transformed random variable,[1] where the random variableX is Beta-distributed with parametersα andβ:X ~ Beta(α, β). The expected value of the variable 1 − X is the mirror-symmetry of the expected value based on X:

E⁡[1−X]=βα+βE⁡[X(1−X)]=E⁡[(1−X)X]=αβ(α+β)(α+β+1){\displaystyle {\begin{aligned}\operatorname {E} [1-X]&={\frac {\beta }{\alpha +\beta }}\\\operatorname {E} [X(1-X)]&=\operatorname {E} [(1-X)X]={\frac {\alpha \beta }{(\alpha +\beta )(\alpha +\beta +1)}}\end{aligned}}}

Due to the mirror-symmetry of the probability density function of the beta distribution, the variances based on variablesX and 1 − X are identical, and the covariance onX(1 − X) is the negative of the variance:

var⁡[(1−X)]=var⁡[X]=−cov⁡[X,(1−X)]=αβ(α+β)2(α+β+1){\displaystyle \operatorname {var} [(1-X)]=\operatorname {var} [X]=-\operatorname {cov} [X,(1-X)]={\frac {\alpha \beta }{(\alpha +\beta )^{2}(\alpha +\beta +1)}}}

These are the expected values for inverted variables, (these are related to the harmonic means, see§ Harmonic mean):

E⁡[1X]=α+β−1α−1 if α>1E⁡[11−X]=α+β−1β−1 if β>1{\displaystyle {\begin{aligned}\operatorname {E} \left[{\frac {1}{X}}\right]&={\frac {\alpha +\beta -1}{\alpha -1}}&&{\text{ if }}\alpha >1\\\operatorname {E} \left[{\frac {1}{1-X}}\right]&={\frac {\alpha +\beta -1}{\beta -1}}&&{\text{ if }}\beta >1\end{aligned}}}

The following transformation by dividing the variableX by its mirror-imageX/(1 − X)) results in the expected value of the "inverted beta distribution" orbeta prime distribution (also known as beta distribution of the second kind orPearson's Type VI):[1]

E⁡[X1−X]=αβ−1 if β>1E⁡[1−XX]=βα−1 if α>1{\displaystyle {\begin{aligned}\operatorname {E} \left[{\frac {X}{1-X}}\right]&={\frac {\alpha }{\beta -1}}&&{\text{ if }}\beta >1\\\operatorname {E} \left[{\frac {1-X}{X}}\right]&={\frac {\beta }{\alpha -1}}&&{\text{ if }}\alpha >1\end{aligned}}}

Variances of these transformed variables can be obtained by integration, as the expected values of the second moments centered on the corresponding variables:

var⁡[1X]=E⁡[(1X−E⁡[1X])2]=var⁡[1−XX]=E⁡[(1−XX−E⁡[1−XX])2]=β(α+β−1)(α−2)(α−1)2 if α>2{\displaystyle {\begin{aligned}\operatorname {var} \left[{\frac {1}{X}}\right]&=\operatorname {E} \left[\left({\frac {1}{X}}-\operatorname {E} \left[{\frac {1}{X}}\right]\right)^{2}\right]=\operatorname {var} \left[{\frac {1-X}{X}}\right]\\&=\operatorname {E} \left[\left({\frac {1-X}{X}}-\operatorname {E} \left[{\frac {1-X}{X}}\right]\right)^{2}\right]={\frac {\beta (\alpha +\beta -1)}{\left(\alpha -2\right)\left(\alpha -1\right)^{2}}}{\text{ if }}\alpha >2\end{aligned}}}

The following variance of the variableX divided by its mirror-image (X/(1−X) results in the variance of the "inverted beta distribution" orbeta prime distribution (also known as beta distribution of the second kind orPearson's Type VI):[1]

var⁡[11−X]=E⁡[(11−X−E⁡[11−X])2]=var⁡[X1−X]=E⁡[(X1−X−E⁡[X1−X])2]=α(α+β−1)(β−2)(β−1)2 if β>2{\displaystyle {\begin{aligned}\operatorname {var} \left[{\frac {1}{1-X}}\right]&=\operatorname {E} \left[\left({\frac {1}{1-X}}-\operatorname {E} \left[{\frac {1}{1-X}}\right]\right)^{2}\right]=\operatorname {var} \left[{\frac {X}{1-X}}\right]\\[1ex]&=\operatorname {E} \left[\left({\frac {X}{1-X}}-\operatorname {E} \left[{\frac {X}{1-X}}\right]\right)^{2}\right]={\frac {\alpha (\alpha +\beta -1)}{\left(\beta -2\right)\left(\beta -1\right)^{2}}}{\text{ if }}\beta >2\end{aligned}}}

The covariances are:

cov⁡[1X,11−X]=cov⁡[1−XX,X1−X]=cov⁡[1X,X1−X]=cov⁡[1−XX,11−X]=α+β−1(α−1)(β−1) if α,β>1{\displaystyle {\begin{aligned}\operatorname {cov} \left[{\frac {1}{X}},{\frac {1}{1-X}}\right]&=\operatorname {cov} \left[{\frac {1-X}{X}},{\frac {X}{1-X}}\right]=\operatorname {cov} \left[{\frac {1}{X}},{\frac {X}{1-X}}\right]\\[1ex]&=\operatorname {cov} \left[{\frac {1-X}{X}},{\frac {1}{1-X}}\right]={\frac {\alpha +\beta -1}{(\alpha -1)(\beta -1)}}{\text{ if }}\alpha ,\beta >1\end{aligned}}} These expectations and variances appear in the four-parameter Fisher information matrix (§ Fisher information.)

Moments of logarithmically transformed random variables
[edit]
Plot of logit(X) = ln(X/(1 −X)) (vertical axis) vs.X in the domain of 0 to 1 (horizontal axis). Logit transformations are interesting, as they usually transform various shapes (including J-shapes) into (usually skewed) bell-shaped densities over the logit variable, and they may remove the end singularities over the original variable

Expected values forlogarithmic transformations (useful formaximum likelihood estimates, see§ Parameter estimation, Maximum likelihood) are discussed in this section. The following logarithmic linear transformations are related to the geometric meansGX andG1−X (see§ Geometric Mean):

E⁡[ln⁡X]=ψ(α)−ψ(α+β)=−E⁡[ln⁡1X],E⁡[ln⁡(1−X)]=ψ(β)−ψ(α+β)=−E⁡[ln⁡11−X].{\displaystyle {\begin{aligned}\operatorname {E} [\ln X]&=\psi (\alpha )-\psi (\alpha +\beta )=-\operatorname {E} \left[\ln {\frac {1}{X}}\right],\\\operatorname {E} [\ln(1-X)]&=\psi (\beta )-\psi (\alpha +\beta )=-\operatorname {E} \left[\ln {\frac {1}{1-X}}\right].\end{aligned}}}

Where thedigamma functionψ(α) is defined as thelogarithmic derivative of thegamma function:[18]

ψ(α)=ddαln⁡Γ(α){\displaystyle \psi (\alpha )={\frac {d}{d\alpha }}\ln \Gamma (\alpha )}

Logit transformations are interesting,[24] as they usually transform various shapes (including J-shapes) into (usually skewed) bell-shaped densities over the logit variable, and they may remove the end singularities over the original variable:

E⁡[ln⁡X1−X]=ψ(α)−ψ(β)=E⁡[ln⁡X]+E⁡[ln⁡11−X],E⁡[ln⁡1−XX]=ψ(β)−ψ(α)=−E⁡[ln⁡X1−X].{\displaystyle {\begin{aligned}\operatorname {E} \left[\ln {\frac {X}{1-X}}\right]&=\psi (\alpha )-\psi (\beta )=\operatorname {E} [\ln X]+\operatorname {E} \left[\ln {\frac {1}{1-X}}\right],\\\operatorname {E} \left[\ln {\frac {1-X}{X}}\right]&=\psi (\beta )-\psi (\alpha )=-\operatorname {E} \left[\ln {\frac {X}{1-X}}\right].\end{aligned}}}

Johnson[25] considered the distribution of thelogit – transformed variable ln(X/1 − X), including its moment generating function and approximations for large values of the shape parameters. This transformation extends the finite support [0, 1] based on the original variableX to infinite support in both directions of the real line (−∞, +∞). The logit of a beta variate has thelogistic-beta distribution.

Higher order logarithmic moments can be derived by using the representation of a beta distribution as a proportion of two gamma distributions and differentiating through the integral. They can be expressed in terms of higher order poly-gamma functions as follows:

E⁡[ln2⁡(X)]=(ψ(α)−ψ(α+β))2+ψ1(α)−ψ1(α+β),E⁡[ln2⁡(1−X)]=(ψ(β)−ψ(α+β))2+ψ1(β)−ψ1(α+β),E⁡[ln⁡(X)ln⁡(1−X)]=(ψ(α)−ψ(α+β))(ψ(β)−ψ(α+β))−ψ1(α+β).{\displaystyle {\begin{aligned}\operatorname {E} \left[\ln ^{2}(X)\right]&=(\psi (\alpha )-\psi (\alpha +\beta ))^{2}+\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta ),\\\operatorname {E} \left[\ln ^{2}(1-X)\right]&=(\psi (\beta )-\psi (\alpha +\beta ))^{2}+\psi _{1}(\beta )-\psi _{1}(\alpha +\beta ),\\\operatorname {E} \left[\ln(X)\ln(1-X)\right]&=(\psi (\alpha )-\psi (\alpha +\beta ))(\psi (\beta )-\psi (\alpha +\beta ))-\psi _{1}(\alpha +\beta ).\end{aligned}}}

therefore thevariance of the logarithmic variables andcovariance of ln(X) and ln(1−X) are:

cov⁡[ln⁡X,ln⁡(1−X)]=E⁡[ln⁡Xln⁡(1−X)]−E⁡[ln⁡X]E⁡[ln⁡(1−X)]=−ψ1(α+β)var⁡[ln⁡X]=E⁡[ln2⁡X]−(E⁡[ln⁡X])2=ψ1(α)−ψ1(α+β)=ψ1(α)+cov⁡[ln⁡X,ln⁡(1−X)]var⁡[ln⁡(1−X)]=E⁡[ln2⁡(1−X)]−(E⁡[ln⁡(1−X)])2=ψ1(β)−ψ1(α+β)=ψ1(β)+cov⁡[ln⁡X,ln⁡(1−X)]{\displaystyle {\begin{aligned}\operatorname {cov} [\ln X,\ln(1-X)]&=\operatorname {E} \left[\ln X\ln(1-X)\right]-\operatorname {E} [\ln X]\operatorname {E} [\ln(1-X)]\\&=-\psi _{1}(\alpha +\beta )\\&\\\operatorname {var} [\ln X]&=\operatorname {E} [\ln ^{2}X]-(\operatorname {E} [\ln X])^{2}\\&=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )\\&=\psi _{1}(\alpha )+\operatorname {cov} [\ln X,\ln(1-X)]\\&\\\operatorname {var} [\ln(1-X)]&=\operatorname {E} [\ln ^{2}(1-X)]-(\operatorname {E} [\ln(1-X)])^{2}\\&=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )\\&=\psi _{1}(\beta )+\operatorname {cov} [\ln X,\ln(1-X)]\end{aligned}}}

where thetrigamma function, denotedψ1(α), is the second of thepolygamma functions, and is defined as the derivative of thedigamma function:

ψ1(α)=d2ln⁡Γ(α)dα2=dψ(α)dα.{\displaystyle \psi _{1}(\alpha )={\frac {d^{2}\ln \Gamma (\alpha )}{d\alpha ^{2}}}={\frac {d\psi (\alpha )}{d\alpha }}.}

The variances and covariance of the logarithmically transformed variablesX and (1 − X) are different, in general, because the logarithmic transformation destroys the mirror-symmetry of the original variablesX and (1 − X), as the logarithm approaches negative infinity for the variable approaching zero.

These logarithmic variances and covariance are the elements of theFisher information matrix for the beta distribution. They are also a measure of the curvature of the log likelihood function (see section on Maximum likelihood estimation).

The variances of the log inverse variables are identical to the variances of the log variables:

var⁡[ln⁡1X]=var⁡[ln⁡X]=ψ1(α)−ψ1(α+β),var⁡[ln⁡11−X]=var⁡[ln⁡(1−X)]=ψ1(β)−ψ1(α+β),cov⁡[ln⁡1X,ln⁡11−X]=cov⁡[ln⁡X,ln⁡(1−X)]=−ψ1(α+β).{\displaystyle {\begin{aligned}\operatorname {var} \left[\ln {\frac {1}{X}}\right]&=\operatorname {var} [\ln X]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta ),\\\operatorname {var} \left[\ln {\frac {1}{1-X}}\right]&=\operatorname {var} [\ln(1-X)]=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta ),\\\operatorname {cov} \left[\ln {\frac {1}{X}},\,\ln {\frac {1}{1-X}}\right]&=\operatorname {cov} [\ln X,\ln(1-X)]=-\psi _{1}(\alpha +\beta ).\end{aligned}}}

It also follows that the variances of thelogit-transformed variables are

var⁡[ln⁡X1−X]=var⁡[ln⁡1−XX]=−cov⁡[ln⁡X1−X,ln⁡1−XX]=ψ1(α)+ψ1(β).{\displaystyle {\begin{aligned}\operatorname {var} \left[\ln {\frac {X}{1-X}}\right]&=\operatorname {var} \left[\ln {\frac {1-X}{X}}\right]\\&=-\operatorname {cov} \left[\ln {\frac {X}{1-X}},\,\ln {\frac {1-X}{X}}\right]\\[1ex]&=\psi _{1}(\alpha )+\psi _{1}(\beta ).\end{aligned}}}

Quantities of information (entropy)

[edit]

Given a beta distributed random variable,X ~ Beta(α, β), thedifferential entropy ofX is (measured innats),[26] the expected value of the negative of the logarithm of theprobability density function:

h(X)=E⁡[−ln⁡f(X;α,β)]=∫01−f(x;α,β)ln⁡f(x;α,β)dx=ln⁡B(α,β)−(α−1)ψ(α)−(β−1)ψ(β)+(α+β−2)ψ(α+β){\displaystyle {\begin{aligned}h(X)&=\operatorname {E} \left[-\ln f(X;\alpha ,\beta )\right]\\[4pt]&=\int _{0}^{1}-f(x;\alpha ,\beta )\ln f(x;\alpha ,\beta )\,dx\\[4pt]&=\ln \mathrm {B} (\alpha ,\beta )-(\alpha -1)\psi (\alpha )-(\beta -1)\psi (\beta )+(\alpha +\beta -2)\psi (\alpha +\beta )\end{aligned}}}

wheref(x;α,β) is theprobability density function of the beta distribution:

f(x;α,β)=xα−1(1−x)β−1B(α,β){\displaystyle f(x;\alpha ,\beta )={\frac {x^{\alpha -1}\left(1-x\right)^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}}

Thedigamma functionψ appears in the formula for the differential entropy as a consequence of Euler's integral formula for theharmonic numbers which follows from the integral:

∫011−xα−11−xdx=ψ(α)−ψ(1){\displaystyle \int _{0}^{1}{\frac {1-x^{\alpha -1}}{1-x}}\,dx=\psi (\alpha )-\psi (1)}

Thedifferential entropy of the beta distribution is negative for all values ofα andβ greater than zero, except atα = β = 1 (for which values the beta distribution is the same as theuniform distribution), where thedifferential entropy reaches itsmaximum value of zero. It is to be expected that the maximum entropy should take place when the beta distribution becomes equal to the uniform distribution, since uncertainty is maximal when all possible events are equiprobable.

Forα orβ approaching zero, thedifferential entropy approaches itsminimum value of negative infinity. For (either or both)α orβ approaching zero, there is a maximum amount of order: all the probability density is concentrated at the ends, and there is zero probability density at points located between the ends. Similarly for (either or both)α orβ approaching infinity, the differential entropy approaches its minimum value of negative infinity, and a maximum amount of order. If eitherα orβ approaches infinity (and the other is finite) all the probability density is concentrated at an end, and the probability density is zero everywhere else. If both shape parameters are equal (the symmetric case),α =β, and they approach infinity simultaneously, the probability density becomes a spike (Dirac delta function) concentrated at the middlex = 1/2, and hence there is 100% probability at the middlex = 1/2 and zero probability everywhere else.

The (continuous case)differential entropy was introduced by Shannon in his original paper (where he named it the "entropy of a continuous distribution"), as the concluding part of the same paper where he defined thediscrete entropy.[27] It is known since then that the differential entropy may differ from the infinitesimal limit of the discrete entropy by an infinite offset, therefore the differential entropy can be negative (as it is for the beta distribution). What really matters is the relative value of entropy.

Given two beta distributed random variables,X1 ~ Beta(α, β) andX2 ~ Beta(α′,β′), thecross-entropy is (measured in nats)[28]

H(X1,X2)=∫01−f(x;α,β)ln⁡f(x;α′,β′)dx=ln⁡B(α′,β′)−(α′−1)ψ(α)−(β′−1)ψ(β)+(α′+β′−2)ψ(α+β).{\displaystyle {\begin{aligned}H(X_{1},X_{2})&=\int _{0}^{1}-f(x;\alpha ,\beta )\ln f(x;\alpha ',\beta ')\,dx\\[4pt]&=\ln \mathrm {B} (\alpha ',\beta ')-(\alpha '-1)\psi (\alpha )-(\beta '-1)\psi (\beta )+\left(\alpha '+\beta '-2\right)\psi (\alpha +\beta ).\end{aligned}}}

Thecross entropy has been used as an error metric to measure the distance between two hypotheses.[29][30] Its absolute value is minimum when the two distributions are identical. It is the information measure most closely related to the log maximum likelihood[28](see section on "Parameter estimation. Maximum likelihood estimation")).

The relative entropy, orKullback–Leibler divergenceDKL(X1 ||X2), is a measure of the inefficiency of assuming that the distribution isX2 ~ Beta(α′,β′) when the distribution is reallyX1 ~ Beta(α,β). It is defined as follows (measured in nats).

DKL(X1∥X2)=∫01f(x;α,β)ln⁡f(x;α,β)f(x;α′,β′)dx=(∫01f(x;α,β)ln⁡f(x;α,β)dx)−(∫01f(x;α,β)ln⁡f(x;α′,β′)dx)=−h(X1)+H(X1,X2)=ln⁡B(α′,β′)B(α,β)+(α−α′)ψ(α)+(β−β′)ψ(β)+(α′−α+β′−β)ψ(α+β).{\displaystyle {\begin{aligned}D_{\mathrm {KL} }(X_{1}\parallel X_{2})&=\int _{0}^{1}f(x;\alpha ,\beta )\,\ln {\frac {f(x;\alpha ,\beta )}{f(x;\alpha ',\beta ')}}\,dx\\[4pt]&=\left(\int _{0}^{1}f(x;\alpha ,\beta )\ln f(x;\alpha ,\beta )\,dx\right)-\left(\int _{0}^{1}f(x;\alpha ,\beta )\ln f(x;\alpha ',\beta ')\,dx\right)\\[4pt]&=-h(X_{1})+H(X_{1},X_{2})\\[4pt]&=\ln {\frac {\mathrm {B} (\alpha ',\beta ')}{\mathrm {B} (\alpha ,\beta )}}+\left(\alpha -\alpha '\right)\psi (\alpha )+\left(\beta -\beta '\right)\psi (\beta )+\left(\alpha '-\alpha +\beta '-\beta \right)\psi (\alpha +\beta ).\end{aligned}}}

The relative entropy, orKullback–Leibler divergence, is always non-negative. A few numerical examples follow:

  • X1 ~ Beta(1, 1) andX2 ~ Beta(3, 3);DKL(X1 ||X2) = 0.598803;DKL(X2 ||X1) = 0.267864;h(X1) = 0;h(X2) = −0.267864
  • X1 ~ Beta(3, 0.5) andX2 ~ Beta(0.5, 3);DKL(X1 ||X2) = 7.21574;DKL(X2 ||X1) = 7.21574;h(X1) = −1.10805;h(X2) = −1.10805.

TheKullback–Leibler divergence is not symmetricDKL(X1 ||X2) ≠DKL(X2 ||X1) for the case in which the individual beta distributions Beta(1, 1) and Beta(3, 3) are symmetric, but have different entropiesh(X1) ≠h(X2). The value of the Kullback divergence depends on the direction traveled: whether going from a higher (differential) entropy to a lower (differential) entropy or the other way around. In the numerical example above, the Kullback divergence measures the inefficiency of assuming that the distribution is (bell-shaped) Beta(3, 3), rather than (uniform) Beta(1, 1). The "h" entropy of Beta(1, 1) is higher than the "h" entropy of Beta(3, 3) because the uniform distribution Beta(1, 1) has a maximum amount of disorder. The Kullback divergence is more than two times higher (0.598803 instead of 0.267864) when measured in the direction of decreasing entropy: the direction that assumes that the (uniform) Beta(1, 1) distribution is (bell-shaped) Beta(3, 3) rather than the other way around. In this restricted sense, the Kullback divergence is consistent with thesecond law of thermodynamics.

TheKullback–Leibler divergence is symmetricDKL(X1 ||X2) =DKL(X2 ||X1) for the skewed cases Beta(3, 0.5) and Beta(0.5, 3) that have equal differential entropyh(X1) =h(X2).

The symmetry condition:

DKL(X1∥X2)=DKL(X2∥X1), if h(X1)=h(X2), for (skewed) α≠β{\displaystyle D_{\mathrm {KL} }(X_{1}\parallel X_{2})=D_{\mathrm {KL} }(X_{2}\parallel X_{1}),{\text{ if }}h(X_{1})=h(X_{2}),{\text{ for (skewed) }}\alpha \neq \beta }

follows from the above definitions and the mirror-symmetryf(x;α,β) =f(1 −x;α,β) enjoyed by the beta distribution.

Relationships between statistical measures

[edit]

Mean, mode and median relationship

[edit]

If 1 <α <β then mode ≤ median ≤ mean.[10] Expressing the mode (only forα,β > 1), and the mean in terms ofα andβ:

α−1α+β−2≤median≤αα+β,{\displaystyle {\frac {\alpha -1}{\alpha +\beta -2}}\leq {\text{median}}\leq {\frac {\alpha }{\alpha +\beta }},}

If 1 <β <α then the order of the inequalities are reversed. Forα,β > 1 the absolute distance between the mean and the median is less than 5% of the distance between the maximum and minimum values ofx. On the other hand, the absolute distance between the mean and the mode can reach 50% of the distance between the maximum and minimum values ofx, for the (pathological) case ofα = 1 andβ = 1, for which values the beta distribution approaches the uniform distribution and thedifferential entropy approaches itsmaximum value, and hence maximum "disorder".

For example, forα = 1.0001 andβ = 1.00000001:

  • mode = 0.9999; PDF(mode) = 1.00010
  • mean = 0.500025; PDF(mean) = 1.00003
  • median = 0.500035; PDF(median) = 1.00003
  • mean − mode = −0.499875
  • mean − median = −9.65538 × 10−6

where PDF stands for the value of theprobability density function.

Mean, geometric mean and harmonic mean relationship

[edit]
:Mean, median, geometric mean and harmonic mean for beta distribution with 0 <α =β < 5

It is known from theinequality of arithmetic and geometric means that the geometric mean is lower than the mean. Similarly, the harmonic mean is lower than the geometric mean. The accompanying plot shows that forα =β, both the mean and the median are exactly equal to 1/2, regardless of the value ofα =β, and the mode is also equal to 1/2 forα =β > 1, however the geometric and harmonic means are lower than 1/2 and they only approach this value asymptotically asα =β → ∞.

Kurtosis bounded by the square of the skewness

[edit]
Beta distributionα andβ parameters vs. excess kurtosis and squared skewness

As remarked byFeller,[5] in thePearson system the beta probability density appears astype I (any difference between the beta distribution and Pearson's type I distribution is only superficial and it makes no difference for the following discussion regarding the relationship between kurtosis and skewness).Karl Pearson showed, in Plate 1 of his paper[21] published in 1916, a graph with thekurtosis as the vertical axis (ordinate) and the square of theskewness as the horizontal axis (abscissa), in which a number of distributions were displayed.[31] The region occupied by the beta distribution is bounded by the following twolines in the (skewness2,kurtosis)plane, or the (skewness2,excess kurtosis)plane:

(skewness)2+1<kurtosis<32(skewness)2+3{\displaystyle ({\text{skewness}})^{2}+1<{\text{kurtosis}}<{\frac {3}{2}}({\text{skewness}})^{2}+3}

or, equivalently,

(skewness)2−2<excess kurtosis<32(skewness)2{\displaystyle ({\text{skewness}})^{2}-2<{\text{excess kurtosis}}<{\frac {3}{2}}({\text{skewness}})^{2}}

At a time when there were no powerful digital computers,Karl Pearson accurately computed further boundaries,[32][21] for example, separating the "U-shaped" from the "J-shaped" distributions. The lower boundary line (excess kurtosis + 2 − skewness2 = 0) is produced by skewed "U-shaped" beta distributions with both values of shape parametersα andβ close to zero. The upper boundary line (excess kurtosis − (3/2) skewness2 = 0) is produced by extremely skewed distributions with very large values of one of the parameters and very small values of the other parameter.Karl Pearson showed[21] that this upper boundary line (excess kurtosis − (3/2) skewness2 = 0) is also the intersection with Pearson's distribution III, which has unlimited support in one direction (towards positive infinity), and can be bell-shaped or J-shaped. His son,Egon Pearson, showed[31] that the region (in the kurtosis/squared-skewness plane) occupied by the beta distribution (equivalently, Pearson's distribution I) as it approaches this boundary (excess kurtosis − (3/2) skewness2 = 0) is shared with thenoncentral chi-squared distribution. Karl Pearson[33] (Pearson 1895, pp. 357, 360, 373–376) also showed that thegamma distribution is a Pearson type III distribution. Hence this boundary line for Pearson's type III distribution is known as the gamma line. (This can be shown from the fact that the excess kurtosis of the gamma distribution is 6/k and the square of the skewness is 4/k, hence (excess kurtosis − (3/2) skewness2 = 0) is identically satisfied by the gamma distribution regardless of the value of the parameter "k"). Pearson later noted that thechi-squared distribution is a special case of Pearson's type III and also shares this boundary line (as it is apparent from the fact that for thechi-squared distribution the excess kurtosis is 12/k and the square of the skewness is 8/k, hence (excess kurtosis − (3/2) skewness2 = 0) is identically satisfied regardless of the value of the parameter "k"). This is to be expected, since the chi-squared distributionX ~ χ2(k) is a special case of the gamma distribution, with parametrization X ~ Γ(k/2, 1/2) where k is a positive integer that specifies the "number of degrees of freedom" of the chi-squared distribution.

An example of a beta distribution near the upper boundary (excess kurtosis − (3/2) skewness2 = 0) is given by α = 0.1, β = 1000, for which the ratio (excess kurtosis)/(skewness2) = 1.49835 approaches the upper limit of 1.5 from below. An example of a beta distribution near the lower boundary (excess kurtosis + 2 − skewness2 = 0) is given by α= 0.0001, β = 0.1, for which values the expression (excess kurtosis + 2)/(skewness2) = 1.01621 approaches the lower limit of 1 from above. In the infinitesimal limit for bothα andβ approaching zero symmetrically, the excess kurtosis reaches its minimum value at −2. This minimum value occurs at the point at which the lower boundary line intersects the vertical axis (ordinate). (However, in Pearson's original chart, the ordinate is kurtosis, instead of excess kurtosis, and it increases downwards rather than upwards).

Values for the skewness and excess kurtosis below the lower boundary (excess kurtosis + 2 − skewness2 = 0) cannot occur for any distribution, and henceKarl Pearson appropriately called the region below this boundary the "impossible region". The boundary for this "impossible region" is determined by (symmetric or skewed) bimodal U-shaped distributions for which the parametersα andβ approach zero and hence all the probability density is concentrated at the ends:x = 0, 1 with practically nothing in between them. Since forα ≈β ≈ 0 the probability density is concentrated at the two endsx = 0 andx = 1, this "impossible boundary" is determined by aBernoulli distribution, where the two only possible outcomes occur with respective probabilitiesp andq = 1 − p. For cases approaching this limit boundary with symmetryα =β, skewness ≈ 0, excess kurtosis ≈ −2 (this is the lowest excess kurtosis possible for any distribution), and the probabilities arep ≈q ≈ 1/2. For cases approaching this limit boundary with skewness, excess kurtosis ≈ −2 + skewness2, and the probability density is concentrated more at one end than the other end (with practically nothing in between), with probabilitiesp=βα+β{\displaystyle p={\tfrac {\beta }{\alpha +\beta }}} at the left endx = 0 andq=1−p=αα+β{\displaystyle q=1-p={\tfrac {\alpha }{\alpha +\beta }}} at the right endx = 1.

Symmetry

[edit]

All statements are conditional onα,β > 0:

Geometry of the probability density function

[edit]

Inflection points

[edit]
Inflection point location versus α and β showing regions with one inflection point
Inflection point location versus α and β showing region with two inflection points

For certain values of the shape parameters α and β, theprobability density function hasinflection points, at which thecurvature changes sign. The position of these inflection points can be useful as a measure of thedispersion or spread of the distribution.

Defining the following quantity:

κ=(α−1)(β−1)α+β−3α+β−2{\displaystyle \kappa ={\frac {\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}{\alpha +\beta -2}}}

Points of inflection occur,[1][8][9][20] depending on the value of the shape parametersα andβ, as follows:

  • (α > 2,β > 2) The distribution is bell-shaped (symmetric forα =β and skewed otherwise), withtwo inflection points, equidistant from the mode:

x=mode±κ=α−1±(α−1)(β−1)α+β−3α+β−2{\displaystyle x={\text{mode}}\pm \kappa ={\frac {\alpha -1\pm {\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}}{\alpha +\beta -2}}}

  • (α = 2,β > 2) The distribution is unimodal, positively skewed, right-tailed, withone inflection point, located to the right of the mode:

x=mode+κ=2β{\displaystyle x={\text{mode}}+\kappa ={\frac {2}{\beta }}}

  • (α > 2, β = 2) The distribution is unimodal, negatively skewed, left-tailed, withone inflection point, located to the left of the mode:

x=mode−κ=1−2α{\displaystyle x={\text{mode}}-\kappa =1-{\frac {2}{\alpha }}}

  • (1 <α < 2, β > 2,α +β > 2) The distribution is unimodal, positively skewed, right-tailed, withone inflection point, located to the right of the mode:

x=mode+κ=α−1+(α−1)(β−1)α+β−3α+β−2{\displaystyle x={\text{mode}}+\kappa ={\frac {\alpha -1+{\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}}{\alpha +\beta -2}}}

  • (0 <α < 1, 1 <β < 2) The distribution has a mode at the left endx = 0 and it is positively skewed, right-tailed. There isone inflection point, located to the right of the mode:

x=α−1+(α−1)(β−1)α+β−3α+β−2{\displaystyle x={\frac {\alpha -1+{\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}}{\alpha +\beta -2}}}

  • (α > 2, 1 <β < 2) The distribution is unimodal negatively skewed, left-tailed, withone inflection point, located to the left of the mode:

x=mode−κ=α−1−(α−1)(β−1)α+β−3α+β−2{\displaystyle x={\text{mode}}-\kappa ={\frac {\alpha -1-{\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}}{\alpha +\beta -2}}}

  • (1 <α < 2, 0 <β < 1) The distribution has a mode at the right endx = 1 and it is negatively skewed, left-tailed. There isone inflection point, located to the left of the mode:

x=α−1−(α−1)(β−1)α+β−3α+β−2{\displaystyle x={\frac {\alpha -1-{\sqrt {\frac {(\alpha -1)(\beta -1)}{\alpha +\beta -3}}}}{\alpha +\beta -2}}}

There are no inflection points in the remaining (symmetric and skewed) regions: U-shaped: (α,β < 1) upside-down-U-shaped: (1 <α < 2, 1 <β < 2), reverse-J-shaped (α < 1,β > 2) or J-shaped: (α > 2,β < 1)

The accompanying plots show the inflection point locations (shown vertically, ranging from 0 to 1) versusα andβ (the horizontal axes ranging from 0 to 5). There are large cuts at surfaces intersecting the linesα = 1,β = 1,α = 2, andβ = 2 because at these values the beta distribution change from 2 modes, to 1 mode to no mode.

Shapes

[edit]
PDF for symmetric beta distribution vs.x andα = β from 0 to 30
PDF for symmetric beta distribution vs. x andα = β from 0 to 2
PDF for skewed beta distribution vs.x andβ = 2.5α from 0 to 9
PDF for skewed beta distribution vs. x andβ = 5.5α from 0 to 9
PDF for skewed beta distribution vs. x andβ = 8α from 0 to 10

The beta density function can take a wide variety of different shapes depending on the values of the two parametersα andβ. The ability of the beta distribution to take this great diversity of shapes (using only two parameters) is partly responsible for finding wide application for modeling actual measurements:

Symmetric (α =β)
[edit]
Skewed (α ≠β)
[edit]

The density function isskewed. An interchange of parameter values yields themirror image (the reverse) of the initial curve, some more specific cases:

Related distributions

[edit]

Transformations

[edit]

Special and limiting cases

[edit]
Example of eight realizations of a random walk in one dimension starting at 0: the probability for the time of the last visit to the origin is distributed as Beta(1/2, 1/2)
Beta(1/2, 1/2): Thearcsine distribution probability density was proposed byHarold Jeffreys to represent uncertainty for aBernoulli or abinomial distribution inBayesian inference, and is now commonly referred to asJeffreys prior:p−1/2(1 − p)−1/2. This distribution also appears in severalrandom walk fundamental theorems

Derived from other distributions

[edit]

Combination with other distributions

[edit]

Compounding with other distributions

[edit]

Generalisations

[edit]

Statistical inference

[edit]

Parameter estimation

[edit]

Method of moments

[edit]
Two unknown parameters
[edit]

Two unknown parameters ((α^,β^){\displaystyle ({\hat {\alpha }},{\hat {\beta }})} of a beta distribution supported in the [0,1] interval) can be estimated, using the method of moments, with the first two moments (sample mean and sample variance) as follows. Let:

sample mean(X)=x¯=1N∑i=1NXi{\displaystyle {\text{sample mean(X)}}={\bar {x}}={\frac {1}{N}}\sum _{i=1}^{N}X_{i}}

be thesample mean estimate and

sample variance(X)=v¯=1N−1∑i=1N(Xi−x¯)2{\displaystyle {\text{sample variance(X)}}={\bar {v}}={\frac {1}{N-1}}\sum _{i=1}^{N}\left(X_{i}-{\bar {x}}\right)^{2}}

be thesample variance estimate. Themethod-of-moments estimates of the parameters are

α^=x¯(x¯(1−x¯)v¯−1) if v¯<x¯(1−x¯),{\displaystyle {\hat {\alpha }}={\bar {x}}\left({\frac {{\bar {x}}(1-{\bar {x}})}{\bar {v}}}-1\right)\ {\text{if}}\ {\bar {v}}<{\bar {x}}(1-{\bar {x}}),}β^=(1−x¯)(x¯(1−x¯)v¯−1) if v¯<x¯(1−x¯).{\displaystyle {\hat {\beta }}=(1-{\bar {x}})\left({\frac {{\bar {x}}(1-{\bar {x}})}{\bar {v}}}-1\right)\ {\text{if}}\ {\bar {v}}<{\bar {x}}(1-{\bar {x}}).}

When the distribution is required over a known interval other than [0, 1] with random variableX, say [a,c] with random variableY, then replacex¯{\displaystyle {\bar {x}}} withy¯−ac−a,{\displaystyle {\frac {{\bar {y}}-a}{c-a}},} andv¯{\displaystyle {\bar {v}}} withvY¯(c−a)2{\displaystyle {\frac {\bar {v_{Y}}}{(c-a)^{2}}}} in the above couple of equations for the shape parameters (see the "Four unknown parameters" section below),[41] where:

sample mean(Y)=y¯=1N∑i=1NYi{\displaystyle {\text{sample mean(Y)}}={\bar {y}}={\frac {1}{N}}\sum _{i=1}^{N}Y_{i}}sample variance(Y)=v¯Y=1N−1∑i=1N(Yi−y¯)2{\displaystyle {\text{sample variance(Y)}}={\bar {v}}_{Y}={\frac {1}{N-1}}\sum _{i=1}^{N}\left(Y_{i}-{\bar {y}}\right)^{2}}

Four unknown parameters
[edit]
Solutions for parameter estimates vs. (sample) excess Kurtosis and (sample) squared Skewness Beta distribution

All four parameters (α^,β^,a^,c^{\displaystyle {\hat {\alpha }},{\hat {\beta }},{\hat {a}},{\hat {c}}} of a beta distribution supported in the [a,c] interval, see section"Alternative parametrizations, Four parameters") can be estimated, using the method of moments developed byKarl Pearson, by equating sample and population values of the first four central moments (mean, variance, skewness and excess kurtosis).[1][42][43] The excess kurtosis was expressed in terms of the square of the skewness, and the sample size ν = α + β, (see previous section"Kurtosis") as follows:

excess kurtosis=63+ν((2+ν)4(skewness)2−1) if (skewness)2−2<excess kurtosis<32(skewness)2{\displaystyle {\text{excess kurtosis}}={\frac {6}{3+\nu }}\left({\frac {(2+\nu )}{4}}({\text{skewness}})^{2}-1\right){\text{ if (skewness)}}^{2}-2<{\text{excess kurtosis}}<{\tfrac {3}{2}}({\text{skewness}})^{2}}

One can use this equation to solve for the sample size ν= α + β in terms of the square of the skewness and the excess kurtosis as follows:[42]

ν^=α^+β^=3(sample excess kurtosis)−(sample skewness)2+232(sample skewness)2−(sample excess kurtosis){\displaystyle {\hat {\nu }}={\hat {\alpha }}+{\hat {\beta }}=3{\frac {({\text{sample excess kurtosis}})-({\text{sample skewness}})^{2}+2}{{\frac {3}{2}}({\text{sample skewness}})^{2}-{\text{(sample excess kurtosis)}}}}} if (sample skewness)2−2<sample excess kurtosis<32(sample skewness)2{\displaystyle {\text{ if (sample skewness)}}^{2}-2<{\text{sample excess kurtosis}}<{\tfrac {3}{2}}({\text{sample skewness}})^{2}}

This is the ratio (multiplied by a factor of 3) between the previously derived limit boundaries for the beta distribution in a space (as originally done by Karl Pearson[21]) defined with coordinates of the square of the skewness in one axis and the excess kurtosis in the other axis (see§ Kurtosis bounded by the square of the skewness):

The case of zero skewness, can be immediately solved because for zero skewness,α =β and henceν = 2α = 2β, thereforeα =β =ν/2

α^=β^=ν^2=32(sample excess kurtosis)+3−(sample excess kurtosis){\displaystyle {\hat {\alpha }}={\hat {\beta }}={\frac {\hat {\nu }}{2}}={\frac {{\frac {3}{2}}({\text{sample excess kurtosis}})+3}{-{\text{(sample excess kurtosis)}}}}} if sample skewness=0 and −2<sample excess kurtosis<0{\displaystyle {\text{ if sample skewness}}=0{\text{ and }}-2<{\text{sample excess kurtosis}}<0}

(Excess kurtosis is negative for the beta distribution with zero skewness, ranging from -2 to 0, so thatν^{\displaystyle {\hat {\nu }}} -and therefore the sample shape parameters- is positive, ranging from zero when the shape parameters approach zero and the excess kurtosis approaches -2, to infinity when the shape parameters approach infinity and the excess kurtosis approaches zero).

For non-zero sample skewness one needs to solve a system of two coupled equations. Since the skewness and the excess kurtosis are independent of the parametersa^,c^{\displaystyle {\hat {a}},{\hat {c}}}, the parametersα^,β^{\displaystyle {\hat {\alpha }},{\hat {\beta }}} can be uniquely determined from the sample skewness and the sample excess kurtosis, by solving the coupled equations with two known variables (sample skewness and sample excess kurtosis) and two unknowns (the shape parameters):

(sample skewness)2=4(β^−α^)2(1+α^+β^)α^β^(2+α^+β^)2{\displaystyle ({\text{sample skewness}})^{2}={\frac {4\left({\hat {\beta }}-{\hat {\alpha }}\right)^{2}\left(1+{\hat {\alpha }}+{\hat {\beta }}\right)}{{\hat {\alpha }}{\hat {\beta }}\left(2+{\hat {\alpha }}+{\hat {\beta }}\right)^{2}}}}sample excess kurtosis=63+α^+β^((2+α^+β^)4(sample skewness)2−1){\displaystyle {\text{sample excess kurtosis}}={\frac {6}{3+{\hat {\alpha }}+{\hat {\beta }}}}\left({\frac {(2+{\hat {\alpha }}+{\hat {\beta }})}{4}}({\text{sample skewness}})^{2}-1\right)} if (sample skewness)2−2<sample excess kurtosis<32(sample skewness)2{\displaystyle {\text{ if (sample skewness)}}^{2}-2<{\text{sample excess kurtosis}}<{\tfrac {3}{2}}({\text{sample skewness}})^{2}}

resulting in the following solution:[42]

α^,β^=ν^2(1±11+16(ν^+1)(ν^+2)2(sample skewness)2){\displaystyle {\hat {\alpha }},{\hat {\beta }}={\frac {\hat {\nu }}{2}}\left(1\pm {\frac {1}{\sqrt {1+{\frac {16({\hat {\nu }}+1)}{({\hat {\nu }}+2)^{2}({\text{sample skewness}})^{2}}}}}}\right)}

 if sample skewness≠0 and (sample skewness)2−2<sample excess kurtosis<32(sample skewness)2{\displaystyle {\text{ if sample skewness}}\neq 0{\text{ and }}({\text{sample skewness}})^{2}-2<{\text{sample excess kurtosis}}<{\tfrac {3}{2}}({\text{sample skewness}})^{2}}

Where one should take the solutions as follows:α^>β^{\displaystyle {\hat {\alpha }}>{\hat {\beta }}} for (negative) sample skewness < 0, andα^<β^{\displaystyle {\hat {\alpha }}<{\hat {\beta }}} for (positive) sample skewness > 0.

The accompanying plot shows these two solutions as surfaces in a space with horizontal axes of (sample excess kurtosis) and (sample squared skewness) and the shape parameters as the vertical axis. The surfaces are constrained by the condition that the sample excess kurtosis must be bounded by the sample squared skewness as stipulated in the above equation. The two surfaces meet at the right edge defined by zero skewness. Along this right edge, both parameters are equal and the distribution is symmetric U-shaped for α = β < 1, uniform for α = β = 1, upside-down-U-shaped for 1 < α = β < 2 and bell-shaped for α = β > 2. The surfaces also meet at the front (lower) edge defined by "the impossible boundary" line (excess kurtosis + 2 - skewness2 = 0). Along this front (lower) boundary both shape parameters approach zero, and the probability density is concentrated more at one end than the other end (with practically nothing in between), with probabilitiesp=βα+β{\displaystyle p={\tfrac {\beta }{\alpha +\beta }}} at the left endx = 0 andq=1−p=αα+β{\displaystyle q=1-p={\tfrac {\alpha }{\alpha +\beta }}} at the right endx = 1. The two surfaces become further apart towards the rear edge. At this rear edge the surface parameters are quite different from each other. As remarked, for example, by Bowman and Shenton,[44] sampling in the neighborhood of the line (sample excess kurtosis - (3/2)(sample skewness)2 = 0) (the just-J-shaped portion of the rear edge where blue meets beige), "is dangerously near to chaos", because at that line the denominator of the expression above for the estimate ν = α + β becomes zero and hence ν approaches infinity as that line is approached. Bowman and Shenton[44] write that "the higher moment parameters (kurtosis and skewness) are extremely fragile (near that line). However, the mean and standard deviation are fairly reliable." Therefore, the problem is for the case of four parameter estimation for very skewed distributions such that the excess kurtosis approaches (3/2) times the square of the skewness. This boundary line is produced by extremely skewed distributions with very large values of one of the parameters and very small values of the other parameter. See§ Kurtosis bounded by the square of the skewness for a numerical example and further comments about this rear edge boundary line (sample excess kurtosis - (3/2)(sample skewness)2 = 0). As remarked by Karl Pearson himself[45] this issue may not be of much practical importance as this trouble arises only for very skewed J-shaped (or mirror-image J-shaped) distributions with very different values of shape parameters that are unlikely to occur much in practice). The usual skewed-bell-shape distributions that occur in practice do not have this parameter estimation problem.

The remaining two parametersa^,c^{\displaystyle {\hat {a}},{\hat {c}}} can be determined using the sample mean and the sample variance using a variety of equations.[1][42] One alternative is to calculate the support interval range(c^−a^){\displaystyle ({\hat {c}}-{\hat {a}})} based on the sample variance and the sample kurtosis. For this purpose one can solve, in terms of the range(c^−a^){\displaystyle ({\hat {c}}-{\hat {a}})}, the equation expressing the excess kurtosis in terms of the sample variance, and the sample size ν (see§ Kurtosis and§ Alternative parametrizations, four parameters):

sample excess kurtosis=6(3+ν^)(2+ν^)((c^−a^)2(sample variance)−6−5ν^){\displaystyle {\text{sample excess kurtosis}}={\frac {6}{(3+{\hat {\nu }})(2+{\hat {\nu }})}}{\bigg (}{\frac {({\hat {c}}-{\hat {a}})^{2}}{\text{(sample variance)}}}-6-5{\hat {\nu }}{\bigg )}}

to obtain:

(c^−a^)=(sample variance)6+5ν^+(2+ν^)(3+ν^)6(sample excess kurtosis){\displaystyle ({\hat {c}}-{\hat {a}})={\sqrt {\text{(sample variance)}}}{\sqrt {6+5{\hat {\nu }}+{\frac {(2+{\hat {\nu }})(3+{\hat {\nu }})}{6}}{\text{(sample excess kurtosis)}}}}}

Another alternative is to calculate the support interval range(c^−a^){\displaystyle ({\hat {c}}-{\hat {a}})} based on the sample variance and the sample skewness.[42] For this purpose one can solve, in terms of the range(c^−a^){\displaystyle ({\hat {c}}-{\hat {a}})}, the equation expressing the squared skewness in terms of the sample variance, and the sample size ν (see section titled "Skewness" and "Alternative parametrizations, four parameters"):

(sample skewness)2=4(2+ν^)2((c^−a^)2(sample variance)−4(1+ν^)){\displaystyle ({\text{sample skewness}})^{2}={\frac {4}{(2+{\hat {\nu }})^{2}}}{\bigg (}{\frac {({\hat {c}}-{\hat {a}})^{2}}{\text{(sample variance)}}}-4(1+{\hat {\nu }}){\bigg )}}

to obtain:[42]

(c^−a^)=(sample variance)2(2+ν^)2(sample skewness)2+16(1+ν^){\displaystyle ({\hat {c}}-{\hat {a}})={\frac {\sqrt {\text{(sample variance)}}}{2}}{\sqrt {(2+{\hat {\nu }})^{2}({\text{sample skewness}})^{2}+16(1+{\hat {\nu }})}}}

The remaining parameter can be determined from the sample mean and the previously obtained parameters:(c^−a^),α^,ν^=α^+β^{\displaystyle ({\hat {c}}-{\hat {a}}),{\hat {\alpha }},{\hat {\nu }}={\hat {\alpha }}+{\hat {\beta }}}:

a^=(sample mean)−(α^ν^)(c^−a^){\displaystyle {\hat {a}}=({\text{sample mean}})-\left({\frac {\hat {\alpha }}{\hat {\nu }}}\right)({\hat {c}}-{\hat {a}})}

and finally,c^=(c^−a^)+a^{\displaystyle {\hat {c}}=({\hat {c}}-{\hat {a}})+{\hat {a}}}.

In the above formulas one may take, for example, as estimates of the sample moments:

sample mean=y¯=1N∑i=1NYisample variance=v¯Y=1N−1∑i=1N(Yi−y¯)2sample skewness=G1=N(N−1)(N−2)∑i=1N(Yi−y¯)3v¯Y32sample excess kurtosis=G2=N(N+1)(N−1)(N−2)(N−3)∑i=1N(Yi−y¯)4v¯Y2−3(N−1)2(N−2)(N−3){\displaystyle {\begin{aligned}{\text{sample mean}}&={\overline {y}}={\frac {1}{N}}\sum _{i=1}^{N}Y_{i}\\{\text{sample variance}}&={\overline {v}}_{Y}={\frac {1}{N-1}}\sum _{i=1}^{N}(Y_{i}-{\overline {y}})^{2}\\{\text{sample skewness}}&=G_{1}={\frac {N}{(N-1)(N-2)}}{\frac {\sum _{i=1}^{N}(Y_{i}-{\overline {y}})^{3}}{{\overline {v}}_{Y}^{\frac {3}{2}}}}\\{\text{sample excess kurtosis}}&=G_{2}={\frac {N(N+1)}{(N-1)(N-2)(N-3)}}{\frac {\sum _{i=1}^{N}(Y_{i}-{\overline {y}})^{4}}{{\overline {v}}_{Y}^{2}}}-{\frac {3(N-1)^{2}}{(N-2)(N-3)}}\end{aligned}}}

The estimatorsG1 forsample skewness andG2 forsample kurtosis are used byDAP/SAS,PSPP/SPSS, andExcel. However, they are not used byBMDP and (according to[46]) they were not used byMINITAB in 1998. Actually, Joanes and Gill in their 1998 study[46] concluded that the skewness and kurtosis estimators used inBMDP and inMINITAB (at that time) had smaller variance and mean-squared error in normal samples, but the skewness and kurtosis estimators used inDAP/SAS,PSPP/SPSS, namelyG1 andG2, had smaller mean-squared error in samples from a very skewed distribution. It is for this reason that we have spelled out "sample skewness", etc., in the above formulas, to make it explicit that the user should choose the best estimator according to the problem at hand, as the best estimator for skewness and kurtosis depends on the amount of skewness (as shown by Joanes and Gill[46]).

Maximum likelihood

[edit]
Two unknown parameters
[edit]
Max (joint log likelihood/N) for beta distribution maxima atα = β = 2
Max (joint log likelihood/N) for Beta distribution maxima atα = β ∈ {0.25,0.5,1,2,4,6,8}

As is also the case formaximum likelihood estimates for thegamma distribution, the maximum likelihood estimates for the beta distribution do not have a general closed form solution for arbitrary values of the shape parameters. IfX1, ...,XN are independent random variables each having a beta distribution, the joint log likelihood function forNiid observations is:

lnL(α,β∣X)=∑i=1Nln⁡Li(α,β∣Xi)=∑i=1Nln⁡f(Xi;α,β)=∑i=1Nln⁡Xiα−1(1−Xi)β−1B(α,β)=(α−1)∑i=1Nln⁡Xi+(β−1)∑i=1Nln⁡(1−Xi)−Nln⁡B(α,β){\displaystyle {\begin{aligned}\ln \,{\mathcal {L}}(\alpha ,\beta \mid X)&=\sum _{i=1}^{N}\ln {\mathcal {L}}_{i}(\alpha ,\beta \mid X_{i})\\&=\sum _{i=1}^{N}\ln f(X_{i};\alpha ,\beta )\\&=\sum _{i=1}^{N}\ln {\frac {X_{i}^{\alpha -1}(1-X_{i})^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}\\&=(\alpha -1)\sum _{i=1}^{N}\ln X_{i}+(\beta -1)\sum _{i=1}^{N}\ln(1-X_{i})-N\ln \mathrm {B} (\alpha ,\beta )\end{aligned}}}

Finding the maximum with respect to a shape parameter involves taking the partial derivative with respect to the shape parameter and setting the expression equal to zero yielding themaximum likelihood estimator of the shape parameters:

∂ln⁡L(α,β∣X)∂α=∑i=1Nln⁡Xi−N∂ln⁡B(α,β)∂α=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \alpha }}=\sum _{i=1}^{N}\ln X_{i}-N{\frac {\partial \ln \mathrm {B} (\alpha ,\beta )}{\partial \alpha }}=0}∂ln⁡L(α,β∣X)∂β=∑i=1Nln⁡(1−Xi)−N∂ln⁡B(α,β)∂β=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \beta }}=\sum _{i=1}^{N}\ln(1-X_{i})-N{\frac {\partial \ln \mathrm {B} (\alpha ,\beta )}{\partial \beta }}=0}

where:

∂ln⁡B(α,β)∂α=−∂ln⁡Γ(α+β)∂α+∂ln⁡Γ(α)∂α+∂ln⁡Γ(β)∂α=−ψ(α+β)+ψ(α)+0{\displaystyle {\begin{aligned}{\frac {\partial \ln \mathrm {B} (\alpha ,\beta )}{\partial \alpha }}&=-{\frac {\partial \ln \Gamma (\alpha +\beta )}{\partial \alpha }}+{\frac {\partial \ln \Gamma (\alpha )}{\partial \alpha }}+{\frac {\partial \ln \Gamma (\beta )}{\partial \alpha }}\\[1ex]&=-\psi (\alpha +\beta )+\psi (\alpha )+0\end{aligned}}}∂ln⁡B(α,β)∂β=−∂ln⁡Γ(α+β)∂β+∂ln⁡Γ(α)∂β+∂ln⁡Γ(β)∂β=−ψ(α+β)+0+ψ(β){\displaystyle {\begin{aligned}{\frac {\partial \ln \mathrm {B} (\alpha ,\beta )}{\partial \beta }}&=-{\frac {\partial \ln \Gamma (\alpha +\beta )}{\partial \beta }}+{\frac {\partial \ln \Gamma (\alpha )}{\partial \beta }}+{\frac {\partial \ln \Gamma (\beta )}{\partial \beta }}\\[1ex]&=-\psi (\alpha +\beta )+0+\psi (\beta )\end{aligned}}}

since thedigamma function denoted ψ(α) is defined as thelogarithmic derivative of thegamma function:[18]

ψ(α)=∂ln⁡Γ(α)∂α{\displaystyle \psi (\alpha )={\frac {\partial \ln \Gamma (\alpha )}{\partial \alpha }}}

To ensure that the values with zero tangent slope are indeed a maximum (instead of a saddle-point or a minimum) one has to also satisfy the condition that the curvature is negative. This amounts to satisfying that the second partial derivative with respect to the shape parameters is negative

∂2ln⁡L(α,β∣X)∂α2=−N∂2ln⁡B(α,β)∂α2<0{\displaystyle {\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \alpha ^{2}}}=-N{\frac {\partial ^{2}\ln \mathrm {B} (\alpha ,\beta )}{\partial \alpha ^{2}}}<0}∂2ln⁡L(α,β∣X)∂β2=−N∂2ln⁡B(α,β)∂β2<0{\displaystyle {\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \beta ^{2}}}=-N{\frac {\partial ^{2}\ln \mathrm {B} (\alpha ,\beta )}{\partial \beta ^{2}}}<0}

using the previous equations, this is equivalent to:

∂2ln⁡B(α,β)∂α2=ψ1(α)−ψ1(α+β)>0{\displaystyle {\frac {\partial ^{2}\ln \mathrm {B} (\alpha ,\beta )}{\partial \alpha ^{2}}}=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )>0}∂2ln⁡B(α,β)∂β2=ψ1(β)−ψ1(α+β)>0{\displaystyle {\frac {\partial ^{2}\ln \mathrm {B} (\alpha ,\beta )}{\partial \beta ^{2}}}=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )>0}

where thetrigamma function, denotedψ1(α), is the second of thepolygamma functions, and is defined as the derivative of thedigamma function:

ψ1(α)=∂2ln⁡Γ(α)∂α2=∂ψ(α)∂α.{\displaystyle \psi _{1}(\alpha )={\frac {\partial ^{2}\ln \Gamma (\alpha )}{\partial \alpha ^{2}}}=\,{\frac {\partial \,\psi (\alpha )}{\partial \alpha }}.}

These conditions are equivalent to stating that the variances of the logarithmically transformed variables are positive, since:

var⁡[ln⁡(X)]=E⁡[ln2⁡(X)]−(E⁡[ln⁡(X)])2=ψ1(α)−ψ1(α+β){\displaystyle \operatorname {var} [\ln(X)]=\operatorname {E} [\ln ^{2}(X)]-(\operatorname {E} [\ln(X)])^{2}=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )}var⁡[ln⁡(1−X)]=E⁡[ln2⁡(1−X)]−(E⁡[ln⁡(1−X)])2=ψ1(β)−ψ1(α+β){\displaystyle \operatorname {var} [\ln(1-X)]=\operatorname {E} [\ln ^{2}(1-X)]-(\operatorname {E} [\ln(1-X)])^{2}=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )}

Therefore, the condition of negative curvature at a maximum is equivalent to the statements:

var⁡[ln⁡(X)]>0{\displaystyle \operatorname {var} [\ln(X)]>0}var⁡[ln⁡(1−X)]>0{\displaystyle \operatorname {var} [\ln(1-X)]>0}

Alternatively, the condition of negative curvature at a maximum is also equivalent to stating that the followinglogarithmic derivatives of thegeometric meansGX andG(1−X) are positive, since:

ψ1(α)−ψ1(α+β)=∂ln⁡GX∂α>0{\displaystyle \psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )={\frac {\partial \ln G_{X}}{\partial \alpha }}>0}ψ1(β)−ψ1(α+β)=∂ln⁡G(1−X)∂β>0{\displaystyle \psi _{1}(\beta )-\psi _{1}(\alpha +\beta )={\frac {\partial \ln G_{(1-X)}}{\partial \beta }}>0}

While these slopes are indeed positive, the other slopes are negative:

∂ln⁡GX∂β,∂ln⁡G1−X∂α<0.{\displaystyle {\frac {\partial \,\ln G_{X}}{\partial \beta }},{\frac {\partial \ln G_{1-X}}{\partial \alpha }}<0.}

The slopes of the mean and the median with respect toα andβ display similar sign behavior.

From the condition that at a maximum, the partial derivative with respect to the shape parameter equals zero, we obtain the following system of coupledmaximum likelihood estimate equations (for the average log-likelihoods) that needs to be inverted to obtain the (unknown) shape parameter estimatesα^,β^{\displaystyle {\hat {\alpha }},{\hat {\beta }}} in terms of the (known) average of logarithms of the samplesX1, ...,XN:[1]

E^[ln⁡(X)]=ψ(α^)−ψ(α^+β^)=1N∑i=1Nln⁡Xi=ln⁡G^XE^[ln⁡(1−X)]=ψ(β^)−ψ(α^+β^)=1N∑i=1Nln⁡(1−Xi)=ln⁡G^1−X{\displaystyle {\begin{aligned}{\hat {\operatorname {E} }}[\ln(X)]&=\psi ({\hat {\alpha }})-\psi ({\hat {\alpha }}+{\hat {\beta }})={\frac {1}{N}}\sum _{i=1}^{N}\ln X_{i}=\ln {\hat {G}}_{X}\\{\hat {\operatorname {E} }}[\ln(1-X)]&=\psi ({\hat {\beta }})-\psi ({\hat {\alpha }}+{\hat {\beta }})={\frac {1}{N}}\sum _{i=1}^{N}\ln(1-X_{i})=\ln {\hat {G}}_{1-X}\end{aligned}}}

where we recognizelog⁡G^X{\displaystyle \log {\hat {G}}_{X}} as the logarithm of the samplegeometric mean andlog⁡G^1−X{\displaystyle \log {\hat {G}}_{1-X}} as the logarithm of the samplegeometric mean based on (1 − X), the mirror-image of X. Forα^=β^{\displaystyle {\hat {\alpha }}={\hat {\beta }}}, it follows thatG^X=G^1−X{\displaystyle {\hat {G}}_{X}={\hat {G}}_{1-X}}.

G^X=∏i=1N(Xi)1/NG^1−X=∏i=1N(1−Xi)1/N{\displaystyle {\begin{aligned}{\hat {G}}_{X}&=\prod _{i=1}^{N}(X_{i})^{1/N}\\{\hat {G}}_{1-X}&=\prod _{i=1}^{N}(1-X_{i})^{1/N}\end{aligned}}}

These coupled equations containingdigamma functions of the shape parameter estimatesα^,β^{\displaystyle {\hat {\alpha }},{\hat {\beta }}} must be solved by numerical methods as done, for example, by Beckman et al.[47] Gnanadesikan et al. give numerical solutions for a few cases.[48]N.L.Johnson andS.Kotz[1] suggest that for "not too small" shape parameter estimatesα^,β^{\displaystyle {\hat {\alpha }},{\hat {\beta }}}, the logarithmic approximation to the digamma functionψ(α^)≈ln⁡(α^−12){\displaystyle \psi ({\hat {\alpha }})\approx \ln({\hat {\alpha }}-{\tfrac {1}{2}})} may be used to obtain initial values for an iterative solution, since the equations resulting from this approximation can be solved exactly:

ln⁡α^−12α^+β^−12≈ln⁡G^X{\displaystyle \ln {\frac {{\hat {\alpha }}-{\frac {1}{2}}}{{\hat {\alpha }}+{\hat {\beta }}-{\frac {1}{2}}}}\approx \ln {\hat {G}}_{X}}ln⁡β^−12α^+β^−12≈ln⁡G^1−X{\displaystyle \ln {\frac {{\hat {\beta }}-{\frac {1}{2}}}{{\hat {\alpha }}+{\hat {\beta }}-{\frac {1}{2}}}}\approx \ln {\hat {G}}_{1-X}}

which leads to the following solution for the initial values (of the estimate shape parameters in terms of the sample geometric means) for an iterative solution:

α^≈12+G^X2(1−G^X−G^1−X) if α^>1{\displaystyle {\hat {\alpha }}\approx {\frac {1}{2}}+{\frac {{\hat {G}}_{X}}{2\left(1-{\hat {G}}_{X}-{\hat {G}}_{1-X}\right)}}{\text{ if }}{\hat {\alpha }}>1}β^≈12+G^1−X2(1−G^X−G^1−X) if β^>1{\displaystyle {\hat {\beta }}\approx {\frac {1}{2}}+{\frac {{\hat {G}}_{1-X}}{2\left(1-{\hat {G}}_{X}-{\hat {G}}_{1-X}\right)}}{\text{ if }}{\hat {\beta }}>1}

Alternatively, the estimates provided by the method of moments can instead be used as initial values for an iterative solution of the maximum likelihood coupled equations in terms of the digamma functions.

When the distribution is required over a known interval other than [0, 1] with random variableX, say [a,c] with random variableY, then replace ln(Xi) in the first equation with

ln⁡Yi−ac−a,{\displaystyle \ln {\frac {Y_{i}-a}{c-a}},}

and replace ln(1−Xi) in the second equation with

ln⁡c−Yic−a{\displaystyle \ln {\frac {c-Y_{i}}{c-a}}}

(see "Alternative parametrizations, four parameters" section below).

If one of the shape parameters is known, the problem is considerably simplified. The followinglogit transformation can be used to solve for the unknown shape parameter (for skewed cases such thatα^≠β^{\displaystyle {\hat {\alpha }}\neq {\hat {\beta }}}, otherwise, if symmetric, both -equal- parameters are known when one is known):

E^[ln⁡X1−X]=ψ(α^)−ψ(β^)=1N∑i=1Nln⁡Xi1−Xi=ln⁡G^X−ln⁡G^1−X{\displaystyle {\hat {\operatorname {E} }}\left[\ln {\frac {X}{1-X}}\right]=\psi ({\hat {\alpha }})-\psi ({\hat {\beta }})={\frac {1}{N}}\sum _{i=1}^{N}\ln {\frac {X_{i}}{1-X_{i}}}=\ln {\hat {G}}_{X}-\ln {\hat {G}}_{1-X}}

Thislogit transformation is the logarithm of the transformation that divides the variableX by its mirror-image (X/(1 -X) resulting in the "inverted beta distribution" orbeta prime distribution (also known as beta distribution of the second kind orPearson's Type VI) with support [0, +∞). As previously discussed in the section "Moments of logarithmically transformed random variables," thelogit transformationln⁡X1−X{\displaystyle \ln {\frac {X}{1-X}}}, studied by Johnson,[25] extends the finite support [0, 1] based on the original variableX to infinite support in both directions of the real line (−∞, +∞).

If, for example,β^{\displaystyle {\hat {\beta }}} is known, the unknown parameterα^{\displaystyle {\hat {\alpha }}} can be obtained in terms of the inverse[49] digamma function of the right hand side of this equation:

ψ(α^)=1N∑i=1Nln⁡Xi1−Xi+ψ(β^){\displaystyle \psi ({\hat {\alpha }})={\frac {1}{N}}\sum _{i=1}^{N}\ln {\frac {X_{i}}{1-X_{i}}}+\psi ({\hat {\beta }})}α^=ψ−1(ln⁡G^X−ln⁡G^(1−X)+ψ(β^)){\displaystyle {\hat {\alpha }}=\psi ^{-1}\left(\ln {\hat {G}}_{X}-\ln {\hat {G}}_{(1-X)}+\psi ({\hat {\beta }})\right)}

In particular, if one of the shape parameters has a value of unity, for example forβ^=1{\displaystyle {\hat {\beta }}=1} (the power function distribution with bounded support [0,1]), using the identity ψ(x + 1) = ψ(x) + 1/x in the equationψ(α^)−ψ(α^+β^)=ln⁡G^X{\displaystyle \psi ({\hat {\alpha }})-\psi ({\hat {\alpha }}+{\hat {\beta }})=\ln {\hat {G}}_{X}}, the maximum likelihood estimator for the unknown parameterα^{\displaystyle {\hat {\alpha }}} is,[1] exactly:

α^=−11N∑i=1Nln⁡Xi=−1ln⁡G^X{\displaystyle {\hat {\alpha }}=-{\frac {1}{{\frac {1}{N}}\sum _{i=1}^{N}\ln X_{i}}}=-{\frac {1}{\ln {\hat {G}}_{X}}}}

The beta has support [0, 1], thereforeG^X<1{\displaystyle {\hat {G}}_{X}<1}, and hence(−ln⁡G^X)>0{\displaystyle (-\ln {\hat {G}}_{X})>0}, and thereforeα^>0.{\displaystyle {\hat {\alpha }}>0.}

In conclusion, the maximum likelihood estimates of the shape parameters of a beta distribution are (in general) a complicated function of the samplegeometric mean, and of the samplegeometric mean based on (1−X)), the mirror-image ofX. One may ask, if the variance (in addition to the mean) is necessary to estimate two shape parameters with the method of moments, why is the (logarithmic or geometric) variance not necessary to estimate two shape parameters with the maximum likelihood method, for which only the geometric means suffice? The answer is because the mean does not provide as much information as the geometric mean. For a beta distribution with equal shape parametersα = β, the mean is exactly 1/2, regardless of the value of the shape parameters, and therefore regardless of the value of the statistical dispersion (the variance). On the other hand, the geometric mean of a beta distribution with equal shape parametersα = β, depends on the value of the shape parameters, and therefore it contains more information. Also, the geometric mean of a beta distribution does not satisfy the symmetry conditions satisfied by the mean, therefore, by employing both the geometric mean based onX and geometric mean based on (1 − X), the maximum likelihood method is able to provide best estimates for both parametersα = β, without need of employing the variance.

One can express the joint log likelihood perNiid observations in terms of thesufficient statistics (the sample geometric means) as follows:

ln⁡L(α,β∣X)N=(α−1)ln⁡G^X+(β−1)ln⁡G^(1−X)−ln⁡B(α,β).{\displaystyle {\frac {\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N}}=(\alpha -1)\ln {\hat {G}}_{X}+(\beta -1)\ln {\hat {G}}_{(1-X)}-\ln \mathrm {B} (\alpha ,\beta ).}

We can plot the joint log likelihood perN observations for fixed values of the sample geometric means to see the behavior of the likelihood function as a function of the shape parameters α and β. In such a plot, the shape parameter estimatorsα^,β^{\displaystyle {\hat {\alpha }},{\hat {\beta }}} correspond to the maxima of the likelihood function. See the accompanying graph that shows that all the likelihood functions intersect at α = β = 1, which corresponds to the values of the shape parameters that give the maximum entropy (the maximum entropy occurs for shape parameters equal to unity: the uniform distribution). It is evident from the plot that the likelihood function gives sharp peaks for values of the shape parameter estimators close to zero, but that for values of the shape parameters estimators greater than one, the likelihood function becomes quite flat, with less defined peaks. Obviously, the maximum likelihood parameter estimation method for the beta distribution becomes less acceptable for larger values of the shape parameter estimators, as the uncertainty in the peak definition increases with the value of the shape parameter estimators. One can arrive at the same conclusion by noticing that the expression for the curvature of the likelihood function is in terms of the geometric variances

∂2ln⁡L(α,β∣X)∂α2=−var⁡[ln⁡X]{\displaystyle {\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \alpha ^{2}}}=-\operatorname {var} [\ln X]}∂2ln⁡L(α,β∣X)∂β2=−var⁡[ln⁡(1−X)]{\displaystyle {\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{\partial \beta ^{2}}}=-\operatorname {var} [\ln(1-X)]}

These variances (and therefore the curvatures) are much larger for small values of the shape parameter α and β. However, for shape parameter values α, β > 1, the variances (and therefore the curvatures) flatten out. Equivalently, this result follows from theCramér–Rao bound, since theFisher information matrix components for the beta distribution are these logarithmic variances. TheCramér–Rao bound states that thevariance of anyunbiased estimatorα^{\displaystyle {\hat {\alpha }}} of α is bounded by thereciprocal of theFisher information:

var(α^)≥1var⁡[ln⁡X]≥1ψ1(α^)−ψ1(α^+β^){\displaystyle \mathrm {var} ({\hat {\alpha }})\geq {\frac {1}{\operatorname {var} [\ln X]}}\geq {\frac {1}{\psi _{1}({\hat {\alpha }})-\psi _{1}({\hat {\alpha }}+{\hat {\beta }})}}}var(β^)≥1var⁡[ln⁡(1−X)]≥1ψ1(β^)−ψ1(α^+β^){\displaystyle \mathrm {var} ({\hat {\beta }})\geq {\frac {1}{\operatorname {var} [\ln(1-X)]}}\geq {\frac {1}{\psi _{1}({\hat {\beta }})-\psi _{1}({\hat {\alpha }}+{\hat {\beta }})}}}

so the variance of the estimators increases with increasing α and β, as the logarithmic variances decrease.

Also one can express the joint log likelihood perNiid observations in terms of thedigamma function expressions for the logarithms of the sample geometric means as follows:

lnL(α,β∣X)N=(α−1)(ψ(α^)−ψ(α^+β^))+(β−1)(ψ(β^)−ψ(α^+β^))−ln⁡B(α,β){\displaystyle {\frac {\ln \,{\mathcal {L}}(\alpha ,\beta \mid X)}{N}}=(\alpha -1)(\psi ({\hat {\alpha }})-\psi ({\hat {\alpha }}+{\hat {\beta }}))+(\beta -1)(\psi ({\hat {\beta }})-\psi ({\hat {\alpha }}+{\hat {\beta }}))-\ln \mathrm {B} (\alpha ,\beta )}

this expression is identical to the negative of the cross-entropy (see section on "Quantities of information (entropy)"). Therefore, finding the maximum of the joint log likelihood of the shape parameters, perNiid observations, is identical to finding the minimum of the cross-entropy for the beta distribution, as a function of the shape parameters.

ln⁡L(α,β∣X)N=−H=−h−DKL=−ln⁡B(α,β)+(α−1)ψ(α^)+(β−1)ψ(β^)−(α+β−2)ψ(α^+β^){\displaystyle {\begin{aligned}{\frac {\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N}}&=-H=-h-D_{\mathrm {KL} }\\&=-\ln \mathrm {B} (\alpha ,\beta )+(\alpha -1)\psi ({\hat {\alpha }})+(\beta -1)\psi ({\hat {\beta }})-(\alpha +\beta -2)\psi ({\hat {\alpha }}+{\hat {\beta }})\end{aligned}}}

with the cross-entropy defined as follows:

H=∫01−f(X;α^,β^)ln⁡(f(X;α,β))dX{\displaystyle H=\int _{0}^{1}-f(X;{\hat {\alpha }},{\hat {\beta }})\ln(f(X;\alpha ,\beta ))\,{\rm {d}}X}

Four unknown parameters
[edit]

The procedure is similar to the one followed in the two unknown parameter case. IfY1, ...,YN are independent random variables each having a beta distribution with four parameters, the joint log likelihood function forNiid observations is:

ln⁡L(α,β,a,c∣Y)=∑i=1NlnLi(α,β,a,c∣Yi)=∑i=1Nln⁡f(Yi;α,β,a,c)=∑i=1Nln⁡(Yi−a)α−1(c−Yi)β−1(c−a)α+β−1B(α,β)=(α−1)∑i=1Nln⁡(Yi−a)+(β−1)∑i=1Nln⁡(c−Yi)−Nln⁡B(α,β)−N(α+β−1)ln⁡(c−a){\displaystyle {\begin{aligned}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)&=\sum _{i=1}^{N}\ln \,{\mathcal {L}}_{i}(\alpha ,\beta ,a,c\mid Y_{i})\\&=\sum _{i=1}^{N}\ln f(Y_{i};\alpha ,\beta ,a,c)\\&=\sum _{i=1}^{N}\ln {\frac {(Y_{i}-a)^{\alpha -1}(c-Y_{i})^{\beta -1}}{(c-a)^{\alpha +\beta -1}\mathrm {B} (\alpha ,\beta )}}\\&=(\alpha -1)\sum _{i=1}^{N}\ln(Y_{i}-a)+(\beta -1)\sum _{i=1}^{N}\ln(c-Y_{i})-N\ln \mathrm {B} (\alpha ,\beta )-N(\alpha +\beta -1)\ln(c-a)\end{aligned}}}

Finding the maximum with respect to a shape parameter involves taking the partial derivative with respect to the shape parameter and setting the expression equal to zero yielding themaximum likelihood estimator of the shape parameters:

∂ln⁡L(α,β,a,c∣Y)∂α=∑i=1Nln⁡(Yi−a)−N(−ψ(α+β)+ψ(α))−Nln⁡(c−a)=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha }}=\sum _{i=1}^{N}\ln(Y_{i}-a)-N(-\psi (\alpha +\beta )+\psi (\alpha ))-N\ln(c-a)=0}∂ln⁡L(α,β,a,c∣Y)∂β=∑i=1Nln⁡(c−Yi)−N(−ψ(α+β)+ψ(β))−Nln⁡(c−a)=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta }}=\sum _{i=1}^{N}\ln(c-Y_{i})-N(-\psi (\alpha +\beta )+\psi (\beta ))-N\ln(c-a)=0}∂ln⁡L(α,β,a,c∣Y)∂a=−(α−1)∑i=1N1Yi−a+N(α+β−1)1c−a=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial a}}=-(\alpha -1)\sum _{i=1}^{N}{\frac {1}{Y_{i}-a}}\,+N(\alpha +\beta -1){\frac {1}{c-a}}=0}∂ln⁡L(α,β,a,c∣Y)∂c=(β−1)∑i=1N1c−Yi−N(α+β−1)1c−a=0{\displaystyle {\frac {\partial \ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial c}}=(\beta -1)\sum _{i=1}^{N}{\frac {1}{c-Y_{i}}}\,-N(\alpha +\beta -1){\frac {1}{c-a}}=0}

these equations can be re-arranged as the following system of four coupled equations (the first two equations are geometric means and the second two equations are the harmonic means) in terms of the maximum likelihood estimates for the four parametersα^,β^,a^,c^{\displaystyle {\hat {\alpha }},{\hat {\beta }},{\hat {a}},{\hat {c}}}:

1N∑i=1Nln⁡Yi−a^c^−a^=ψ(α^)−ψ(α^+β^)=ln⁡G^X{\displaystyle {\frac {1}{N}}\sum _{i=1}^{N}\ln {\frac {Y_{i}-{\hat {a}}}{{\hat {c}}-{\hat {a}}}}=\psi ({\hat {\alpha }})-\psi ({\hat {\alpha }}+{\hat {\beta }})=\ln {\hat {G}}_{X}}1N∑i=1Nln⁡c^−Yic^−a^=ψ(β^)−ψ(α^+β^)=ln⁡G^1−X{\displaystyle {\frac {1}{N}}\sum _{i=1}^{N}\ln {\frac {{\hat {c}}-Y_{i}}{{\hat {c}}-{\hat {a}}}}=\psi ({\hat {\beta }})-\psi ({\hat {\alpha }}+{\hat {\beta }})=\ln {\hat {G}}_{1-X}}11N∑i=1Nc^−a^Yi−a^=α^−1α^+β^−1=H^X{\displaystyle {\frac {1}{{\frac {1}{N}}\sum _{i=1}^{N}{\frac {{\hat {c}}-{\hat {a}}}{Y_{i}-{\hat {a}}}}}}={\frac {{\hat {\alpha }}-1}{{\hat {\alpha }}+{\hat {\beta }}-1}}={\hat {H}}_{X}}11N∑i=1Nc^−a^c^−Yi=β^−1α^+β^−1=H^1−X{\displaystyle {\frac {1}{{\frac {1}{N}}\sum _{i=1}^{N}{\frac {{\hat {c}}-{\hat {a}}}{{\hat {c}}-Y_{i}}}}}={\frac {{\hat {\beta }}-1}{{\hat {\alpha }}+{\hat {\beta }}-1}}={\hat {H}}_{1-X}}

with sample geometric means:

G^X=∏i=1N(Yi−a^c^−a^)1N{\displaystyle {\hat {G}}_{X}=\prod _{i=1}^{N}\left({\frac {Y_{i}-{\hat {a}}}{{\hat {c}}-{\hat {a}}}}\right)^{\frac {1}{N}}}G^(1−X)=∏i=1N(c^−Yic^−a^)1N{\displaystyle {\hat {G}}_{(1-X)}=\prod _{i=1}^{N}\left({\frac {{\hat {c}}-Y_{i}}{{\hat {c}}-{\hat {a}}}}\right)^{\frac {1}{N}}}

The parametersa^,c^{\displaystyle {\hat {a}},{\hat {c}}} are embedded inside the geometric mean expressions in a nonlinear way (to the power 1/N). This precludes, in general, a closed form solution, even for an initial value approximation for iteration purposes. One alternative is to use as initial values for iteration the values obtained from the method of moments solution for the four parameter case. Furthermore, the expressions for the harmonic means are well-defined only forα^,β^>1{\displaystyle {\hat {\alpha }},{\hat {\beta }}>1}, which precludes a maximum likelihood solution for shape parameters less than unity in the four-parameter case. Fisher's information matrix for the four parameter case ispositive-definite only for α, β > 2 (for further discussion, see section on Fisher information matrix, four parameter case), for bell-shaped (symmetric or unsymmetric) beta distributions, with inflection points located to either side of the mode. The following Fisher information components (that represent the expectations of the curvature of the log likelihood function) havesingularities at the following values:

α=2:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂a2]=Ia,a{\displaystyle \alpha =2:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial a^{2}}}\right]={\mathcal {I}}_{a,a}}β=2:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂c2]=Ic,c{\displaystyle \beta =2:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial c^{2}}}\right]={\mathcal {I}}_{c,c}}α=2:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂α∂a]=Iα,a{\displaystyle \alpha =2:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha \partial a}}\right]={\mathcal {I}}_{\alpha ,a}}β=1:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂β∂c]=Iβ,c{\displaystyle \beta =1:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta \partial c}}\right]={\mathcal {I}}_{\beta ,c}}

(for further discussion see section on Fisher information matrix). Thus, it is not possible to strictly carry on the maximum likelihood estimation for some well known distributions belonging to the four-parameter beta distribution family, like theuniform distribution (Beta(1, 1,a,c)), and thearcsine distribution (Beta(1/2, 1/2,a,c)).N.L.Johnson andS.Kotz[1] ignore the equations for the harmonic means and instead suggest "If a and c are unknown, and maximum likelihood estimators ofa,c, α and β are required, the above procedure (for the two unknown parameter case, withX transformed asX = (Y − a)/(c − a)) can be repeated using a succession of trial values ofa andc, until the pair (a,c) for which maximum likelihood (givena andc) is as great as possible, is attained" (where, for the purpose of clarity, their notation for the parameters has been translated into the present notation).

Fisher information matrix

[edit]

Let a random variable X have a probability densityf(x;α). The partial derivative with respect to the (unknown, and to be estimated) parameter α of the loglikelihood function is called thescore. The second moment of the score is called theFisher information:

I(α)=E⁡[(∂∂αln⁡L(α∣X))2],{\displaystyle {\mathcal {I}}(\alpha )=\operatorname {E} \left[\left({\frac {\partial }{\partial \alpha }}\ln {\mathcal {L}}(\alpha \mid X)\right)^{2}\right],}

Theexpectation of thescore is zero, therefore the Fisher information is also the second moment centered on the mean of the score: thevariance of the score.

If the loglikelihood function is twice differentiable with respect to the parameter α, and under certain regularity conditions,[50] then the Fisher information may also be written as follows (which is often a more convenient form for calculation purposes):

I(α)=−E⁡[∂2∂α2ln⁡L(α∣X)].{\displaystyle {\mathcal {I}}(\alpha )=-\operatorname {E} \left[{\frac {\partial ^{2}}{\partial \alpha ^{2}}}\ln {\mathcal {L}}(\alpha \mid X)\right].}

Thus, the Fisher information is the negative of the expectation of the secondderivative with respect to the parameter α of the loglikelihood function. Therefore, Fisher information is a measure of thecurvature of the log likelihood function of α. A lowcurvature (and therefore highradius of curvature), flatter log likelihood function curve has low Fisher information; while a log likelihood function curve with largecurvature (and therefore lowradius of curvature) has high Fisher information. When the Fisher information matrix is computed at the evaluates of the parameters ("the observed Fisher information matrix") it is equivalent to the replacement of the true log likelihood surface by a Taylor's series approximation, taken as far as the quadratic terms.[51] The word information, in the context of Fisher information, refers to information about the parameters. Information such as: estimation, sufficiency and properties of variances of estimators. TheCramér–Rao bound states that the inverse of the Fisher information is a lower bound on the variance of anyestimator of a parameter α:

var⁡[α^]≥1I(α).{\displaystyle \operatorname {var} [{\hat {\alpha }}]\geq {\frac {1}{{\mathcal {I}}(\alpha )}}.}

The precision to which one can estimate the estimator of a parameter α is limited by the Fisher Information of the log likelihood function. The Fisher information is a measure of the minimum error involved in estimating a parameter of a distribution and it can be viewed as a measure of the resolving power of an experiment needed to discriminate between twoalternative hypothesis of a parameter.[52]

When there areN parameters

[θ1θ2⋮θN],{\displaystyle {\begin{bmatrix}\theta _{1}\\\theta _{2}\\\vdots \\\theta _{N}\end{bmatrix}},}

then the Fisher information takes the form of anN×Npositive semidefinitesymmetric matrix, the Fisher information matrix, with typical element:

(I(θ))i,j=E⁡[∂ln⁡L∂θi⋅∂ln⁡L∂θj].{\displaystyle ({\mathcal {I}}(\theta ))_{i,j}=\operatorname {E} \left[{\frac {\partial \ln {\mathcal {L}}}{\partial \theta _{i}}}\cdot {\frac {\partial \ln {\mathcal {L}}}{\partial \theta _{j}}}\right].}

Under certain regularity conditions,[50] the Fisher Information Matrix may also be written in the following form, which is often more convenient for computation:

(I(θ))i,j=−E⁡[∂2ln⁡L∂θi∂θj].{\displaystyle ({\mathcal {I}}(\theta ))_{i,j}=-\operatorname {E} \left[{\frac {\partial ^{2}\ln {\mathcal {L}}}{\partial \theta _{i}\,\partial \theta _{j}}}\right]\,.}

WithX1, ...,XNiid random variables, anN-dimensional "box" can be constructed with sidesX1, ...,XN. Costa and Cover[53] show that the (Shannon) differential entropyh(X) is related to the volume of the typical set (having the sample entropy close to the true entropy), while the Fisher information is related to the surface of this typical set.

Two parameters
[edit]

ForX1, ...,XN independent random variables each having a beta distribution parametrized with shape parametersα andβ, the joint log likelihood function forNiid observations is:

ln⁡L(α,β∣X)=(α−1)∑i=1Nln⁡Xi+(β−1)∑i=1Nln⁡(1−Xi)−Nln⁡B(α,β){\displaystyle \ln {\mathcal {L}}(\alpha ,\beta \mid X)=(\alpha -1)\sum _{i=1}^{N}\ln X_{i}+(\beta -1)\sum _{i=1}^{N}\ln(1-X_{i})-N\ln \mathrm {B} (\alpha ,\beta )}

therefore the joint log likelihood function perNiid observations is

1Nln⁡L(α,β∣X)=(α−1)1N∑i=1Nln⁡Xi+(β−1)1N∑i=1Nln⁡(1−Xi)−ln⁡B(α,β).{\displaystyle {\frac {1}{N}}\ln {\mathcal {L}}(\alpha ,\beta \mid X)=(\alpha -1){\frac {1}{N}}\sum _{i=1}^{N}\ln X_{i}+(\beta -1){\frac {1}{N}}\sum _{i=1}^{N}\ln(1-X_{i})-\,\ln \mathrm {B} (\alpha ,\beta ).}

For the two parameter case, the Fisher information has 4 components: 2 diagonal and 2 off-diagonal. Since the Fisher information matrix is symmetric, one of these off diagonal components is independent. Therefore, the Fisher information matrix has 3 independent components (2 diagonal and 1 off diagonal).

Aryal and Nadarajah[54] calculated Fisher's information matrix for the four-parameter case, from which the two parameter case can be obtained as follows:

−∂2ln⁡L(α,β∣X)N∂α2=var⁡[ln⁡(X)]=ψ1(α)−ψ1(α+β)=Iα,α=E⁡[−∂2ln⁡L(α,β∣X)N∂α2]=ln⁡varGX{\displaystyle -{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\partial \alpha ^{2}}}=\operatorname {var} [\ln(X)]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\alpha ,\alpha }=\operatorname {E} \left[-{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\partial \alpha ^{2}}}\right]=\ln \operatorname {var} _{GX}}−∂2ln⁡L(α,β∣X)N∂β2=var⁡[ln⁡(1−X)]=ψ1(β)−ψ1(α+β)=Iβ,β=E⁡[−∂2ln⁡L(α,β∣X)N∂β2]=ln⁡varG(1−X){\displaystyle -{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\,\partial \beta ^{2}}}=\operatorname {var} [\ln(1-X)]=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\beta ,\beta }=\operatorname {E} \left[-{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\partial \beta ^{2}}}\right]=\ln \operatorname {var} _{G(1-X)}}−∂2ln⁡L(α,β∣X)N∂α∂β=cov⁡[ln⁡X,ln⁡(1−X)]=−ψ1(α+β)=Iα,β=E⁡[−∂2ln⁡L(α,β∣X)N∂α∂β]=ln⁡covGX,(1−X){\displaystyle -{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\,\partial \alpha \,\partial \beta }}=\operatorname {cov} [\ln X,\ln(1-X)]=-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\alpha ,\beta }=\operatorname {E} \left[-{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta \mid X)}{N\,\partial \alpha \,\partial \beta }}\right]=\ln \operatorname {cov} _{G{X,(1-X)}}}

Since the Fisher information matrix is symmetric

Iα,β=Iβ,α=ln⁡covGX,(1−X){\displaystyle {\mathcal {I}}_{\alpha ,\beta }={\mathcal {I}}_{\beta ,\alpha }=\ln \operatorname {cov} _{G{X,(1-X)}}}

The Fisher information components are equal to the log geometric variances and log geometric covariance. Therefore, they can be expressed astrigamma functions, denoted ψ1(α), the second of thepolygamma functions, defined as the derivative of thedigamma function:

ψ1(α)=d2ln⁡Γ(α)∂α2=∂ψ(α)∂α.{\displaystyle \psi _{1}(\alpha )={\frac {d^{2}\ln \Gamma (\alpha )}{\partial \alpha ^{2}}}=\,{\frac {\partial \psi (\alpha )}{\partial \alpha }}.}

These derivatives are also derived in the§ Two unknown parameters and plots of the log likelihood function are also shown in that section.§ Geometric variance and covariance contains plots and further discussion of the Fisher information matrix components: the log geometric variances and log geometric covariance as a function of the shape parameters α and β.§ Moments of logarithmically transformed random variables contains formulas for moments of logarithmically transformed random variables. Images for the Fisher information componentsIα,α,Iβ,β{\displaystyle {\mathcal {I}}_{\alpha ,\alpha },{\mathcal {I}}_{\beta ,\beta }} andIα,β{\displaystyle {\mathcal {I}}_{\alpha ,\beta }} are shown in§ Geometric variance.

The determinant of Fisher's information matrix is of interest (for example for the calculation ofJeffreys prior probability). From the expressions for the individual components of the Fisher information matrix, it follows that the determinant of Fisher's (symmetric) information matrix for the beta distribution is:

det(I(α,β))=Iα,αIβ,β−Iα,βIα,β=(ψ1(α)−ψ1(α+β))(ψ1(β)−ψ1(α+β))−(−ψ1(α+β))(−ψ1(α+β))=ψ1(α)ψ1(β)−(ψ1(α)+ψ1(β))ψ1(α+β)limα→0det(I(α,β))=limβ→0det(I(α,β))=∞limα→∞det(I(α,β))=limβ→∞det(I(α,β))=0{\displaystyle {\begin{aligned}\det({\mathcal {I}}(\alpha ,\beta ))&={\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,\beta }-{\mathcal {I}}_{\alpha ,\beta }{\mathcal {I}}_{\alpha ,\beta }\\[4pt]&=(\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta ))(\psi _{1}(\beta )-\psi _{1}(\alpha +\beta ))-(-\psi _{1}(\alpha +\beta ))(-\psi _{1}(\alpha +\beta ))\\[4pt]&=\psi _{1}(\alpha )\psi _{1}(\beta )-(\psi _{1}(\alpha )+\psi _{1}(\beta ))\psi _{1}(\alpha +\beta )\\[4pt]\lim _{\alpha \to 0}\det({\mathcal {I}}(\alpha ,\beta ))&=\lim _{\beta \to 0}\det({\mathcal {I}}(\alpha ,\beta ))=\infty \\[4pt]\lim _{\alpha \to \infty }\det({\mathcal {I}}(\alpha ,\beta ))&=\lim _{\beta \to \infty }\det({\mathcal {I}}(\alpha ,\beta ))=0\end{aligned}}}

FromSylvester's criterion (checking whether the diagonal elements are all positive), it follows that the Fisher information matrix for the two parameter case ispositive-definite (under the standard condition that the shape parameters are positiveα > 0 and β > 0).

Four parameters
[edit]
Fisher InformationI(a,a) forα = β vs range (c − a) and exponent α = β
Fisher InformationI(α,a) forα = β, vs. range (c − a) and exponentα = β

IfY1, ...,YN are independent random variables each having a beta distribution with four parameters: the exponentsα andβ, and alsoa (the minimum of the distribution range), andc (the maximum of the distribution range) (section titled "Alternative parametrizations", "Four parameters"), withprobability density function:

f(y;α,β,a,c)=f(x;α,β)c−a=(y−ac−a)α−1(c−yc−a)β−1(c−a)B(α,β)=(y−a)α−1(c−y)β−1(c−a)α+β−1B(α,β).{\displaystyle f(y;\alpha ,\beta ,a,c)={\frac {f(x;\alpha ,\beta )}{c-a}}={\frac {\left({\frac {y-a}{c-a}}\right)^{\alpha -1}\left({\frac {c-y}{c-a}}\right)^{\beta -1}}{(c-a)B(\alpha ,\beta )}}={\frac {(y-a)^{\alpha -1}(c-y)^{\beta -1}}{(c-a)^{\alpha +\beta -1}B(\alpha ,\beta )}}.}

the joint log likelihood function perNiid observations is:

1Nln⁡L(α,β,a,c∣Y)=α−1N∑i=1Nln⁡(Yi−a)+β−1N∑i=1Nln⁡(c−Yi)−ln⁡B(α,β)−(α+β−1)ln⁡(c−a){\displaystyle {\frac {1}{N}}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)={\frac {\alpha -1}{N}}\sum _{i=1}^{N}\ln(Y_{i}-a)+{\frac {\beta -1}{N}}\sum _{i=1}^{N}\ln(c-Y_{i})-\ln \mathrm {B} (\alpha ,\beta )-(\alpha +\beta -1)\ln(c-a)}

For the four parameter case, the Fisher information has 4*4=16 components. It has 12 off-diagonal components = (4×4 total − 4 diagonal). Since the Fisher information matrix is symmetric, half of these components (12/2=6) are independent. Therefore, the Fisher information matrix has 6 independent off-diagonal + 4 diagonal = 10 independent components. Aryal and Nadarajah[54] calculated Fisher's information matrix for the four parameter case as follows:

−1N∂2ln⁡L(α,β,a,c∣Y)∂α2=var⁡[ln⁡(X)]=ψ1(α)−ψ1(α+β)=Iα,α=E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂α2]=ln⁡(varGX){\displaystyle -{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha ^{2}}}=\operatorname {var} [\ln(X)]=\psi _{1}(\alpha )-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\alpha ,\alpha }=\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha ^{2}}}\right]=\ln(\operatorname {var_{GX}} )}−1N∂2ln⁡L(α,β,a,c∣Y)∂β2=var⁡[ln⁡(1−X)]=ψ1(β)−ψ1(α+β)=Iβ,β=E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂β2]=ln⁡(varG(1-X)){\displaystyle -{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta ^{2}}}=\operatorname {var} [\ln(1-X)]=\psi _{1}(\beta )-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\beta ,\beta }=\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta ^{2}}}\right]=\ln(\operatorname {var_{G(1-X)}} )}−1N∂2ln⁡L(α,β,a,c∣Y)∂α∂β=cov⁡[ln⁡X,(1−X)]=−ψ1(α+β)=Iα,β=E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂α∂β]=ln⁡(covGX,(1−X)){\displaystyle -{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha \,\partial \beta }}=\operatorname {cov} [\ln X,(1-X)]=-\psi _{1}(\alpha +\beta )={\mathcal {I}}_{\alpha ,\beta }=\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha \,\partial \beta }}\right]=\ln(\operatorname {cov} _{G{X,(1-X)}})}

In the above expressions, the use ofX instead ofY in the expressions var[ln(X)] = ln(varGX) isnot an error. The expressions in terms of the log geometric variances and log geometric covariance occur as functions of the two parameterX ~ Beta(α,β) parametrization because when taking the partial derivatives with respect to the exponents (α,β) in the four parameter case, one obtains the identical expressions as for the two parameter case: these terms of the four parameter Fisher information matrix are independent of the minimuma and maximumc of the distribution's range. The only non-zero term upon double differentiation of the log likelihood function with respect to the exponentsα andβ is the second derivative of the log of the beta function: ln(B(α,β)). This term is independent of the minimuma and maximumc of the distribution's range. Double differentiation of this term results in trigamma functions. The sections titled "Maximum likelihood", "Two unknown parameters" and "Four unknown parameters" also show this fact.

The Fisher information forNi.i.d. samples isN times the individual Fisher information (eq. 11.279, page 394 of Cover and Thomas[28]). (Aryal and Nadarajah[54] take a single observation,N = 1, to calculate the following components of the Fisher information, which leads to the same result as considering the derivatives of the log likelihood perN observations. Moreover, below the erroneous expression forIa,a{\displaystyle {\mathcal {I}}_{a,a}} in Aryal and Nadarajah has been corrected.)

α>2:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂a2]=Ia,a=β(α+β−1)(α−2)(c−a)2β>2:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂c2]=Ic,c=α(α+β−1)(β−2)(c−a)2E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂a∂c]=Ia,c=(α+β−1)(c−a)2α>1:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂α∂a]=Iα,a=β(α−1)(c−a)E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂α∂c]=Iα,c=1(c−a)E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂β∂a]=Iβ,a=−1(c−a)β>1:E⁡[−1N∂2ln⁡L(α,β,a,c∣Y)∂β∂c]=Iβ,c=−α(β−1)(c−a){\displaystyle {\begin{aligned}\alpha >2:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial a^{2}}}\right]&={\mathcal {I}}_{a,a}={\frac {\beta (\alpha +\beta -1)}{(\alpha -2)(c-a)^{2}}}\\\beta >2:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial c^{2}}}\right]&={\mathcal {I}}_{c,c}={\frac {\alpha (\alpha +\beta -1)}{(\beta -2)(c-a)^{2}}}\\\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial a\,\partial c}}\right]&={\mathcal {I}}_{a,c}={\frac {(\alpha +\beta -1)}{(c-a)^{2}}}\\\alpha >1:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha \,\partial a}}\right]&={\mathcal {I}}_{\alpha ,a}={\frac {\beta }{(\alpha -1)(c-a)}}\\\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \alpha \,\partial c}}\right]&={\mathcal {I}}_{\alpha ,c}={\frac {1}{(c-a)}}\\\operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta \,\partial a}}\right]&={\mathcal {I}}_{\beta ,a}=-{\frac {1}{(c-a)}}\\\beta >1:\quad \operatorname {E} \left[-{\frac {1}{N}}{\frac {\partial ^{2}\ln {\mathcal {L}}(\alpha ,\beta ,a,c\mid Y)}{\partial \beta \,\partial c}}\right]&={\mathcal {I}}_{\beta ,c}=-{\frac {\alpha }{(\beta -1)(c-a)}}\end{aligned}}}

The lower two diagonal entries of the Fisher information matrix, with respect to the parametera (the minimum of the distribution's range):Ia,a{\displaystyle {\mathcal {I}}_{a,a}}, and with respect to the parameterc (the maximum of the distribution's range):Ic,c{\displaystyle {\mathcal {I}}_{c,c}} are only defined for exponentsα > 2 andβ > 2 respectively. The Fisher information matrix componentIa,a{\displaystyle {\mathcal {I}}_{a,a}} for the minimuma approaches infinity for exponent α approaching 2 from above, and the Fisher information matrix componentIc,c{\displaystyle {\mathcal {I}}_{c,c}} for the maximumc approaches infinity for exponentβ approaching 2 from above.

The Fisher information matrix for the four parameter case does not depend on the individual values of the minimuma and the maximumc, but only on the total range (c − a). Moreover, the components of the Fisher information matrix that depend on the range (c − a), depend only through its inverse (or the square of the inverse), such that the Fisher information decreases for increasing range (c − a).

The accompanying images show the Fisher information componentsIa,a{\displaystyle {\mathcal {I}}_{a,a}} andIα,a{\displaystyle {\mathcal {I}}_{\alpha ,a}}. Images for the Fisher information componentsIα,α{\displaystyle {\mathcal {I}}_{\alpha ,\alpha }} andIβ,β{\displaystyle {\mathcal {I}}_{\beta ,\beta }} are shown in§ Geometric variance. All these Fisher information components look like a basin, with the "walls" of the basin being located at low values of the parameters.

The following four-parameter-beta-distribution Fisher information components can be expressed in terms of the two-parameter:X ~ Beta(α, β) expectations of the transformed ratio ((1 − X)/X) and of its mirror image (X/(1 − X)), scaled by the range (c − a), which may be helpful for interpretation:

Iα,a=E⁡[1−XX]c−a=β(α−1)(c−a) if α>1{\displaystyle {\mathcal {I}}_{\alpha ,a}={\frac {\operatorname {E} \left[{\frac {1-X}{X}}\right]}{c-a}}={\frac {\beta }{(\alpha -1)(c-a)}}{\text{ if }}\alpha >1}Iβ,c=−E⁡[X1−X]c−a=−α(β−1)(c−a) if β>1{\displaystyle {\mathcal {I}}_{\beta ,c}=-{\frac {\operatorname {E} \left[{\frac {X}{1-X}}\right]}{c-a}}=-{\frac {\alpha }{(\beta -1)(c-a)}}{\text{ if }}\beta >1}

These are also the expected values of the "inverted beta distribution" orbeta prime distribution (also known as beta distribution of the second kind orPearson's Type VI)[1] and its mirror image, scaled by the range (c − a).

Also, the following Fisher information components can be expressed in terms of the harmonic (1/X) variances or of variances based on the ratio transformed variables ((1-X)/X) as follows:

α>2:Ia,a=var⁡[1X](α−1c−a)2=var⁡[1−XX](α−1c−a)2=β(α+β−1)(α−2)(c−a)2β>2:Ic,c=var⁡[11−X](β−1c−a)2=var⁡[X1−X](β−1c−a)2=α(α+β−1)(β−2)(c−a)2Ia,c=cov⁡[1X,11−X](α−1)(β−1)(c−a)2=cov⁡[1−XX,X1−X](α−1)(β−1)(c−a)2=(α+β−1)(c−a)2{\displaystyle {\begin{aligned}\alpha >2:\quad {\mathcal {I}}_{a,a}&=\operatorname {var} \left[{\frac {1}{X}}\right]\left({\frac {\alpha -1}{c-a}}\right)^{2}=\operatorname {var} \left[{\frac {1-X}{X}}\right]\left({\frac {\alpha -1}{c-a}}\right)^{2}={\frac {\beta (\alpha +\beta -1)}{(\alpha -2)(c-a)^{2}}}\\\beta >2:\quad {\mathcal {I}}_{c,c}&=\operatorname {var} \left[{\frac {1}{1-X}}\right]\left({\frac {\beta -1}{c-a}}\right)^{2}=\operatorname {var} \left[{\frac {X}{1-X}}\right]\left({\frac {\beta -1}{c-a}}\right)^{2}={\frac {\alpha (\alpha +\beta -1)}{(\beta -2)(c-a)^{2}}}\\{\mathcal {I}}_{a,c}&=\operatorname {cov} \left[{\frac {1}{X}},{\frac {1}{1-X}}\right]{\frac {(\alpha -1)(\beta -1)}{(c-a)^{2}}}=\operatorname {cov} \left[{\frac {1-X}{X}},{\frac {X}{1-X}}\right]{\frac {(\alpha -1)(\beta -1)}{(c-a)^{2}}}={\frac {(\alpha +\beta -1)}{(c-a)^{2}}}\end{aligned}}}

See section "Moments of linearly transformed, product and inverted random variables" for these expectations.

The determinant of Fisher's information matrix is of interest (for example for the calculation ofJeffreys prior probability). From the expressions for the individual components, it follows that the determinant of Fisher's (symmetric) information matrix for the beta distribution with four parameters is:

det(I(α,β,a,c))=−Ia,c2Iα,aIα,β+Ia,aIa,cIα,cIα,β+Ia,c2Iα,β2−Ia,aIc,cIα,β2−Ia,cIα,aIα,cIβ,a+Ia,c2Iα,αIβ,a+2Ic,cIα,aIα,βIβ,a−2Ia,cIα,cIα,βIβ,a+Iα,c2Iβ,a2−Ic,cIα,αIβ,a2+Ia,cIα,a2Iβ,c−Ia,aIa,cIα,αIβ,c−Ia,cIα,aIα,βIβ,c+Ia,aIα,cIα,βIβ,c−Iα,aIα,cIβ,aIβ,c+Ia,cIα,αIβ,aIβ,c−Ic,cIα,a2Iβ,β+2Ia,cIα,aIα,cIβ,β−Ia,aIα,c2Iβ,β−Ia,c2Iα,αIβ,β+Ia,aIc,cIα,αIβ,β if α,β>2{\displaystyle {\begin{aligned}\det({\mathcal {I}}(\alpha ,\beta ,a,c))={}&-{\mathcal {I}}_{a,c}^{2}{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,\beta }+{\mathcal {I}}_{a,a}{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\alpha ,\beta }+{\mathcal {I}}_{a,c}^{2}{\mathcal {I}}_{\alpha ,\beta }^{2}-{\mathcal {I}}_{a,a}{\mathcal {I}}_{c,c}{\mathcal {I}}_{\alpha ,\beta }^{2}\\&{}-{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\beta ,a}+{\mathcal {I}}_{a,c}^{2}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,a}+2{\mathcal {I}}_{c,c}{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,\beta }{\mathcal {I}}_{\beta ,a}\\&{}-2{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\alpha ,\beta }{\mathcal {I}}_{\beta ,a}+{\mathcal {I}}_{\alpha ,c}^{2}{\mathcal {I}}_{\beta ,a}^{2}-{\mathcal {I}}_{c,c}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,a}^{2}+{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,a}^{2}{\mathcal {I}}_{\beta ,c}\\&{}-{\mathcal {I}}_{a,a}{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,c}-{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,\beta }{\mathcal {I}}_{\beta ,c}+{\mathcal {I}}_{a,a}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\alpha ,\beta }{\mathcal {I}}_{\beta ,c}\\&{}-{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\beta ,a}{\mathcal {I}}_{\beta ,c}+{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,a}{\mathcal {I}}_{\beta ,c}-{\mathcal {I}}_{c,c}{\mathcal {I}}_{\alpha ,a}^{2}{\mathcal {I}}_{\beta ,\beta }\\&{}+2{\mathcal {I}}_{a,c}{\mathcal {I}}_{\alpha ,a}{\mathcal {I}}_{\alpha ,c}{\mathcal {I}}_{\beta ,\beta }-{\mathcal {I}}_{a,a}{\mathcal {I}}_{\alpha ,c}^{2}{\mathcal {I}}_{\beta ,\beta }-{\mathcal {I}}_{a,c}^{2}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,\beta }+{\mathcal {I}}_{a,a}{\mathcal {I}}_{c,c}{\mathcal {I}}_{\alpha ,\alpha }{\mathcal {I}}_{\beta ,\beta }{\text{ if }}\alpha ,\beta >2\end{aligned}}}

UsingSylvester's criterion (checking whether the diagonal elements are all positive), and since diagonal componentsIa,a{\displaystyle {\mathcal {I}}_{a,a}} andIc,c{\displaystyle {\mathcal {I}}_{c,c}} havesingularities at α=2 and β=2 it follows that the Fisher information matrix for the four parameter case ispositive-definite for α>2 and β>2. Since for α > 2 and β > 2 the beta distribution is (symmetric or unsymmetric) bell shaped, it follows that the Fisher information matrix is positive-definite only for bell-shaped (symmetric or unsymmetric) beta distributions, with inflection points located to either side of the mode. Thus, important well known distributions belonging to the four-parameter beta distribution family, like the parabolic distribution (Beta(2,2,a,c)) and theuniform distribution (Beta(1,1,a,c)) have Fisher information components (Ia,a,Ic,c,Iα,a,Iβ,c{\displaystyle {\mathcal {I}}_{a,a},{\mathcal {I}}_{c,c},{\mathcal {I}}_{\alpha ,a},{\mathcal {I}}_{\beta ,c}}) that blow up (approach infinity) in the four-parameter case (although their Fisher information components are all defined for the two parameter case). The four-parameterWigner semicircle distribution (Beta(3/2,3/2,a,c)) andarcsine distribution (Beta(1/2,1/2,a,c)) have negative Fisher information determinants for the four-parameter case.

Bayesian inference

[edit]
Main article:Bayesian inference
Beta(1,1){\displaystyle Beta(1,1)}: Theuniform distribution probability density was proposed byThomas Bayes to represent ignorance of prior probabilities inBayesian inference.

The use of Beta distributions inBayesian inference is due to the fact that they provide a family ofconjugate prior probability distributions forbinomial (includingBernoulli) andgeometric distributions. The domain of the beta distribution can be viewed as a probability, and in fact the beta distribution is often used to describe the distribution of a probability valuep:[24]

P(p;α,β)=pα−1(1−p)β−1B(α,β).{\displaystyle P(p;\alpha ,\beta )={\frac {p^{\alpha -1}(1-p)^{\beta -1}}{\mathrm {B} (\alpha ,\beta )}}.}

Examples of beta distributions used as prior probabilities to represent ignorance of prior parameter values in Bayesian inference are Beta(1,1), Beta(0,0) and Beta(1/2,1/2).

Rule of succession

[edit]
Main article:Rule of succession

A classic application of the beta distribution is therule of succession, introduced in the 18th century byPierre-Simon Laplace[55] in the course of treating thesunrise problem. It states that, givens successes innconditionally independentBernoulli trials with probabilityp, that the estimate of the expected value in the next trial iss+1n+2{\displaystyle {\frac {s+1}{n+2}}}. This estimate is the expected value of the posterior distribution overp, namely Beta(s+1,n−s+1), which is given byBayes' rule if one assumes a uniform prior probability overp (i.e., Beta(1, 1)) and then observes thatp generateds successes inn trials. Laplace's rule of succession has been criticized by prominent scientists. R. T. Cox described Laplace's application of the rule of succession to thesunrise problem ([56] p. 89) as "a travesty of the proper use of the principle". Keynes remarks ([57] Ch.XXX, p. 382) "indeed this is so foolish a theorem that to entertain it is discreditable". Karl Pearson[58] showed that the probability that the next (n + 1) trials will be successes, after n successes in n trials, is only 50%, which has been considered too low by scientists like Jeffreys and unacceptable as a representation of the scientific process of experimentation to test a proposed scientific law. As pointed out by Jeffreys ([59] p. 128) (creditingC. D. Broad[60] ) Laplace's rule of succession establishes a high probability of success ((n+1)/(n+2)) in the next trial, but only a moderate probability (50%) that a further sample (n+1) comparable in size will be equally successful. As pointed out by Perks,[61] "The rule of succession itself is hard to accept. It assigns a probability to the next trial which implies the assumption that the actual run observed is an average run and that we are always at the end of an average run. It would, one would think, be more reasonable to assume that we were in the middle of an average run. Clearly a higher value for both probabilities is necessary if they are to accord with reasonable belief." These problems with Laplace's rule of succession motivated Haldane, Perks, Jeffreys and others to search for other forms of prior probability (see the next§ Bayesian inference). According to Jaynes,[52] the main problem with the rule of succession is that it is not valid when s=0 or s=n (seerule of succession, for an analysis of its validity).

Bayes–Laplace prior probability (Beta(1,1))

[edit]

The beta distribution achieves maximum differential entropy for Beta(1,1): theuniform probability density, for which all values in the domain of the distribution have equal density. This uniform distribution Beta(1,1) was suggested ("with a great deal of doubt") byThomas Bayes[62] as the prior probability distribution to express ignorance about the correct prior distribution. This prior distribution was adopted (apparently, from his writings, with little sign of doubt[55]) byPierre-Simon Laplace, and hence it was also known as the "Bayes–Laplace rule" or the "Laplace rule" of "inverse probability" in publications of the first half of the 20th century. In the later part of the 19th century and early part of the 20th century, scientists realized that the assumption of uniform "equal" probability density depended on the actual functions (for example whether a linear or a logarithmic scale was most appropriate) and parametrizations used. In particular, the behavior near the ends of distributions with finite support (for example nearx = 0, for a distribution with initial support atx = 0) required particular attention. Keynes ([57] Ch.XXX, p. 381) criticized the use of Bayes's uniform prior probability (Beta(1,1)) that all values between zero and one are equiprobable, as follows: "Thus experience, if it shows anything, shows that there is a very marked clustering of statistical ratios in the neighborhoods of zero and unity, of those for positive theories and for correlations between positive qualities in the neighborhood of zero, and of those for negative theories and for correlations between negative qualities in the neighborhood of unity. "

Haldane's prior probability (Beta(0,0))

[edit]
Beta(0,0){\displaystyle Beta(0,0)}: The Haldane prior probability expressing total ignorance about prior information, where we are not even sure whether it is physically possible for an experiment to yield either a success or a failure. As α, β → 0, the beta distribution approaches a two-pointBernoulli distribution with all probability density concentrated at each end, at 0 and 1, and nothing in between. A coin-toss: one face of the coin being at 0 and the other face being at 1.

The Beta(0,0) distribution was proposed byJ.B.S. Haldane,[63] who suggested that the prior probability representing complete uncertainty should be proportional top−1(1−p)−1. The functionp−1(1−p)−1 can be viewed as the limit of the numerator of the beta distribution as both shape parameters approach zero: α, β → 0. The Beta function (in the denominator of the beta distribution) approaches infinity, for both parameters approaching zero, α, β → 0. Therefore,p−1(1−p)−1 divided by the Beta function approaches a 2-pointBernoulli distribution with equal probability 1/2 at each end, at 0 and 1, and nothing in between, as α, β → 0. A coin-toss: one face of the coin being at 0 and the other face being at 1. The Haldane prior probability distribution Beta(0,0) is an "improper prior" because its integration (from 0 to 1) fails to strictly converge to 1 due to the singularities at each end. However, this is not an issue for computing posterior probabilities unless the sample size is very small. Furthermore, Zellner[64] points out that on thelog-odds scale, (thelogit transformationlog⁡(p/(1−p)){\displaystyle \log(p/(1-p))}), the Haldane prior is the uniformly flat prior. The fact that a uniform prior probability on thelogit transformed variable ln(p/1 − p) (with domain (−∞, ∞)) is equivalent to the Haldane prior on the domain [0, 1] was pointed out byHarold Jeffreys in the first edition (1939) of his book Theory of Probability ([59] p. 123). Jeffreys writes "Certainly if we take the Bayes–Laplace rule right up to the extremes we are led to results that do not correspond to anybody's way of thinking. The (Haldane) rule dx/(x(1 − x)) goes too far the other way. It would lead to the conclusion that if a sample is of one type with respect to some property there is a probability 1 that the whole population is of that type." The fact that "uniform" depends on the parametrization, led Jeffreys to seek a form of prior that would be invariant under different parametrizations.

Jeffreys' prior probability (Beta(1/2,1/2) for a Bernoulli or for a binomial distribution)

[edit]
Main article:Jeffreys prior
Jeffreys prior probability for the beta distribution: the square root of the determinant ofFisher's information matrix:det(I(α,β))=ψ1(α)ψ1(β)−(ψ1(α)+ψ1(β))ψ1(α+β){\displaystyle \scriptstyle {\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}={\sqrt {\psi _{1}(\alpha )\psi _{1}(\beta )-(\psi _{1}(\alpha )+\psi _{1}(\beta ))\psi _{1}(\alpha +\beta )}}} is a function of thetrigamma function ψ1 of shape parameters α, β
Posterior Beta densities with samples having success = "s", failure = "f" ofs/(s +f) = 1/2, ands +f = {3,10,50}, based on 3 different prior probability functions: Haldane (Beta(0,0), Jeffreys (Beta(1/2,1/2)) and Bayes (Beta(1,1)). The image shows that there is little difference between the priors for the posterior with sample size of 50 (with more pronounced peak nearp = 1/2). Significant differences appear for very small sample sizes (the flatter distribution for sample size of 3)
Posterior Beta densities with samples having success = "s", failure = "f" ofs/(s +f) = 1/4, ands +f ∈ {3,10,50}, based on three different prior probability functions: Haldane (Beta(0,0), Jeffreys (Beta(1/2,1/2)) and Bayes (Beta(1,1)). The image shows that there is little difference between the priors for the posterior with sample size of 50 (with more pronounced peak nearp = 1/4). Significant differences appear for very small sample sizes (the very skewed distribution for the degenerate case of sample size = 3, in this degenerate and unlikely case the Haldane prior results in a reverse "J" shape with mode atp = 0 instead ofp = 1/4. If there is sufficientsampling data, the three priors of Bayes (Beta(1,1)), Jeffreys (Beta(1/2,1/2)) and Haldane (Beta(0,0)) should yield similarposterior probability densities.
Posterior Beta densities with samples having success =s, failure =f ofs/(s +f) = 1/4, ands +f ∈ {4,12,40}, based on three different prior probability functions: Haldane (Beta(0,0), Jeffreys (Beta(1/2,1/2)) and Bayes (Beta(1,1)). The image shows that there is little difference between the priors for the posterior with sample size of 40 (with more pronounced peak nearp = 1/4). Significant differences appear for very small sample sizes

Harold Jeffreys[59][65] proposed to use anuninformative prior probability measure that should beinvariant under reparameterization: proportional to the square root of thedeterminant ofFisher's information matrix. For theBernoulli distribution, this can be shown as follows: for a coin that is "heads" with probabilityp ∈ [0, 1] and is "tails" with probability 1 −p, for a given (H,T) ∈ {(0,1), (1,0)} the probability ispH(1 −p)T. SinceT = 1 −H, theBernoulli distribution ispH(1 −p)1 −H. Consideringp as the only parameter, it follows that the log likelihood for the Bernoulli distribution is

ln⁡L(p∣H)=Hln⁡p+(1−H)ln⁡(1−p).{\displaystyle \ln {\mathcal {L}}(p\mid H)=H\ln p+(1-H)\ln(1-p).}

The Fisher information matrix has only one component (it is a scalar, because there is only one parameter:p), therefore:

I(p)=E[(ddpln⁡L(p∣H))2]=E[(Hp−1−H1−p)2]=p1(1−p)0(1p−01−p)2+p0(1−p)1(0p−11−p)2=1p(1−p).{\displaystyle {\begin{aligned}{\sqrt {{\mathcal {I}}(p)}}&={\sqrt {\operatorname {E} \!\left[\left({\frac {d}{dp}}\ln {\mathcal {L}}(p\mid H)\right)^{2}\right]}}\\[6pt]&={\sqrt {\operatorname {E} \!\left[\left({\frac {H}{p}}-{\frac {1-H}{1-p}}\right)^{2}\right]}}\\[6pt]&={\sqrt {p^{1}(1-p)^{0}\left({\frac {1}{p}}-{\frac {0}{1-p}}\right)^{2}+p^{0}(1-p)^{1}\left({\frac {0}{p}}-{\frac {1}{1-p}}\right)^{2}}}\\&={\frac {1}{\sqrt {p(1-p)}}}.\end{aligned}}}

Similarly, for theBinomial distribution withnBernoulli trials, it can be shown that

I(p)=np(1−p).{\displaystyle {\sqrt {{\mathcal {I}}(p)}}={\sqrt {\frac {n}{p(1-p)}}}.}

Thus, for theBernoulli, andBinomial distributions,Jeffreys prior is proportional to1p(1−p){\displaystyle \scriptstyle {\frac {1}{\sqrt {p(1-p)}}}}, which happens to be proportional to a beta distribution with domain variablex =p, and shape parameters α = β = 1/2, thearcsine distribution:

Beta⁡(12,12)=1πp(1−p).{\displaystyle \operatorname {Beta} ({\tfrac {1}{2}},{\tfrac {1}{2}})={\frac {1}{\pi {\sqrt {p(1-p)}}}}.}

It will be shown in the next section that the normalizing constant for Jeffreys prior is immaterial to the final result because the normalizing constant cancels out in Bayes' theorem for the posterior probability. Hence Beta(1/2,1/2) is used as the Jeffreys prior for both Bernoulli and binomial distributions. As shown in the next section, when using this expression as a prior probability times the likelihood inBayes' theorem, the posterior probability turns out to be a beta distribution. It is important to realize, however, that Jeffreys prior is proportional to1p(1−p){\textstyle {\frac {1}{\sqrt {p(1-p)}}}} for the Bernoulli and binomial distribution, but not for the beta distribution. Jeffreys prior for the beta distribution is given by the determinant of Fisher's information for the beta distribution, which, as shown in the§ Fisher information matrix is a function of thetrigamma function ψ1 of shape parameters α and β as follows:

det(I(α,β))=ψ1(α)ψ1(β)−(ψ1(α)+ψ1(β))ψ1(α+β)limα→0det(I(α,β))=limβ→0det(I(α,β))=∞limα→∞det(I(α,β))=limβ→∞det(I(α,β))=0{\displaystyle {\begin{aligned}{\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}&={\sqrt {\psi _{1}(\alpha )\psi _{1}(\beta )-(\psi _{1}(\alpha )+\psi _{1}(\beta ))\psi _{1}(\alpha +\beta )}}\\\lim _{\alpha \to 0}{\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}&=\lim _{\beta \to 0}{\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}=\infty \\\lim _{\alpha \to \infty }{\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}&=\lim _{\beta \to \infty }{\sqrt {\det({\mathcal {I}}(\alpha ,\beta ))}}=0\end{aligned}}}

As previously discussed, Jeffreys prior for the Bernoulli and binomial distributions is proportional to thearcsine distribution Beta(1/2,1/2), a one-dimensionalcurve that looks like a basin as a function of the parameterp of the Bernoulli and binomial distributions. The walls of the basin are formed byp approaching the singularities at the endsp → 0 andp → 1, where Beta(1/2,1/2) approaches infinity. Jeffreys prior for the beta distribution is a2-dimensional surface (embedded in a three-dimensional space) that looks like a basin with only two of its walls meeting at the corner α = β = 0 (and missing the other two walls) as a function of the shape parameters α and β of the beta distribution. The two adjoining walls of this 2-dimensional surface are formed by the shape parameters α and β approaching the singularities (of the trigamma function) at α, β → 0. It has no walls for α, β → ∞ because in this case the determinant of Fisher's information matrix for the beta distribution approaches zero.

It will be shown in the next section that Jeffreys prior probability results in posterior probabilities (when multiplied by the binomial likelihood function) that are intermediate between the posterior probability results of the Haldane and Bayes prior probabilities.

Jeffreys prior may be difficult to obtain analytically, and for some cases it just doesn't exist (even for simple distribution functions like the asymmetrictriangular distribution). Berger, Bernardo and Sun, in a 2009 paper[66] defined a reference prior probability distribution that (unlike Jeffreys prior) exists for the asymmetrictriangular distribution. They cannot obtain a closed-form expression for their reference prior, but numerical calculations show it to be nearly perfectly fitted by the (proper) prior

Beta⁡(12,12)∼1θ(1−θ){\displaystyle \operatorname {Beta} ({\tfrac {1}{2}},{\tfrac {1}{2}})\sim {\frac {1}{\sqrt {\theta (1-\theta )}}}}

where θ is the vertex variable for the asymmetric triangular distribution with support [0, 1] (corresponding to the following parameter values in Wikipedia's article on thetriangular distribution: vertexc =θ, left enda = 0, and right endb = 1). Berger et al. also give a heuristic argument that Beta(1/2,1/2) could indeed be the exact Berger–Bernardo–Sun reference prior for the asymmetric triangular distribution. Therefore, Beta(1/2,1/2) not only is Jeffreys prior for the Bernoulli and binomial distributions, but also seems to be the Berger–Bernardo–Sun reference prior for the asymmetric triangular distribution (for which the Jeffreys prior does not exist), a distribution used in project management andPERT analysis to describe the cost and duration of project tasks.

Clarke and Barron[67] prove that, among continuous positive priors, Jeffreys prior (when it exists) asymptotically maximizes Shannon'smutual information between a sample of size n and the parameter, and thereforeJeffreys prior is the most uninformative prior (measuring information as Shannon information). The proof rests on an examination of theKullback–Leibler divergence between probability density functions foriid random variables.

Effect of different prior probability choices on the posterior beta distribution

[edit]

If samples are drawn from the population of a random variableX that result ins successes andf failures innBernoulli trialsn = s + f, then thelikelihood function for parameterss andf givenx = p (the notationx = p in the expressions below will emphasize that the domainx stands for the value of the parameterp in the binomial distribution), is the followingbinomial distribution:

L(s,f∣x=p)=(s+fs)xs(1−x)f=(ns)xs(1−x)n−s.{\displaystyle {\mathcal {L}}(s,f\mid x=p)={s+f \choose s}x^{s}(1-x)^{f}={n \choose s}x^{s}(1-x)^{n-s}.}

If beliefs aboutprior probability information are reasonably well approximated by a beta distribution with parametersα Prior andβ Prior, then:

PriorProbability(x=p;αPrior,βPrior)=xαPrior−1(1−x)βPrior−1B(αPrior,βPrior){\displaystyle {\operatorname {PriorProbability} }(x=p;\alpha \operatorname {Prior} ,\beta \operatorname {Prior} )={\frac {x^{\alpha \operatorname {Prior} -1}(1-x)^{\beta \operatorname {Prior} -1}}{\mathrm {B} (\alpha \operatorname {Prior} ,\beta \operatorname {Prior} )}}}

According toBayes' theorem for a continuous event space, theposterior probability density is given by the product of theprior probability and the likelihood function (given the evidences andf = n − s), normalized so that the area under the curve equals one, as follows:

posterior probability density(x=p∣s,n−s)=priorprobabilitydensity⁡(x=p;αprior,βprior)L(s,f∣x=p)∫01prior probability density(x=p;αprior,βprior)L(s,f∣x=p)dx=(ns)xs+αprior−1(1−x)n−s+βprior−1/B(αprior,βprior)∫01((ns)xs+αprior−1(1−x)n−s+βprior−1/B(αprior,βprior))dx=xs+αprior−1(1−x)n−s+βprior−1∫01(xs+αprior−1(1−x)n−s+βprior−1)dx=xs+αprior−1(1−x)n−s+βprior−1B(s+αprior,n−s+βprior).{\displaystyle {\begin{aligned}&{\text{posterior probability density}}(x=p\mid s,n-s)\\[6pt]={}&{\frac {\operatorname {priorprobabilitydensity} (x=p;\alpha \operatorname {prior} ,\beta \operatorname {prior} ){\mathcal {L}}(s,f\mid x=p)}{\int _{0}^{1}{\text{prior probability density}}(x=p;\alpha \operatorname {prior} ,\beta \operatorname {prior} ){\mathcal {L}}(s,f\mid x=p)\,dx}}\\[6pt]={}&{\frac {{n \choose s}x^{s+\alpha \operatorname {prior} -1}(1-x)^{n-s+\beta \operatorname {prior} -1}/\mathrm {B} (\alpha \operatorname {prior} ,\beta \operatorname {prior} )}{\int _{0}^{1}\left({n \choose s}x^{s+\alpha \operatorname {prior} -1}(1-x)^{n-s+\beta \operatorname {prior} -1}/\mathrm {B} (\alpha \operatorname {prior} ,\beta \operatorname {prior} )\right)\,dx}}\\[6pt]={}&{\frac {x^{s+\alpha \operatorname {prior} -1}(1-x)^{n-s+\beta \operatorname {prior} -1}}{\int _{0}^{1}\left(x^{s+\alpha \operatorname {prior} -1}(1-x)^{n-s+\beta \operatorname {prior} -1}\right)\,dx}}\\[6pt]={}&{\frac {x^{s+\alpha \operatorname {prior} -1}(1-x)^{n-s+\beta \operatorname {prior} -1}}{\mathrm {B} (s+\alpha \operatorname {prior} ,n-s+\beta \operatorname {prior} )}}.\end{aligned}}}

Thebinomial coefficient

(s+fs)=(ns)=(s+f)!s!f!=n!s!(n−s)!{\displaystyle {s+f \choose s}={n \choose s}={\frac {(s+f)!}{s!f!}}={\frac {n!}{s!(n-s)!}}}

appears both in the numerator and the denominator of the posterior probability, and it does not depend on the integration variablex, hence it cancels out, and it is irrelevant to the final result. Similarly the normalizing factor for the prior probability, the beta function B(αPrior,βPrior) cancels out and it is immaterial to the final result. The same posterior probability result can be obtained if one uses an un-normalized prior

xαprior−1(1−x)βprior−1{\displaystyle x^{\alpha \operatorname {prior} -1}(1-x)^{\beta \operatorname {prior} -1}}

because the normalizing factors all cancel out. Several authors (including Jeffreys himself) thus use an un-normalized prior formula since the normalization constant cancels out. The numerator of the posterior probability ends up being just the (un-normalized) product of the prior probability and the likelihood function, and the denominator is its integral from zero to one. The beta function in the denominator, B(s + α Prior, n − s + β Prior), appears as a normalization constant to ensure that the total posterior probability integrates to unity.

The ratios/n of the number of successes to the total number of trials is asufficient statistic in the binomial case, which is relevant for the following results.

For theBayes' prior probability (Beta(1,1)), the posterior probability is:

posteriorprobability⁡(p=x∣s,f)=xs(1−x)n−sB(s+1,n−s+1), with mean =s+1n+2, (and mode=sn if 0<s<n).{\displaystyle \operatorname {posteriorprobability} (p=x\mid s,f)={\frac {x^{s}(1-x)^{n-s}}{\mathrm {B} (s+1,n-s+1)}},{\text{ with mean }}={\frac {s+1}{n+2}},{\text{ (and mode}}={\frac {s}{n}}{\text{ if }}0<s<n).}

For theJeffreys' prior probability (Beta(1/2,1/2)), the posterior probability is:

posteriorprobability⁡(p=x∣s,f)=xs−12(1−x)n−s−12B(s+12,n−s+12), with mean=s+12n+1, (and mode=s−12n−1 if 12<s<n−12).{\displaystyle \operatorname {posteriorprobability} (p=x\mid s,f)={x^{s-{\tfrac {1}{2}}}(1-x)^{n-s-{\frac {1}{2}}} \over \mathrm {B} (s+{\tfrac {1}{2}},n-s+{\tfrac {1}{2}})},{\text{ with mean}}={\frac {s+{\tfrac {1}{2}}}{n+1}},{\text{ (and mode}}={\frac {s-{\tfrac {1}{2}}}{n-1}}{\text{ if }}{\tfrac {1}{2}}<s<n-{\tfrac {1}{2}}).}

and for theHaldane prior probability (Beta(0,0)), the posterior probability is:

posteriorprobability⁡(p=x∣s,f)=xs−1(1−x)n−s−1B(s,n−s), with mean=sn, (and mode=s−1n−2 if 1<s<n−1).{\displaystyle \operatorname {posteriorprobability} (p=x\mid s,f)={\frac {x^{s-1}(1-x)^{n-s-1}}{\mathrm {B} (s,n-s)}},{\text{ with mean}}={\frac {s}{n}},{\text{ (and mode}}={\frac {s-1}{n-2}}{\text{ if }}1<s<n-1).}

From the above expressions it follows that fors/n = 1/2) all the above three prior probabilities result in the identical location for the posterior probability mean = mode = 1/2. Fors/n < 1/2, the mean of the posterior probabilities, using the following priors, are such that: mean for Bayes prior > mean for Jeffreys prior > mean for Haldane prior. Fors/n > 1/2 the order of these inequalities is reversed such that the Haldane prior probability results in the largest posterior mean. TheHaldane prior probability Beta(0,0) results in a posterior probability density withmean (the expected value for the probability of success in the "next" trial) identical to the ratios/n of the number of successes to the total number of trials. Therefore, the Haldane prior results in a posterior probability with expected value in the next trial equal to the maximum likelihood. TheBayes prior probability Beta(1,1) results in a posterior probability density withmode identical to the ratios/n (the maximum likelihood).

In the case that 100% of the trials have been successfuls = n, theBayes prior probability Beta(1,1) results in a posterior expected value equal to the rule of succession (n + 1)/(n + 2), while the Haldane prior Beta(0,0) results in a posterior expected value of 1 (absolute certainty of success in the next trial). Jeffreys prior probability results in a posterior expected value equal to (n + 1/2)/(n + 1). Perks[61] (p. 303) points out: "This provides a new rule of succession and expresses a 'reasonable' position to take up, namely, that after an unbroken run of n successes we assume a probability for the next trial equivalent to the assumption that we are about half-way through an average run, i.e. that we expect a failure once in (2n + 2) trials. The Bayes–Laplace rule implies that we are about at the end of an average run or that we expect a failure once in (n + 2) trials. The comparison clearly favours the new result (what is now called Jeffreys prior) from the point of view of 'reasonableness'."

Conversely, in the case that 100% of the trials have resulted in failure (s = 0), theBayes prior probability Beta(1,1) results in a posterior expected value for success in the next trial equal to 1/(n + 2), while the Haldane prior Beta(0,0) results in a posterior expected value of success in the next trial of 0 (absolute certainty of failure in the next trial). Jeffreys prior probability results in a posterior expected value for success in the next trial equal to (1/2)/(n + 1), which Perks[61] (p. 303) points out: "is a much more reasonably remote result than the Bayes–Laplace result 1/(n + 2)".

Jaynes[52] questions (for the Haldane prior Beta(0,0)) the use of these formulas for the casess = 0 ors = n because the integrals do not converge (Beta(0,0) is an improper prior fors = 0 ors = n). In practice, the conditions 0<s<n necessary for a mode to exist between both ends for the Bayes prior are usually met, and therefore the Bayes prior (as long as 0 < s < n) results in a posterior mode located between both ends of the domain.

As remarked in the section on the rule of succession, K. Pearson showed that aftern successes inn trials the posterior probability (based on the Bayes Beta(1,1) distribution as the prior probability) that the next (n + 1) trials will all be successes is exactly 1/2, whatever the value of n. Based on the Haldane Beta(0,0) distribution as the prior probability, this posterior probability is 1 (absolute certainty that after n successes inn trials the next (n + 1) trials will all be successes). Perks[61] (p. 303) shows that, for what is now known as the Jeffreys prior, this probability is ((n + 1/2)/(n + 1))((n + 3/2)/(n + 2))...(2n + 1/2)/(2n + 1), which forn = 1, 2, 3 gives 15/24, 315/480, 9009/13440; rapidly approaching a limiting value of1/2=0.70710678…{\displaystyle 1/{\sqrt {2}}=0.70710678\ldots } as n tends to infinity. Perks remarks that what is now known as the Jeffreys prior: "is clearly more 'reasonable' than either the Bayes–Laplace result or the result on the (Haldane) alternative rule rejected by Jeffreys which gives certainty as the probability. It clearly provides a very much better correspondence with the process of induction. Whether it is 'absolutely' reasonable for the purpose, i.e. whether it is yet large enough, without the absurdity of reaching unity, is a matter for others to decide. But it must be realized that the result depends on the assumption of complete indifference and absence of knowledge prior to the sampling experiment."

Following are the variances of the posterior distribution obtained with these three prior probability distributions:

for theBayes' prior probability (Beta(1,1)), the posterior variance is:

variance=(n−s+1)(s+1)(3+n)(2+n)2, which for s=n2 results in variance=112+4n{\displaystyle {\text{variance}}={\frac {(n-s+1)(s+1)}{(3+n)(2+n)^{2}}},{\text{ which for }}s={\frac {n}{2}}{\text{ results in variance}}={\frac {1}{12+4n}}}

for theJeffreys' prior probability (Beta(1/2,1/2)), the posterior variance is:

variance=(n−s+12)(s+12)(2+n)(1+n)2, which for s=n2 results in var=18+4n{\displaystyle {\text{variance}}={\frac {(n-s+{\frac {1}{2}})(s+{\frac {1}{2}})}{(2+n)(1+n)^{2}}},{\text{ which for }}s={\frac {n}{2}}{\text{ results in var}}={\frac {1}{8+4n}}}

and for theHaldane prior probability (Beta(0,0)), the posterior variance is:

variance=(n−s)s(1+n)n2, which for s=n2 results in variance=14+4n{\displaystyle {\text{variance}}={\frac {(n-s)s}{(1+n)n^{2}}},{\text{ which for }}s={\frac {n}{2}}{\text{ results in variance}}={\frac {1}{4+4n}}}

So, as remarked by Silvey,[50] for largen, the variance is small and hence the posterior distribution is highly concentrated, whereas the assumed prior distribution was very diffuse. This is in accord with what one would hope for, as vague prior knowledge is transformed (through Bayes' theorem) into a more precise posterior knowledge by an informative experiment. For smalln the Haldane Beta(0,0) prior results in the largest posterior variance while the Bayes Beta(1,1) prior results in the more concentrated posterior. Jeffreys prior Beta(1/2,1/2) results in a posterior variance in between the other two. Asn increases, the variance rapidly decreases so that the posterior variance for all three priors converges to approximately the same value (approaching zero variance asn → ∞). Recalling the previous result that theHaldane prior probability Beta(0,0) results in a posterior probability density withmean (the expected value for the probability of success in the "next" trial) identical to the ratio s/n of the number of successes to the total number of trials, it follows from the above expression that also theHaldane prior Beta(0,0) results in a posterior withvariance identical to the variance expressed in terms of the max. likelihood estimate s/n and sample size (in§ Variance):

variance=μ(1−μ)1+ν=(n−s)s(1+n)n2{\displaystyle {\text{variance}}={\frac {\mu (1-\mu )}{1+\nu }}={\frac {(n-s)s}{(1+n)n^{2}}}}

with the meanμ = s/n and the sample size ν = n.

In Bayesian inference, using aprior distribution Beta(αPrior,βPrior) prior to a binomial distribution is equivalent to adding (αPrior − 1) pseudo-observations of "success" and (βPrior − 1) pseudo-observations of "failure" to the actual number of successes and failures observed, then estimating the parameterp of the binomial distribution by the proportion of successes over both real- and pseudo-observations. A uniform prior Beta(1,1) does not add (or subtract) any pseudo-observations since for Beta(1,1) it follows that (αPrior − 1) = 0 and (βPrior − 1) = 0. The Haldane prior Beta(0,0) subtracts one pseudo observation from each and Jeffreys prior Beta(1/2,1/2) subtracts 1/2 pseudo-observation of success and an equal number of failure. This subtraction has the effect ofsmoothing out the posterior distribution. If the proportion of successes is not 50% (s/n ≠ 1/2) values ofαPrior andβPrior less than 1 (and therefore negative (αPrior − 1) and (βPrior − 1)) favor sparsity, i.e. distributions where the parameterp is closer to either 0 or 1. In effect, values ofαPrior andβPrior between 0 and 1, when operating together, function as aconcentration parameter.

The accompanying plots show the posterior probability density functions for sample sizesn ∈ {3,10,50}, successess ∈ {n/2,n/4} and Beta(αPrior,βPrior) ∈ {Beta(0,0),Beta(1/2,1/2),Beta(1,1)}. Also shown are the cases forn = {4,12,40}, successs = {n/4} and Beta(αPrior,βPrior) ∈ {Beta(0,0),Beta(1/2,1/2),Beta(1,1)}. The first plot shows the symmetric cases, for successess ∈ {n/2}, with mean = mode = 1/2 and the second plot shows the skewed casess ∈ {n/4}. The images show that there is little difference between the priors for the posterior with sample size of 50 (characterized by a more pronounced peak nearp = 1/2). Significant differences appear for very small sample sizes (in particular for the flatter distribution for the degenerate case of sample size = 3). Therefore, the skewed cases, with successess = {n/4}, show a larger effect from the choice of prior, at small sample size, than the symmetric cases. For symmetric distributions, the Bayes prior Beta(1,1) results in the most "peaky" and highest posterior distributions and the Haldane prior Beta(0,0) results in the flattest and lowest peak distribution. The Jeffreys prior Beta(1/2,1/2) lies in between them. For nearly symmetric, not too skewed distributions the effect of the priors is similar. For very small sample size (in this case for a sample size of 3) and skewed distribution (in this example fors ∈ {n/4}) the Haldane prior can result in a reverse-J-shaped distribution with a singularity at the left end. However, this happens only in degenerate cases (in this examplen = 3 and hences = 3/4 < 1, a degenerate value because s should be greater than unity in order for the posterior of the Haldane prior to have a mode located between the ends, and becauses = 3/4 is not an integer number, hence it violates the initial assumption of a binomial distribution for the likelihood) and it is not an issue in generic cases of reasonable sample size (such that the condition 1 < s < n − 1, necessary for a mode to exist between both ends, is fulfilled).

In Chapter 12 (p. 385) of his book, Jaynes[52] asserts that theHaldane prior Beta(0,0) describes aprior state of knowledge of complete ignorance, where we are not even sure whether it is physically possible for an experiment to yield either a success or a failure, while theBayes (uniform) prior Beta(1,1) applies if one knows thatboth binary outcomes are possible. Jaynes states: "interpret the Bayes–Laplace (Beta(1,1)) prior as describing not a state of complete ignorance, but the state of knowledge in which we have observed one success and one failure...once we have seen at least one success and one failure, then we know that the experiment is a true binary one, in the sense of physical possibility." Jaynes[52] does not specifically discuss Jeffreys prior Beta(1/2,1/2) (Jaynes discussion of "Jeffreys prior" on pp. 181, 423 and on chapter 12 of Jaynes book[52] refers instead to the improper, un-normalized, prior "1/p dp" introduced by Jeffreys in the 1939 edition of his book,[59] seven years before he introduced what is now known as Jeffreys' invariant prior: the square root of the determinant of Fisher's information matrix."1/p" is Jeffreys' (1946) invariant prior for theexponential distribution, not for the Bernoulli or binomial distributions). However, it follows from the above discussion that Jeffreys Beta(1/2,1/2) prior represents a state of knowledge in between the Haldane Beta(0,0) and Bayes Beta (1,1) prior.

Similarly,Karl Pearson in his 1892 bookThe Grammar of Science[68][69] (p. 144 of 1900 edition) maintained that the Bayes (Beta(1,1) uniform prior was not a complete ignorance prior, and that it should be used when prior information justified to "distribute our ignorance equally"". K. Pearson wrote: "Yet the only supposition that we appear to have made is this: that, knowing nothing of nature, routine and anomy (from the Greek ανομία, namely: a- "without", and nomos "law") are to be considered as equally likely to occur. Now we were not really justified in making even this assumption, for it involves a knowledge that we do not possess regarding nature. We use ourexperience of the constitution and action of coins in general to assert that heads and tails are equally probable, but we have no right to assert before experience that, as we know nothing of nature, routine and breach are equally probable. In our ignorance we ought to consider before experience that nature may consist of all routines, all anomies (normlessness), or a mixture of the two in any proportion whatever, and that all such are equally probable. Which of these constitutions after experience is the most probable must clearly depend on what that experience has been like."

If there is sufficientsampling data,and the posterior probability mode is not located at one of the extremes of the domain (x = 0 orx = 1), the three priors of Bayes (Beta(1,1)), Jeffreys (Beta(1/2,1/2)) and Haldane (Beta(0,0)) should yield similarposterior probability densities. Otherwise, as Gelman et al.[70] (p. 65) point out, "if so few data are available that the choice of noninformative prior distribution makes a difference, one should put relevant information into the prior distribution", or as Berger[4] (p. 125) points out "when different reasonable priors yield substantially different answers, can it be right to state that thereis a single answer? Would it not be better to admit that there is scientific uncertainty, with the conclusion depending on prior beliefs?."

Occurrence and applications

[edit]

Order statistics

[edit]
Main article:Order statistic

The beta distribution has an important application in the theory oforder statistics. A basic result is that the distribution of thekth smallest of a sample of sizen from a continuousuniform distribution has a beta distribution.[40] This result is summarized as

U(k)∼Beta⁡(k,n+1−k).{\displaystyle U_{(k)}\sim \operatorname {Beta} (k,n+1-k).}

From this, and application of the theory related to theprobability integral transform, the distribution of any individual order statistic from anycontinuous distribution can be derived.[40]

Subjective logic

[edit]
Main article:Subjective logic

In standard logic, propositions are considered to be either true or false. In contradistinction,subjective logic assumes that humans cannot determine with absolute certainty whether a proposition about the real world is absolutely true or false. Insubjective logic theposteriori probability estimates of binary events can be represented by beta distributions.[71]

Wavelet analysis

[edit]
Main article:Beta wavelet

Awavelet is a wave-likeoscillation with anamplitude that starts out at zero, increases, and then decreases back to zero. It can typically be visualized as a "brief oscillation" that promptly decays. Wavelets can be used to extract information from many different kinds of data, including – but certainly not limited to – audio signals and images. Thus, wavelets are purposefully crafted to have specific properties that make them useful forsignal processing. Wavelets are localized in both time andfrequency whereas the standardFourier transform is only localized in frequency. Therefore, standard Fourier Transforms are only applicable tostationary processes, whilewavelets are applicable to non-stationary processes. Continuous wavelets can be constructed based on the beta distribution.Beta wavelets[72] can be viewed as a soft variety ofHaar wavelets whose shape is fine-tuned by two shape parameters α and β.

Population genetics

[edit]
Main article:Balding–Nichols model
Further information:F-statistics,Fixation index, andCoefficient of relationship

TheBalding–Nichols model is a two-parameterparametrization of the beta distribution used inpopulation genetics.[73] It is a statistical description of theallele frequencies in the components of a sub-divided population:

α=μν,β=(1−μ)ν,{\displaystyle {\begin{aligned}\alpha &=\mu \nu ,\\\beta &=(1-\mu )\nu ,\end{aligned}}}whereν=α+β=1−FF{\displaystyle \nu =\alpha +\beta ={\frac {1-F}{F}}} and0<F<1{\displaystyle 0<F<1}; hereF is (Wright's) genetic distance between two populations.

Project management: task cost and schedule modeling

[edit]

The beta distribution can be used to model events which are constrained to take place within an interval defined by a minimum and maximum value. For this reason, the beta distribution — along with thetriangular distribution — is used extensively inPERT,critical path method (CPM), Joint Cost Schedule Modeling (JCSM) and otherproject management/control systems to describe the time to completion and the cost of a task. In project management, shorthand computations are widely used to estimate themean andstandard deviation of the beta distribution:[39]

μ(X)=a+4b+c6σ(X)=c−a6{\displaystyle {\begin{aligned}\mu (X)&={\frac {a+4b+c}{6}}\\[8pt]\sigma (X)&={\frac {c-a}{6}}\end{aligned}}}

wherea is the minimum,c is the maximum, andb is the most likely value (themode forα > 1 andβ > 1).

The above estimate for themeanμ(X)=a+4b+c6{\displaystyle \mu (X)={\frac {a+4b+c}{6}}} is known as thePERTthree-point estimation and it is exact for either of the following values ofβ (for arbitrary α within these ranges):

β =α > 1 (symmetric case) withstandard deviationσ(X)=c−a21+2α{\displaystyle \sigma (X)={\frac {c-a}{2{\sqrt {1+2\alpha }}}}},skewness = 0, andexcess kurtosis =−63+2α{\displaystyle {\frac {-6}{3+2\alpha }}}

or

β = 6 −α for 5 >α > 1 (skewed case) withstandard deviation

σ(X)=(c−a)α(6−α)67,{\displaystyle \sigma (X)={\frac {(c-a){\sqrt {\alpha (6-\alpha )}}}{6{\sqrt {7}}}},}

skewness=(3−α)72α(6−α){\displaystyle {}={\frac {(3-\alpha ){\sqrt {7}}}{2{\sqrt {\alpha (6-\alpha )}}}}}, andexcess kurtosis=21α(6−α)−3{\displaystyle {}={\frac {21}{\alpha (6-\alpha )}}-3}

The above estimate for thestandard deviationσ(X) = (c −a)/6 is exact for either of the following values ofα andβ:

α =β = 4 (symmetric) withskewness = 0, andexcess kurtosis = −6/11.
β = 6 −α andα=3−2{\displaystyle \alpha =3-{\sqrt {2}}} (right-tailed, positive skew) withskewness=12{\displaystyle {}={\frac {1}{\sqrt {2}}}}, andexcess kurtosis = 0
β = 6 −α andα=3+2{\displaystyle \alpha =3+{\sqrt {2}}} (left-tailed, negative skew) withskewness=−12{\displaystyle {}={\frac {-1}{\sqrt {2}}}}, andexcess kurtosis = 0

Otherwise, these can be poor approximations for beta distributions with other values of α and β, exhibiting average errors of 40% in the mean and 549% in the variance.[74][75][76]

Random variate generation

[edit]
Further information:Non-uniform random variate generation

IfX andY are independent, withX∼Γ(α,θ){\displaystyle X\sim \Gamma (\alpha ,\theta )} andY∼Γ(β,θ){\displaystyle Y\sim \Gamma (\beta ,\theta )} then

XX+Y∼B(α,β).{\displaystyle {\frac {X}{X+Y}}\sim \mathrm {B} (\alpha ,\beta ).}

So one algorithm for generating beta variates is to generateXX+Y{\displaystyle {\frac {X}{X+Y}}}, whereX is agamma variate with parameters (α, 1) andY is an independent gamma variate with parameters (β, 1).[77] In fact, hereXX+Y{\displaystyle {\frac {X}{X+Y}}} andX+Y{\displaystyle X+Y} are independent, andX+Y∼Γ(α+β,θ){\displaystyle X+Y\sim \Gamma (\alpha +\beta ,\theta )}. IfZ∼Γ(γ,θ){\displaystyle Z\sim \Gamma (\gamma ,\theta )} andZ{\displaystyle Z} is independent ofX{\displaystyle X} andY{\displaystyle Y}, thenX+YX+Y+Z∼B(α+β,γ){\displaystyle {\frac {X+Y}{X+Y+Z}}\sim \mathrm {B} (\alpha +\beta ,\gamma )} andX+YX+Y+Z{\displaystyle {\frac {X+Y}{X+Y+Z}}} is independent ofXX+Y{\displaystyle {\frac {X}{X+Y}}}. This shows that the product of independentB(α,β){\displaystyle \mathrm {B} (\alpha ,\beta )} andB(α+β,γ){\displaystyle \mathrm {B} (\alpha +\beta ,\gamma )} random variables is aB(α,β+γ){\displaystyle \mathrm {B} (\alpha ,\beta +\gamma )} random variable.

Also, thekthorder statistic ofnuniformly distributed variates isB(k,n+1−k){\displaystyle \mathrm {B} (k,n+1-k)}, so an alternative ifα andβ are small integers is to generate α + β − 1 uniform variates and choose the α-th smallest.[40]

Another way to generate the Beta distribution is byPólya urn model. According to this method, one starts with an "urn" with α "black" balls and β "white" balls and draws uniformly with replacement. Every trial an additional ball is added according to the color of the last ball which was drawn. Asymptotically, the proportion of black and white balls will be distributed according to the Beta distribution, where each repetition of the experiment will produce a different value.

It is also possible to use theinverse transform sampling.

Normal approximation to the Beta distribution

[edit]

A beta distributionB(α,β){\displaystyle \mathrm {B} (\alpha ,\beta )} withα ~β andα andβ >> 1 is approximately normal with mean 1/2 and variance 1/(4(2α + 1)). Ifα ≥β the normal approximation can be improved by taking the cube-root of the logarithm of the reciprocal ofB(α,β){\displaystyle \mathrm {B} (\alpha ,\beta )}[78][79]

History

[edit]

Thomas Bayes, in a posthumous paper[62] published in 1763 byRichard Price, obtained a beta distribution as the density of the probability of success in Bernoulli trials (see§ Applications, Bayesian inference), but the paper does not analyze any of the moments of the beta distribution or discuss any of its properties.

Karl Pearson analyzed the beta distribution as the solution Type I of Pearson distributions

The first systematic modern discussion of the beta distribution is probably due toKarl Pearson.[80][81] In Pearson's papers[21][33] the beta distribution is couched as a solution of a differential equation:Pearson's Type I distribution which it is essentially identical to except for arbitrary shifting and re-scaling (the beta and Pearson Type I distributions can always be equalized by proper choice of parameters). In fact, in several English books and journal articles in the few decades prior to World War II, it was common to refer to the beta distribution as Pearson's Type I distribution.William P. Elderton in his 1906 monograph "Frequency curves and correlation"[42] further analyzes the beta distribution as Pearson's Type I distribution, including a full discussion of the method of moments for the four parameter case, and diagrams of (what Elderton describes as) U-shaped, J-shaped, twisted J-shaped, "cocked-hat" shapes, horizontal and angled straight-line cases. Elderton wrote "I am chiefly indebted to Professor Pearson, but the indebtedness is of a kind for which it is impossible to offer formal thanks."Elderton in his 1906 monograph[42] provides an impressive amount of information on the beta distribution, including equations for the origin of the distribution chosen to be the mode, as well as for other Pearson distributions: types I through VII. Elderton also included a number of appendixes, including one appendix ("II") on the beta and gamma functions. In later editions, Elderton added equations for the origin of the distribution chosen to be the mean, and analysis of Pearson distributions VIII through XII.

As remarked by Bowman and Shenton[44] "Fisher and Pearson had a difference of opinion in the approach to (parameter) estimation, in particular relating to (Pearson's method of) moments and (Fisher's method of) maximum likelihood in the case of the Beta distribution." Also according to Bowman and Shenton, "the case of a Type I (beta distribution) model being the center of the controversy was pure serendipity. A more difficult model of 4 parameters would have been hard to find." The long running public conflict of Fisher with Karl Pearson can be followed in a number of articles in prestigious journals. For example, concerning the estimation of the four parameters for the beta distribution, and Fisher's criticism of Pearson's method of moments as being arbitrary, see Pearson's article "Method of moments and method of maximum likelihood"[45] (published three years after his retirement from University College, London, where his position had been divided between Fisher and Pearson's son Egon) in which Pearson writes "I read (Koshai's paper in the Journal of the Royal Statistical Society, 1933) which as far as I am aware is the only case at present published of the application of Professor Fisher's method. To my astonishment that method depends on first working out the constants of the frequency curve by the (Pearson) Method of Moments and then superposing on it, by what Fisher terms "the Method of Maximum Likelihood" a further approximation to obtain, what he holds, he will thus get, 'more efficient values' of the curve constants".

David and Edwards's treatise on the history of statistics[82] cites the first modern treatment of the beta distribution, in 1911,[83] using the beta designation that has become standard, due toCorrado Gini, an Italianstatistician,demographer, andsociologist, who developed theGini coefficient.N.L.Johnson andS.Kotz, in their comprehensive and very informative monograph[84] on leading historical personalities in statistical sciences creditCorrado Gini[85] as "an early Bayesian...who dealt with the problem of eliciting the parameters of an initial Beta distribution, by singling out techniques which anticipated the advent of the so-called empirical Bayes approach."

References

[edit]
  1. ^abcdefghijklmnopqrstuvwxyJohnson, Norman L.; Kotz, Samuel; Balakrishnan, N. (1995). "Chapter 25: Beta Distributions".Continuous Univariate Distributions Vol. 2 (2nd ed.). Wiley.ISBN 978-0-471-58494-0.
  2. ^abRose, Colin; Smith, Murray D. (2002).Mathematical Statistics with MATHEMATICA. Springer.ISBN 978-0387952345.
  3. ^abcKruschke, John K. (2011).Doing Bayesian data analysis: A tutorial with R and BUGS. Academic Press / Elsevier. p. 83.ISBN 978-0123814852.
  4. ^abBerger, James O. (2010).Statistical Decision Theory and Bayesian Analysis (2nd ed.). Springer.ISBN 978-1441930743.
  5. ^abcFeller, William (1971).An Introduction to Probability Theory and Its Applications, Vol. 2. Wiley.ISBN 978-0471257097.
  6. ^Wadsworth, G. P. (1960).Introduction to Probability and Random Variables. New York: McGraw-Hill. p. 52.
  7. ^Kruschke, John K. (2015).Doing Bayesian Data Analysis: A Tutorial with R, JAGS and Stan. Academic Press / Elsevier.ISBN 978-0-12-405888-0.
  8. ^abWadsworth, George P. and Joseph Bryan (1960).Introduction to Probability and Random Variables. McGraw-Hill.
  9. ^abcdefgGupta, Arjun K., ed. (2004).Handbook of Beta Distribution and Its Applications. CRC Press.ISBN 978-0824753962.
  10. ^abKerman, Jouni (2011). "A closed-form approximation for the median of the beta distribution".arXiv:1111.0433 [math.ST].
  11. ^Mosteller, Frederick and John Tukey (1977).Data Analysis and Regression: A Second Course in Statistics. Addison-Wesley Pub. Co.Bibcode:1977dars.book.....M.ISBN 978-0201048544.
  12. ^Feller, William (1968).An Introduction to Probability Theory and Its Applications. Vol. 1 (3rd ed.). Wiley.ISBN 978-0471257080.
  13. ^Philip J. Fleming and John J. Wallace.How not to lie with statistics: the correct way to summarize benchmark results. Communications of the ACM, 29(3):218–221, March 1986.
  14. ^"NIST/SEMATECH e-Handbook of Statistical Methods 1.3.6.6.17. Beta Distribution".National Institute of Standards and Technology Information Technology Laboratory. April 2012. RetrievedMay 31, 2016.
  15. ^Oguamanam, D.C.D.; Martin, H. R.; Huissoon, J. P. (1995). "On the application of the beta distribution to gear damage analysis".Applied Acoustics.45 (3):247–261.doi:10.1016/0003-682X(95)00001-P.
  16. ^Zhiqiang Liang; Jianming Wei; Junyu Zhao; Haitao Liu; Baoqing Li; Jie Shen; Chunlei Zheng (27 August 2008)."The Statistical Meaning of Kurtosis and Its New Application to Identification of Persons Based on Seismic Signals".Sensors.8 (8):5106–5119.Bibcode:2008Senso...8.5106L.doi:10.3390/s8085106.PMC 3705491.PMID 27873804.
  17. ^Kenney, J. F., and E. S. Keeping (1951).Mathematics of Statistics Part Two, 2nd edition. D. Van Nostrand Company Inc.{{cite book}}: CS1 maint: multiple names: authors list (link)
  18. ^abcdAbramowitz, Milton and Irene A. Stegun (1965).Handbook Of Mathematical Functions With Formulas, Graphs, And Mathematical Tables. Dover.ISBN 978-0-486-61272-0.
  19. ^Weisstein., Eric W."Kurtosis". MathWorld--A Wolfram Web Resource. Retrieved13 August 2012.
  20. ^abPanik, Michael J (2005).Advanced Statistics from an Elementary Point of View. Academic Press.ISBN 978-0120884940.
  21. ^abcdefPearson, Karl (1916)."Mathematical contributions to the theory of evolution, XIX: Second supplement to a memoir on skew variation".Philosophical Transactions of the Royal Society A.216 (538–548):429–457.Bibcode:1916RSPTA.216..429P.doi:10.1098/rsta.1916.0009.JSTOR 91092.
  22. ^Gradshteyn, Izrail Solomonovich;Ryzhik, Iosif Moiseevich;Geronimus, Yuri Veniaminovich;Tseytlin, Michail Yulyevich; Jeffrey, Alan (2015) [October 2014]. Zwillinger, Daniel;Moll, Victor Hugo (eds.).Table of Integrals, Series, and Products. Translated by Scripta Technica, Inc. (8 ed.).Academic Press, Inc.ISBN 978-0-12-384933-5.LCCN 2014010276.
  23. ^Billingsley, Patrick (1995). "Section 30: The Method of Moments".Probability and measure (3rd ed.). Wiley-Interscience.ISBN 978-0-471-00710-4.
  24. ^abMacKay, David (2003).Information Theory, Inference and Learning Algorithms. Cambridge University Press; First Edition.Bibcode:2003itil.book.....M.ISBN 978-0521642989.
  25. ^abJohnson, N.L. (1949)."Systems of frequency curves generated by methods of translation"(PDF).Biometrika.36 (1–2):149–176.doi:10.1093/biomet/36.1-2.149.hdl:10338.dmlcz/135506.PMID 18132090.
  26. ^Verdugo Lazo, A. C. G.; Rathie, P. N. (1978). "On the entropy of continuous probability distributions".IEEE Trans. Inf. Theory.24 (1):120–122.doi:10.1109/TIT.1978.1055832.
  27. ^Shannon, Claude E. (1948). "A Mathematical Theory of Communication".Bell System Technical Journal.27 (4):623–656.doi:10.1002/j.1538-7305.1948.tb01338.x.
  28. ^abcCover, Thomas M. and Joy A. Thomas (2006).Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience; 2 edition.ISBN 978-0471241959.
  29. ^Plunkett, Kim, and Jeffrey Elman (1997).Exercises in Rethinking Innateness: A Handbook for Connectionist Simulations (Neural Network Modeling and Connectionism). A Bradford Book. p. 166.ISBN 978-0262661058.{{cite book}}: CS1 maint: multiple names: authors list (link)
  30. ^Nallapati, Ramesh (2006).The smoothed dirichlet distribution: understanding cross-entropy ranking in information retrieval (Thesis). Computer Science Dept., University of Massachusetts Amherst.
  31. ^abPearson, Egon S. (July 1969)."Some historical reflections traced through the development of the use of frequency curves".THEMIS Statistical Analysis Research Program, Technical Report 38. Office of Naval Research, Contract N000014-68-A-0515 (Project NR 042–260).
  32. ^Hahn, Gerald J.; Shapiro, S. (1994).Statistical Models in Engineering (Wiley Classics Library). Wiley-Interscience.ISBN 978-0471040651.
  33. ^abPearson, Karl (1895)."Contributions to the mathematical theory of evolution, II: Skew variation in homogeneous material".Philosophical Transactions of the Royal Society.186:343–414.Bibcode:1895RSPTA.186..343P.doi:10.1098/rsta.1895.0010.JSTOR 90649.
  34. ^Buchanan, K.; Rockway, J.; Sternberg, O.; Mai, N. N. (May 2016)."Sum-difference beamforming for radar applications using circularly tapered random arrays".2016 IEEE Radar Conference (RadarConf). pp. 1–5.doi:10.1109/RADAR.2016.7485289.ISBN 978-1-5090-0863-6.S2CID 32525626.
  35. ^Buchanan, K.; Flores, C.; Wheeland, S.; Jensen, J.; Grayson, D.; Huff, G. (May 2017). "Transmit beamforming for radar applications using circularly tapered random arrays".2017 IEEE Radar Conference (RadarConf). pp. 0112–0117.doi:10.1109/RADAR.2017.7944181.ISBN 978-1-4673-8823-8.S2CID 38429370.
  36. ^Ryan, Buchanan, Kristopher (2014-05-29)."Theory and Applications of Aperiodic (Random) Phased Arrays".{{cite web}}: CS1 maint: multiple names: authors list (link)
  37. ^Pham-Gia, T. (January 2000)."Distributions of the ratios of independent beta variables and applications".Communications in Statistics - Theory and Methods.29 (12):2693–2715.doi:10.1080/03610920008832632.ISSN 0361-0926. Retrieved13 November 2024.
  38. ^Herrerías-Velasco, José Manuel and Herrerías-Pleguezuelo, Rafael and René van Dorp, Johan. (2011). Revisiting the PERT mean and Variance. European Journal of Operational Research (210), p. 448–451.
  39. ^abMalcolm, D. G.; Roseboom, J. H.; Clark, C. E.; Fazar, W. (September–October 1958). "Application of a Technique for Research and Development Program Evaluation".Operations Research.7 (5):646–669.doi:10.1287/opre.7.5.646.ISSN 0030-364X.
  40. ^abcdDavid, H. A., Nagaraja, H. N. (2003)Order Statistics (3rd Edition). Wiley, New Jersey pp 458.ISBN 0-471-38926-9
  41. ^"1.3.6.6.17. Beta Distribution".www.itl.nist.gov.
  42. ^abcdefghElderton, William Palin (1906).Frequency-Curves and Correlation. Charles and Edwin Layton (London).
  43. ^Elderton, William Palin and Norman Lloyd Johnson (2009).Systems of Frequency Curves. Cambridge University Press.ISBN 978-0521093361.
  44. ^abcBowman, K. O.; Shenton, L. R. (2007)."The beta distribution, moment method, Karl Pearson and R.A. Fisher"(PDF).Far East J. Theo. Stat.23 (2):133–164.
  45. ^abPearson, Karl (June 1936). "Method of moments and method of maximum likelihood".Biometrika.28 (1/2):34–59.doi:10.2307/2334123.JSTOR 2334123.
  46. ^abcJoanes, D. N.; C. A. Gill (1998). "Comparing measures of sample skewness and kurtosis".The Statistician.47 (Part 1):183–189.doi:10.1111/1467-9884.00122.
  47. ^Beckman, R. J.; G. L. Tietjen (1978). "Maximum likelihood estimation for the beta distribution".Journal of Statistical Computation and Simulation.7 (3–4):253–258.doi:10.1080/00949657808810232.
  48. ^Gnanadesikan, R., Pinkham and Hughes (1967). "Maximum likelihood estimation of the parameters of the beta distribution from smallest order statistics".Technometrics.9 (4):607–620.doi:10.2307/1266199.JSTOR 1266199.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  49. ^Fackler, Paul."Inverse Digamma Function (Matlab)". Harvard University School of Engineering and Applied Sciences. Retrieved2012-08-18.
  50. ^abcSilvey, S.D. (1975).Statistical Inference. Chapman and Hal. p. 40.ISBN 978-0412138201.
  51. ^Edwards, A. W. F. (1992).Likelihood. The Johns Hopkins University Press.ISBN 978-0801844430.
  52. ^abcdefJaynes, E.T. (2003).Probability theory, the logic of science. Cambridge University Press.ISBN 978-0521592710.
  53. ^Costa, Max, and Cover, Thomas (September 1983).On the similarity of the entropy power inequality and the Brunn Minkowski inequality(PDF). Tech.Report 48, Dept. Statistics, Stanford University.{{cite book}}: CS1 maint: multiple names: authors list (link)
  54. ^abcAryal, Gokarna; Saralees Nadarajah (2004)."Information matrix for beta distributions"(PDF).Serdica Mathematical Journal (Bulgarian Academy of Science).30:513–526.
  55. ^abLaplace, Pierre Simon, marquis de (1902).A philosophical essay on probabilities. New York : J. Wiley; London : Chapman & Hall.ISBN 978-1-60206-328-0.{{cite book}}:ISBN / Date incompatibility (help)CS1 maint: multiple names: authors list (link)
  56. ^Cox, Richard T. (1961).Algebra of Probable Inference. The Johns Hopkins University Press.ISBN 978-0801869822.{{cite book}}:ISBN / Date incompatibility (help)
  57. ^abKeynes, John Maynard (2010) [1921].A Treatise on Probability: The Connection Between Philosophy and the History of Science. Wildside Press.ISBN 978-1434406965.
  58. ^Pearson, Karl (1907). "On the Influence of Past Experience on Future Expectation".Philosophical Magazine.6 (13):365–378.
  59. ^abcdJeffreys, Harold (1998).Theory of Probability. Oxford University Press, 3rd edition.ISBN 978-0198503682.
  60. ^Broad, C. D. (October 1918). "On the relation between induction and probability".MIND, A Quarterly Review of Psychology and Philosophy. 27 (New Series) (108):389–404.doi:10.1093/mind/XXVII.4.389.JSTOR 2249035.
  61. ^abcdPerks, Wilfred (January 1947)."Some observations on inverse probability including a new indifference rule".Journal of the Institute of Actuaries.73 (2):285–334.doi:10.1017/S0020268100012270. Archived fromthe original on 2014-01-12. Retrieved2012-09-19.
  62. ^abBayes, Thomas; communicated by Richard Price (1763)."An Essay towards solving a Problem in the Doctrine of Chances".Philosophical Transactions of the Royal Society.53:370–418.doi:10.1098/rstl.1763.0053.JSTOR 105741.
  63. ^Haldane, J. B. S. (1932). "A note on inverse probability".Mathematical Proceedings of the Cambridge Philosophical Society.28 (1):55–61.Bibcode:1932PCPS...28...55H.doi:10.1017/s0305004100010495.S2CID 122773707.
  64. ^Zellner, Arnold (1971).An Introduction to Bayesian Inference in Econometrics. Wiley-Interscience.ISBN 978-0471169376.
  65. ^Jeffreys, Harold (September 1946)."An Invariant Form for the Prior Probability in Estimation Problems".Proceedings of the Royal Society. A 24.186 (1007):453–461.Bibcode:1946RSPSA.186..453J.doi:10.1098/rspa.1946.0056.PMID 20998741.
  66. ^Berger, James; Bernardo, Jose; Sun, Dongchu (2009)."The formal definition of reference priors".The Annals of Statistics.37 (2):905–938.arXiv:0904.0156.Bibcode:2009arXiv0904.0156B.doi:10.1214/07-AOS587.S2CID 3221355.
  67. ^Clarke, Bertrand S.; Andrew R. Barron (1994)."Jeffreys' prior is asymptotically least favorable under entropy risk"(PDF).Journal of Statistical Planning and Inference.41:37–60.doi:10.1016/0378-3758(94)90153-8.
  68. ^Pearson, Karl (1892).The Grammar of Science. Walter Scott, London.
  69. ^Pearson, Karl (2009).The Grammar of Science. BiblioLife.ISBN 978-1110356119.
  70. ^Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2003).Bayesian Data Analysis. Chapman and Hall/CRC.ISBN 978-1584883883.{{cite book}}: CS1 maint: multiple names: authors list (link)
  71. ^Jøsang, Audun (2001)."A logic for uncertain probabilities".International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems.9 (3):279–311.doi:10.1142/S0218488501000831.MR 1843261.
  72. ^H.M. de Oliveira and G.A.A. Araújo,. Compactly Supported One-cyclic Wavelets Derived from Beta Distributions.Journal of Communication and Information Systems. vol.20, n.3, pp.27-33, 2005.
  73. ^Balding, David J.; Nichols, Richard A. (1995). "A method for quantifying differentiation between populations at multi-allelic loci and its implications for investigating identity and paternity".Genetica.96 (1–2). Springer:3–12.doi:10.1007/BF01441146.PMID 7607457.S2CID 30680826.
  74. ^Keefer, Donald L. and Verdini, William A. (1993). Better Estimation of PERT Activity Time Parameters. Management Science 39(9), p. 1086–1091.
  75. ^Keefer, Donald L. and Bodily, Samuel E. (1983). Three-point Approximations for Continuous Random variables. Management Science 29(5), p. 595–609.
  76. ^"Defense Resource Management Institute - Naval Postgraduate School".www.nps.edu.
  77. ^van der Waerden, B. L., "Mathematical Statistics", Springer,ISBN 978-3-540-04507-6.
  78. ^On normalizing the incomplete beta-function for fitting to dose-response curves M.E. Wise Biometrika vol 47, No. 1/2, June 1960, pp. 173–175
  79. ^Pratt, John W. “A Normal Approximation for Binomial, F, Beta, and Other Common, Related Tail Probabilities, II.” Journal of the American Statistical Association, vol. 63, no. 324, 1968, pp. 1457–83. JSTOR,https://doi.org/10.2307/2285896. Accessed 21 Oct. 2025.
  80. ^Yule, G. U.; Filon, L. N. G. (1936)."Karl Pearson. 1857–1936".Obituary Notices of Fellows of the Royal Society.2 (5): 72.doi:10.1098/rsbm.1936.0007.JSTOR 769130.
  81. ^"Library and Archive catalogue".Sackler Digital Archive. Royal Society. Archived fromthe original on 2011-10-25. Retrieved2011-07-01.
  82. ^David, H. A. and A.W.F. Edwards (2001).Annotated Readings in the History of Statistics. Springer; 1 edition.ISBN 978-0387988443.
  83. ^Gini, Corrado (1911). "Considerazioni Sulle Probabilità Posteriori e Applicazioni al Rapporto dei Sessi Nelle Nascite Umane".Studi Economico-Giuridici della Università de Cagliari. Anno III (reproduced in Metron 15, 133, 171, 1949):5–41.
  84. ^Johnson, Norman L. and Samuel Kotz, ed. (1997).Leading Personalities in Statistical Sciences: From the Seventeenth Century to the Present (Wiley Series in Probability and Statistics. Wiley.ISBN 978-0471163817.
  85. ^Metron journal."Biography of Corrado Gini". Metron Journal. Archived fromthe original on 2012-07-16. Retrieved2012-08-18.

External links

[edit]
Wikimedia Commons has media related toBeta distribution.
Discrete
univariate
with finite
support
with infinite
support
Continuous
univariate
supported on a
bounded interval
supported on a
semi-infinite
interval
supported
on the whole
real line
with support
whose type varies
Mixed
univariate
continuous-
discrete
Multivariate
(joint)
Directional
Degenerate
andsingular
Degenerate
Dirac delta function
Singular
Cantor
Families
Retrieved from "https://en.wikipedia.org/w/index.php?title=Beta_distribution&oldid=1338557858"
Categories:
Hidden categories:
[8]ページ先頭
©2009-2026 Movatter.jp