Dirichlet distribution
Probability density function
Parameters	${\displaystyle K\ge 2}$ number of categories (integer) ${\displaystyle \mathit{\alpha }=({\alpha }_1,\dots ,{\alpha }_K)}$concentration parameters, where${\displaystyle {\alpha }_i>0}$
Support	${\displaystyle x_1,\dots ,x_K}$ where${\displaystyle x_i\in [0,1]}$ and${\displaystyle \sum _{i=1}^K{x}_i=1}$ (i.e. a${\displaystyle K-1}$simplex)
PDF	${\displaystyle \frac{1}{\mathrm{B}(\mathit{\alpha })}\prod _{i=1}^K{x}_i^{{\alpha }_i-1}}$ where${\displaystyle \mathrm{B}(\mathit{\alpha })=\frac{\prod _{i=1}^K\mathrm{\Gamma }({\alpha }_i)}{\mathrm{\Gamma }({\alpha }_0)}}$ where${\displaystyle {\alpha }_0=\sum _{i=1}^K{\alpha }_i}$
Mean	${\displaystyle \mathrm{E}[X_i]=\frac{{\alpha }_i}{{\alpha }_0}}$ ${\displaystyle \mathrm{E}[\ln X_i]=\psi ({\alpha }_i)-\psi ({\alpha }_0)}$ (where${\displaystyle \psi }$ is thedigamma function)
Mode	${\displaystyle x_i=\frac{{\alpha }_i-1}{{\alpha }_0-K},{\alpha }_i>1.}$
Variance	${\displaystyle \mathrm{Var}[X_i]=\frac{{\tilde{\alpha }}_i(1-{\tilde{\alpha }}_i)}{{\alpha }_0+1},}$${\displaystyle \mathrm{Cov}[X_i,X_j]=\frac{{\delta }_{ij}{\tilde{\alpha }}_i-{\tilde{\alpha }}_i{\tilde{\alpha }}_j}{{\alpha }_0+1}}$ where${\displaystyle {\tilde{\alpha }}_i=\frac{{\alpha }_i}{{\alpha }_0}}$, and${\displaystyle {\delta }_{ij}}$ is theKronecker delta
Entropy	${\displaystyle H(X)=\log \mathrm{B}(\mathit{\alpha })}$${\displaystyle +({\alpha }_0-K)\psi ({\alpha }_0)-}$${\displaystyle \sum _{j=1}^K({\alpha }_j-1)\psi ({\alpha }_j)}$ with${\displaystyle {\alpha }_0}$ defined as for variance, above; and${\displaystyle \psi }$ is thedigamma function
Method of moments	${\displaystyle {\alpha }_i=E[X_i]\left(\frac{E[X_j](1-E[X_j])}{V[X_j]}-1\right)}$ wherej is any index, possiblyi itself

Inprobability andstatistics, theDirichlet distribution (afterPeter Gustav Lejeune Dirichlet), often denoted${\displaystyle \mathrm{Dir}(\mathit{\alpha })}$, is a family ofcontinuousmultivariateprobability distributions parameterized by a vectorα of positivereals. It is a multivariate generalization of thebeta distribution,^[1] hence its alternative name ofmultivariate beta distribution (MBD).^[2] Dirichlet distributions are commonly used asprior distributions inBayesian statistics, and in fact, the Dirichlet distribution is theconjugate prior of thecategorical distribution andmultinomial distribution.

The infinite-dimensional generalization of the Dirichlet distribution is theDirichlet process.

The Dirichlet distribution of order${\displaystyle K\ge 2}$ with parameters${\displaystyle {\alpha }_1,\dots ,{\alpha }_K>0}$ has aprobability density function given by

$${\displaystyle f\left(x_1,\dots ,x_K;{\alpha }_1,\dots ,{\alpha }_K\right)=\frac{1}{\mathrm{B}(\mathit{\alpha })}\prod _{i=1}^K{x}_i^{{\alpha }_i-1}}$$where$${\displaystyle x_i\in \left[0,1\right]\text{ for all }i\in \{1,\dots ,K\}\text{ and }\sum _{i=1}^K{x}_i=1.}$$That is, the probability density function is defined on the standard${\displaystyle K-1}$simplex embedded in⁠${\displaystyle K}$⁠-dimensionalEuclidean space,${\displaystyle {\mathbb{R}}^K}$.

Thenormalizing constant is the multivariatebeta function, which can be expressed in terms of thegamma function:

$${\displaystyle \mathrm{B}(\mathit{\alpha })=\frac{\prod _{i=1}^K\mathrm{\Gamma }({\alpha }_i)}{\mathrm{\Gamma }\left(\sum _{i=1}^K{\alpha }_i\right)},\mathit{\alpha }=({\alpha }_1,\dots ,{\alpha }_K).}$$

Thesupport of the Dirichlet distribution is the set ofK-dimensional vectorsx whose entries are real numbers in the interval [0,1] such that${\displaystyle \Vert \mathit{x}{\Vert }_1=1}$, i.e. the sum of the coordinates is equal to 1. These can be viewed as the probabilities of aK-waycategorical event. Another way to express this is that the domain of the Dirichlet distribution is itself a set ofprobability distributions, specifically the set ofK-dimensionaldiscrete distributions. The technical term for the set of points in the support of aK-dimensional Dirichlet distribution is theopenstandard(K − 1)-simplex,^[3] which is a generalization of atriangle, embedded in the next-higher dimension. For example, withK = 3, the support is anequilateral triangle embedded in a downward-angle fashion in three-dimensional space, with vertices at (1,0,0), (0,1,0) and (0,0,1), i.e. touching each of the coordinate axes at a point 1 unit away from the origin.

A common special case is thesymmetric Dirichlet distribution, where all of the elements making up the parameter vectorα have the same value. The symmetric case might be useful, for example, when a Dirichlet prior over components is called for, but there is no prior knowledge favoring one component over another. Since all elements of the parameter vector have the same value, the symmetric Dirichlet distribution can be parametrized by a single scalar valueα, called theconcentration parameter. In terms ofα, the density function has the form

$${\displaystyle f(x_1,\dots ,x_K;\alpha )=\frac{\mathrm{\Gamma }(\alpha K)}{\mathrm{\Gamma }(\alpha {)}^K}\prod _{i=1}^K{x}_i^{\alpha -1}.}$$

Whenα = 1,^[1] the symmetric Dirichlet distribution is equivalent to a uniform distribution over the openstandard(K−1)-simplex, i.e. it is uniform over all points in itssupport. This particular distribution is known as theflat Dirichlet distribution. Values of the concentration parameter above 1 prefervariates that are dense, evenly distributed distributions, i.e. all the values within a single sample are similar to each other. Values of the concentration parameter below 1 prefer sparse distributions, i.e. most of the values within a single sample will be close to 0, and the vast majority of the mass will be concentrated in a few of the values.

Whenα = 1/2, the distribution is the same as would be obtained by choosing a point uniformly at random from the surface of a(K−1)-dimensionalunit hypersphere and squaring each coordinate. Theα = 1/2 distribution is theJeffreys prior for the Dirichlet distribution.

More generally, the parameter vector is sometimes written as the product${\displaystyle \alpha \mathit{n}}$ of a (scalar)concentration parameterα and a (vector)base measure${\displaystyle \mathit{n}=(n_1,\dots ,n_K)}$ wheren lies within the(K − 1)-simplex (i.e.: its coordinates${\displaystyle n_i}$ sum to one). The concentration parameter in this case is larger by a factor ofK than the concentration parameter for a symmetric Dirichlet distribution described above. This construction ties in with concept of a base measure when discussingDirichlet processes and is often used in the topic modelling literature.

Let${\displaystyle X=(X_1,\dots ,X_K)\sim \mathrm{Dir}(\mathit{\alpha })}$.

Let

$${\displaystyle {\alpha }_0=\sum _{i=1}^K{\alpha }_i.}$$

Then^[4][5]

$${\displaystyle \mathrm{E}[X_i]=\frac{{\alpha }_i}{{\alpha }_0},}$$$${\displaystyle \mathrm{Var}[X_i]=\frac{{\alpha }_i({\alpha }_0-{\alpha }_i)}{{\alpha }_0^2({\alpha }_0+1)}.}$$

Furthermore, if${\displaystyle i\ne j}$

$${\displaystyle \mathrm{Cov}[X_i,X_j]=\frac{-{\alpha }_i{\alpha }_j}{{\alpha }_0^2({\alpha }_0+1)}.}$$

The covariance matrix issingular.

More generally, moments of Dirichlet-distributed random variables can be expressed in the following way. For${\displaystyle \mathit{t}=(t_1,\dots ,t_K)\in {\mathbb{R}}^K}$, denote by${\displaystyle {\mathit{t}}^{\circ i}=(t_1^i,\dots ,t_K^i)}$ itsi-thHadamard power. Then,^[6]

${\displaystyle \mathrm{E}\left[(\mathit{t}\cdot \mathit{X}{)}^n\right]=\frac{n!\mathrm{\Gamma }({\alpha }_0)}{\mathrm{\Gamma }({\alpha }_0+n)}\sum \frac{{t_1}^{k_1}\cdots {t_K}^{k_K}}{k_1!\cdots k_K!}\prod _{i=1}^K\frac{\mathrm{\Gamma }({\alpha }_i+k_i)}{\mathrm{\Gamma }({\alpha }_i)}=\frac{n!\mathrm{\Gamma }({\alpha }_0)}{\mathrm{\Gamma }({\alpha }_0+n)}{Z}_n({\mathit{t}}^{\circ 1}\cdot \mathit{\alpha },\cdots ,{\mathit{t}}^{\circ n}\cdot \mathit{\alpha }),}$

where the sum is over non-negative integers${\displaystyle k_1,\dots ,k_K}$ with${\displaystyle n=k_1+\cdots +k_K}$, and${\displaystyle Z_n}$ is thecycle index polynomial of theSymmetric group of degreen.

We have the special case${\displaystyle \mathrm{E}\left[\mathit{t}\cdot \mathit{X}\right]=\frac{\mathit{t}\cdot \mathit{\alpha }}{{\alpha }_0}.}$

The multivariate analogue$\mathrm{E}\left[({\mathit{t}}_1\cdot \mathit{X}{)}^{n_1}\cdots ({\mathit{t}}_q\cdot \mathit{X}{)}^{n_q}\right]$ for vectors${\displaystyle {\mathit{t}}_1,\dots ,{\mathit{t}}_q\in {\mathbb{R}}^K}$ can be expressed^[7] in terms of a color pattern of the exponents${\displaystyle n_1,\dots ,n_q}$ in the sense of thePólya enumeration theorem.

Particular cases include the simple computation^[8]

$${\displaystyle \mathrm{E}\left[\prod _{i=1}^K{X}_i^{{\beta }_i}\right]=\frac{B\left(\mathit{\alpha }+\mathit{\beta }\right)}{B\left(\mathit{\alpha }\right)}=\frac{\mathrm{\Gamma }\left(\sum _{i=1}^K{\alpha }_i\right)}{\mathrm{\Gamma }\left[\sum _{i=1}^K({\alpha }_i+{\beta }_i)\right]}\times \prod _{i=1}^K\frac{\mathrm{\Gamma }({\alpha }_i+{\beta }_i)}{\mathrm{\Gamma }({\alpha }_i)}.}$$

Themode of the distribution is^[9] the vector(x₁, ...,x_K) with

$${\displaystyle x_i=\frac{{\alpha }_i-1}{{\alpha }_0-K},{\alpha }_i>1.}$$

Themarginal distributions arebeta distributions:^[10]

$${\displaystyle X_i\sim \mathrm{Beta}({\alpha }_i,{\alpha }_0-{\alpha }_i).}$$

Also see§ Related distributions below.

The Dirichlet distribution is theconjugate prior distribution of thecategorical distribution (a genericdiscrete probability distribution with a given number of possible outcomes) andmultinomial distribution (the distribution over observed counts of each possible category in a set of categorically distributed observations). This means that if a data point has either a categorical or multinomial distribution, and theprior distribution of the distribution's parameter (the vector of probabilities that generates the data point) is distributed as a Dirichlet, then theposterior distribution of the parameter is also a Dirichlet. Intuitively, in such a case, starting from what we know about the parameter prior to observing the data point, we then can update our knowledge based on the data point and end up with a new distribution of the same form as the old one. This means that we can successively update our knowledge of a parameter by incorporating new observations one at a time, without running into mathematical difficulties.

Formally, this can be expressed as follows. Given a model

then the following holds:

This relationship is used inBayesian statistics to estimate the underlying parameterp of acategorical distribution given a collection ofN samples. Intuitively, we can view thehyperprior vectorα aspseudocounts, i.e. as representing the number of observations in each category that we have already seen. Then we simply add in the counts for all the new observations (the vectorc) in order to derive the posterior distribution.

In Bayesianmixture models and otherhierarchical Bayesian models with mixture components, Dirichlet distributions are commonly used as the prior distributions for thecategorical variables appearing in the models. See the section onapplications below for more information.

In a model where a Dirichlet prior distribution is placed over a set ofcategorical-valued observations, themarginaljoint distribution of the observations (i.e. the joint distribution of the observations, with the prior parametermarginalized out) is aDirichlet-multinomial distribution. This distribution plays an important role inhierarchical Bayesian models, because when doinginference over such models using methods such asGibbs sampling orvariational Bayes, Dirichlet prior distributions are often marginalized out. See thearticle on this distribution for more details.

IfX is a${\displaystyle \mathrm{Dir}(\mathit{\alpha })}$ random variable, thedifferential entropy ofX (innat units) is^[11]

$${\displaystyle h(\mathit{X})=\mathrm{E}[-\ln f(\mathit{X})]=\ln \mathrm{B}(\mathit{\alpha })+({\alpha }_0-K)\psi ({\alpha }_0)-\sum _{j=1}^K({\alpha }_j-1)\psi ({\alpha }_j)}$$

where${\displaystyle \psi }$ is thedigamma function.

The following formula for${\displaystyle \mathrm{E}[\ln (X_i)]}$ can be used to derive the differentialentropy above. Since the functions${\displaystyle \ln (X_i)}$ are the sufficient statistics of the Dirichlet distribution, theexponential family differential identities can be used to get an analytic expression for the expectation of${\displaystyle \ln (X_i)}$ (see equation (2.62) in^[12]) and its associated covariance matrix:

$${\displaystyle \mathrm{E}[\ln (X_i)]=\psi ({\alpha }_i)-\psi ({\alpha }_0)}$$

and

$${\displaystyle \mathrm{Cov}[\ln (X_i),\ln (X_j)]={\psi }^{\prime }({\alpha }_i){\delta }_{ij}-{\psi }^{\prime }({\alpha }_0)}$$

where${\displaystyle \psi }$ is thedigamma function,${\displaystyle {\psi }^{\prime }}$ is thetrigamma function, and${\displaystyle {\delta }_{ij}}$ is theKronecker delta.

The spectrum ofRényi information for values other than${\displaystyle \lambda =1}$ is given by^[13]

$${\displaystyle F_R(\lambda )=(1-\lambda {)}^{-1}\left(-\lambda \log \mathrm{B}(\mathit{\alpha })+\sum _{i=1}^K\log \mathrm{\Gamma }(\lambda ({\alpha }_i-1)+1)-\log \mathrm{\Gamma }(\lambda ({\alpha }_0-K)+K)\right)}$$

and the information entropy is the limit as${\displaystyle \lambda }$ goes to 1.

Another related interesting measure is the entropy of a discrete categorical (one-of-K binary) vectorZ with probability-mass distributionX, i.e.,${\displaystyle P(Z_i=1,Z_{j\ne i}=0|\mathit{X})=X_i}$. The conditionalinformation entropy ofZ, givenX is

$${\displaystyle S(\mathit{X})=H(\mathit{Z}|\mathit{X})={\mathrm{E}}_{\mathit{Z}}[-\log P(\mathit{Z}|\mathit{X})]=\sum _{i=1}^K-X_i\log X_i}$$

This function ofX is a scalar random variable. IfX has a symmetric Dirichlet distribution with all${\displaystyle {\alpha }_i=\alpha }$, the expected value of the entropy (innat units) is^[14]

$${\displaystyle \mathrm{E}[S(\mathit{X})]=\sum _{i=1}^K\mathrm{E}[-X_i\ln X_i]=\psi (K\alpha +1)-\psi (\alpha +1)}$$

TheKullback–Leibler (KL) divergence between two Dirichlet distributions,${\displaystyle \text{Dir}(\mathit{\alpha })}$ and${\displaystyle \text{Dir}(\mathit{\beta })}$, over the same simplex is:^[15]

If

$${\displaystyle X=(X_1,\dots ,X_K)\sim \mathrm{Dir}({\alpha }_1,\dots ,{\alpha }_K)}$$

then, if the random variables with subscriptsi andj are dropped from the vector and replaced by their sum,

$${\displaystyle X^{\prime }=(X_1,\dots ,X_i+X_j,\dots ,X_K)\sim \mathrm{Dir}({\alpha }_1,\dots ,{\alpha }_i+{\alpha }_j,\dots ,{\alpha }_K).}$$

This aggregation property may be used to derive the marginal distribution of${\displaystyle X_i}$ mentioned above.

If${\displaystyle X=(X_1,\dots ,X_K)\sim \mathrm{Dir}(\mathit{\alpha })}$, then the vector X is said to beneutral^[16] in the sense thatX_K is independent of${\displaystyle X^{(-K)}}$^[3] where

$${\displaystyle X^{(-K)}=\left(\frac{X_1}{1-X_K},\frac{X_2}{1-X_K},\dots ,\frac{X_{K-1}}{1-X_K}\right),}$$

and similarly for removing any of${\displaystyle X_2,\dots ,X_{K-1}}$. Observe that any permutation ofX is also neutral (a property not possessed by samples drawn from ageneralized Dirichlet distribution).^[17]

Combining this with the property of aggregation it follows thatX_j + ... +X_K is independent of${\displaystyle \left(\frac{X_1}{X_1+\cdots +X_{j-1}},\frac{X_2}{X_1+\cdots +X_{j-1}},\dots ,\frac{X_{j-1}}{X_1+\cdots +X_{j-1}}\right)}$. In fact it is true, further, for the Dirichlet distribution, that for${\displaystyle 3\le j\le K-1}$, the pair${\displaystyle \left(X_1+\cdots +X_{j-1},X_j+\cdots +X_K\right)}$, and the two vectors${\displaystyle \left(\frac{X_1}{X_1+\cdots +X_{j-1}},\frac{X_2}{X_1+\cdots +X_{j-1}},\dots ,\frac{X_{j-1}}{X_1+\cdots +X_{j-1}}\right)}$ and${\displaystyle \left(\frac{X_j}{X_j+\cdots +X_K},\frac{X_{j+1}}{X_j+\cdots +X_K},\dots ,\frac{X_K}{X_j+\cdots +X_K}\right)}$, viewed as triple of normalised random vectors, aremutually independent. The analogous result is true for partition of the indices{1, 2, ...,K} into any other pair of non-singleton subsets.

The characteristic function of the Dirichlet distribution is aconfluent form of theLauricella hypergeometric series. It is given byPhillips as^[18]

$${\displaystyle CF\left(s_1,\dots ,s_{K-1}\right)=\mathrm{E}\left(e^{i\left(s_1{X}_1+\cdots +s_{K-1}{X}_{K-1}\right)}\right)={\mathrm{\Psi }}^{\left[K-1\right]}({\alpha }_1,\dots ,{\alpha }_{K-1};{\alpha }_0;i{s}_1,\dots ,i{s}_{K-1})}$$

where

$${\displaystyle {\mathrm{\Psi }}^{[m]}(a_1,\dots ,a_m;c;z_1,\dots z_m)=\sum \frac{(a_1{)}_{k_1}\cdots (a_m{)}_{k_m}{z}_1^{k_1}\cdots z_m^{k_m}}{(c{)}_k{k}_1!\cdots k_m!}.}$$

The sum is over non-negative integers${\displaystyle k_1,\dots ,k_m}$ and${\displaystyle k=k_1+\cdots +k_m}$. Phillips goes on to state that this form is "inconvenient for numerical calculation" and gives an alternative in terms of acomplex path integral:

$${\displaystyle {\mathrm{\Psi }}^{[m]}=\frac{\mathrm{\Gamma }(c)}{2\pi i}{\int }_L{e}^t{t}^{a_1+\cdots +a_m-c}\prod _{j=1}^m(t-z_j{)}^{-a_j}dt}$$

whereL denotes any path in the complex plane originating at${\displaystyle -\mathrm{\infty }}$, encircling in the positive direction all the singularities of the integrand and returning to${\displaystyle -\mathrm{\infty }}$.

Probability density function${\displaystyle f\left(x_1,\dots ,x_{K-1};{\alpha }_1,\dots ,{\alpha }_K\right)}$ plays a key role in a multifunctional inequality which implies various bounds for the Dirichlet distribution.^[19]

Another inequality relates the moment-generating function of the Dirichlet distribution to the convex conjugate of the scaled reversed Kullback-Leibler divergence:^[20]

$${\displaystyle \log \mathrm{E}\left(\exp \sum _{i=1}^K{s}_i{X}_i\right)\le \underset{p}{sup}\sum _{i=1}^K\left(p_i{s}_i-{\alpha }_i\log \left(\frac{{\alpha }_i}{{\alpha }_0{p}_i}\right)\right),}$$where the supremum is taken overp spanning the(K − 1)-simplex.

When${\displaystyle \mathit{X}=(X_1,\dots ,X_K)\sim \mathrm{Dir}\left({\alpha }_1,\dots ,{\alpha }_K\right)}$, the marginal distribution of each component${\displaystyle X_i\sim \mathrm{Beta}({\alpha }_i,{\alpha }_0-{\alpha }_i)}$, aBeta distribution. In particular, ifK = 2 then${\displaystyle X_1\sim \mathrm{Beta}({\alpha }_1,{\alpha }_2)}$ is equivalent to${\displaystyle \mathit{X}=(X_1,1-X_1)\sim \mathrm{Dir}\left({\alpha }_1,{\alpha }_2\right)}$.

ForK independently distributedGamma distributions:

$${\displaystyle Y_1\sim \mathrm{Gamma}({\alpha }_1,\theta ),\dots ,Y_K\sim \mathrm{Gamma}({\alpha }_K,\theta )}$$

we have:^[21]: 402

$${\displaystyle V=\sum _{i=1}^K{Y}_i\sim \mathrm{Gamma}\left({\alpha }_0,\theta \right),}$$$${\displaystyle X=(X_1,\dots ,X_K)=\left(\frac{Y_1}{V},\dots ,\frac{Y_K}{V}\right)\sim \mathrm{Dir}\left({\alpha }_1,\dots ,{\alpha }_K\right).}$$

Although theX_is are not independent from one another, they can be seen to be generated from a set ofK independentgamma random variables.^[21]: 594 Unfortunately, since the sumV is lost in formingX (in fact it can be shown thatV is stochastically independent ofX), it is not possible to recover the original gamma random variables from these values alone. Nevertheless, because independent random variables are simpler to work with, this reparametrization can still be useful for proofs about properties of the Dirichlet distribution.

Because the Dirichlet distribution is anexponential family distribution it has a conjugate prior.The conjugate prior is of the form:^[22]

$${\displaystyle \mathrm{CD}(\mathit{\alpha }\mid \mathit{v},\eta )\propto {\left(\frac{1}{\mathrm{B}(\mathit{\alpha })}\right)}^{\eta }\exp \left(-\sum _k{v}_k{\alpha }_k\right).}$$

Here${\displaystyle \mathit{v}}$ is aK-dimensional real vector and${\displaystyle \eta }$ is a scalar parameter. The domain of${\displaystyle (\mathit{v},\eta )}$ is restricted to the set of parameters for which the above unnormalized density function can be normalized. The (necessary and sufficient) condition is:^[23]

The conjugation property can be expressed as

if [prior:${\displaystyle \mathit{\alpha }\sim \mathrm{CD}(\cdot \mid \mathit{v},\eta )}$] and [observation:${\displaystyle \mathit{x}\mid \mathit{\alpha }\sim \mathrm{Dirichlet}(\cdot \mid \mathit{\alpha })}$] then [posterior:${\displaystyle \mathit{\alpha }\mid \mathit{x}\sim \mathrm{CD}(\cdot \mid \mathit{v}-\log \mathit{x},\eta +1)}$].

In the published literature there is no practical algorithm to efficiently generate samples from${\displaystyle \mathrm{CD}(\mathit{\alpha }\mid \mathit{v},\eta )}$.

As noted above, Dirichlet variates can be generated by normalizing independentgamma variates. If instead one normalizesgeneralized gamma variates, one obtains variates from the simplicial generalized beta distribution (SGB).^[24] On the other hand, SGB variates can also be obtained by applying thesoftmax function to scaled and translated logarithms of Dirichlet variates. Specifically, let${\displaystyle \mathbf{x}=(x_1,\dots ,x_K)\sim \mathrm{Dir}(\mathit{\alpha })}$ and let${\displaystyle \mathbf{y}=(y_1,\dots ,y_K)}$, where applying the logarithm elementwise:$${\displaystyle \mathbf{y}=\mathrm{softmax}(a^{-1}\log \mathbf{x}+\log \mathbf{b})⟺\mathbf{x}=\mathrm{softmax}(a\log \mathbf{y}-a\log \mathbf{b})}$$or$${\displaystyle y_k=\frac{b_k{x}_k^{1/a}}{\sum _{i=1}^K{b}_i{x}_i^{1/a}}⟺x_k=\frac{(y_k/b_k{)}^a}{\sum _{i=1}^K(y_i/b_i{)}^a}}$$where${\displaystyle a>0}$ and${\displaystyle \mathbf{b}=(b_1,\dots ,b_K)}$, with all${\displaystyle b_k>0}$, then${\displaystyle \mathbf{y}\sim \mathrm{SGB}(a,\mathbf{b},\mathit{\alpha })}$. The SGB density function can be derived by noting that the transformation${\displaystyle \mathbf{x}\mapsto \mathbf{y}}$, which is abijection from the simplex to itself, induces adifferential volume change factor^[25] of:$${\displaystyle R(\mathbf{y},a,\mathbf{b})=a^{1-K}\prod _{k=1}^K\frac{y_k}{x_k}}$$where it is understood that${\displaystyle \mathbf{x}}$ is recovered as a function of${\displaystyle \mathbf{y}}$, as shown above. This facilitates writing the SGB density in terms of the Dirichlet density, as:$${\displaystyle f_{\text{SGB}}(\mathbf{y}\mid a,\mathbf{b},\mathit{\alpha })=\frac{f_{\text{Dir}}(\mathbf{x}\mid \mathit{\alpha })}{R(\mathbf{y},a,\mathbf{b})}}$$This generalization of the Dirichlet density, via achange of variables, is closely related to anormalizing flow, while it must be noted that the differential volume change is not given by theJacobian determinant of${\displaystyle \mathbf{x}\mapsto \mathbf{y}:{\mathbb{R}}^K\to {\mathbb{R}}^K}$ which is zero, but by the Jacobian determinant of${\displaystyle (x_1,\dots ,x_{K-1})\mapsto \mathbf{(}{y}_1,\dots ,y_{K-1})}$, as explained in more detail atNormalizing flow § Simplex flow.

For further insight into the interaction between the Dirichlet shape parameters${\displaystyle \mathit{\alpha }}$, and the transformation parameters${\displaystyle a,\mathbf{b}}$, it may be helpful to consider the logarithmic marginals,${\displaystyle \log \frac{x_k}{1-x_k}}$, which follow thelogistic-beta distribution,${\displaystyle B_{\sigma }({\alpha }_k,\sum _{i\ne k}{\alpha }_i)}$. See in particular the sections ontail behaviour andgeneralization with location and scale parameters.

When${\displaystyle b_1=b_2=\cdots =b_K}$, then the transformation simplifies to${\displaystyle \mathbf{x}\mapsto \mathrm{softmax}(a^{-1}\log \mathbf{x})}$, which is known astemperature scaling inmachine learning, where it is used as a calibration transform for multiclass probabilistic classifiers.^[26] Traditionally the temperature parameter (${\displaystyle a}$ here) is learntdiscriminatively by minimizing multiclasscross-entropy over a supervised calibration data set with known class labels. But the above PDF transformation mechanism can be used to facilitate also the design ofgeneratively trained calibration models with a temperature scaling component.

Dirichlet distributions are most commonly used as theprior distribution ofcategorical variables ormultinomial variables in Bayesianmixture models and otherhierarchical Bayesian models. (In many fields, such as innatural language processing, categorical variables are often imprecisely called "multinomial variables". Such a usage is unlikely to cause confusion, just as whenBernoulli distributions andbinomial distributions are commonly conflated.)

Inference over hierarchical Bayesian models is often done usingGibbs sampling, and in such a case, instances of the Dirichlet distribution are typicallymarginalized out of the model by integrating out the Dirichletrandom variable. This causes the various categorical variables drawn from the same Dirichlet random variable to become correlated, and the joint distribution over them assumes aDirichlet-multinomial distribution, conditioned on the hyperparameters of the Dirichlet distribution (theconcentration parameters). One of the reasons for doing this is that Gibbs sampling of theDirichlet-multinomial distribution is extremely easy; see that article for more information.

Dirichlet distributions are very often used asprior distributions inBayesian inference. The simplest and perhaps most common type of Dirichlet prior is the symmetric Dirichlet distribution, where all parameters are equal. This corresponds to the case where you have no prior information to favor one component over any other. As described above, the single valueα to which all parameters are set is called theconcentration parameter. If the sample space of the Dirichlet distribution is interpreted as adiscrete probability distribution, then intuitively the concentration parameter can be thought of as determining how "concentrated" the probability mass of the Dirichlet distribution to its center, leading to samples with mass dispersed almost equally among all components, i.e., with a value much less than 1, the mass will be highly concentrated in a few components, and all the rest will have almost no mass, and with a value much greater than 1, the mass will be dispersed almost equally among all the components. See the article on theconcentration parameter for further discussion.

One example use of the Dirichlet distribution is if one wanted to cut strings (each of initial length 1.0) intoK pieces with different lengths, where each piece had a designated average length, but allowing some variation in the relative sizes of the pieces. Recall that${\displaystyle {\alpha }_0=\sum _{i=1}^K{\alpha }_i.}$ The${\displaystyle {\alpha }_i/{\alpha }_0}$ values specify the mean lengths of the cut pieces of string resulting from the distribution. The variance around this mean varies inversely with${\displaystyle {\alpha }_0}$.

Consider an urn containing balls ofK different colors. Initially, the urn containsα₁ balls of color 1,α₂ balls of color 2, and so on. Now performN draws from the urn, where after each draw, the ball is placed back into the urn with an additional ball of the same color. In the limit asN approaches infinity, the proportions of different colored balls in the urn will be distributed asDir(α₁, ...,α_K).^[27]

For a formal proof, note that the proportions of the different colored balls form a bounded[0,1]^K-valuedmartingale, hence by themartingale convergence theorem, these proportions convergealmost surely andin mean to a limiting random vector. To see that this limiting vector has the above Dirichlet distribution, check that all mixedmoments agree.

Each draw from the urn modifies the probability of drawing a ball of any one color from the urn in the future. This modification diminishes with the number of draws, since the relative effect of adding a new ball to the urn diminishes as the urn accumulates increasing numbers of balls.

With a source of Gamma-distributed random variates, one can easily sample a random vector${\displaystyle x=(x_1,\dots ,x_K)}$ from theK-dimensional Dirichlet distribution with parameters${\displaystyle ({\alpha }_1,\dots ,{\alpha }_K)}$ . First, drawK independent random samples${\displaystyle y_1,\dots ,y_K}$ fromGamma distributions each with density

$${\displaystyle \mathrm{Gamma}({\alpha }_i,1)=\frac{y_i^{{\alpha }_i-1}{e}^{-y_i}}{\mathrm{\Gamma }({\alpha }_i)},}$$

and then set

$${\displaystyle x_i=\frac{y_i}{\sum _{j=1}^K{y}_j}.}$$

Below is example Python code to draw the sample:

params=[a1,a2,...,ak]sample=[random.gammavariate(a,1)forainparams]sample=[v/sum(sample)forvinsample]

This formulation is correct regardless of how the Gamma distributions are parameterized (shape/scale vs. shape/rate) because they are equivalent when scale and rate equal 1.0.

A less efficient algorithm^[28] relies on the univariate marginal and conditional distributions being beta and proceeds as follows. Simulate${\displaystyle x_1}$ from

$${\displaystyle \text{Beta}\left({\alpha }_1,\sum _{i=2}^K{\alpha }_i\right)}$$

Then simulate${\displaystyle x_2,\dots ,x_{K-1}}$ in order, as follows. For${\displaystyle j=2,\dots ,K-1}$, simulate${\displaystyle {\varphi }_j}$ from

$${\displaystyle \text{Beta}\left({\alpha }_j,\sum _{i=j+1}^K{\alpha }_i\right),}$$

and let

$${\displaystyle x_j=\left(1-\sum _{i=1}^{j-1}{x}_i\right){\varphi }_j.}$$

Finally, set

$${\displaystyle x_K=1-\sum _{i=1}^{K-1}{x}_i.}$$

This iterative procedure corresponds closely to the "string cutting" intuition described above.

Below is example Python code to draw the sample:

params=[a1,a2,...,ak]xs=[random.betavariate(params[0],sum(params[1:]))]forjinrange(1,len(params)-1):phi=random.betavariate(params[j],sum(params[j+1:]))xs.append((1-sum(xs))*phi)xs.append(1-sum(xs))

Whenα₁ = ... =α_K = 1, a sample from the distribution can be found by randomly drawing a set ofK − 1 values independently and uniformly from the interval[0, 1], adding the values0 and1 to the set to make it haveK + 1 values, sorting the set, and computing the difference between each pair of order-adjacent values, to givex₁, ...,x_K.

Whenα₁ = ... =α_K = 1/2, a sample from the distribution can be found by randomly drawingK values independently from the standard normal distribution, squaring these values, and normalizing them by dividing by their sum, to givex₁, ...,x_K.

A point(x₁, ...,x_K) can be drawn uniformly at random from the (K−1)-dimensional unit hypersphere (which is the surface of aK-dimensionalhyperball) via a similar procedure. Randomly drawK values independently from the standard normal distribution and normalize these coordinate values by dividing each by the constant that is the square root of the sum of their squares.

• Generalized Dirichlet distribution
• Grouped Dirichlet distribution
• Inverted Dirichlet distribution
• Latent Dirichlet allocation
• Dirichlet process
• Matrix variate Dirichlet distribution

• "Dirichlet distribution",Encyclopedia of Mathematics,EMS Press, 2001 [1994]
• Dirichlet Distribution
• How to estimate the parameters of the compound Dirichlet distribution (Pólya distribution) using expectation-maximization (EM)
• Luc Devroye."Non-Uniform Random Variate Generation". Retrieved19 October 2019.
• Dirichlet Random Measures, Method of Construction via Compound Poisson Random Variables, and Exchangeability Properties of the resulting Gamma Distribution
• SciencesPo: R package that contains functions for simulating parameters of the Dirichlet distribution.

• Multivariate continuous distributions
• Conjugate prior distributions
• Exponential family distributions
• Continuous distributions

• CS1 maint: numeric names: authors list
• Articles with short description
• Short description matches Wikidata