Movatterモバイル変換

[0]ホーム
URL:

Jump to content
WikipediaThe Free Encyclopedia
Search

Multinomial distribution

From Wikipedia, the free encyclopedia
Generalization of the binomial distribution
Multinomial Distribution
Parameters

n∈{0,1,2,…}{\displaystyle n\in \{0,1,2,\ldots \}} number of trials
k>0{\displaystyle k>0} number of mutually exclusive events (integer)

p1,…,pk{\displaystyle p_{1},\ldots ,p_{k}} event probabilities, wherep1+⋯+pk=1{\displaystyle p_{1}+\dots +p_{k}=1}
Support{(x1,…,xk)|∑i=1kxi=n,xi≥0 (i=1,…,k)}{\displaystyle \left\lbrace (x_{1},\dots ,x_{k})\,{\Big \vert }\,\sum _{i=1}^{k}x_{i}=n,x_{i}\geq 0\ (i=1,\dots ,k)\right\rbrace }
PMFn!x1!⋯xk!p1x1⋯pkxk{\displaystyle {\frac {n!}{x_{1}!\cdots x_{k}!}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}}
MeanE⁡(Xi)=npi{\displaystyle \operatorname {E} (X_{i})=np_{i}}
VarianceVar⁡(Xi)=npi(1−pi){\displaystyle \operatorname {Var} (X_{i})=np_{i}(1-p_{i})}
Cov⁡(Xi,Xj)=−npipj  (i≠j){\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}~~(i\neq j)}
Entropy−log⁡(n!)−n∑i=1kpilog⁡(pi)+∑i=1k∑xi=0n(nxi)pixi(1−pi)n−xilog⁡(xi!){\displaystyle {\begin{aligned}&-\log(n!)-n\sum _{i=1}^{k}p_{i}\log(p_{i})\\&+\sum _{i=1}^{k}\sum _{x_{i}=0}^{n}{\binom {n}{x_{i}}}p_{i}^{x_{i}}(1-p_{i})^{n-x_{i}}\log(x_{i}!)\end{aligned}}}
MGF(∑i=1kpieti)n{\displaystyle \left(\sum _{i=1}^{k}p_{i}e^{t_{i}}\right)^{n}}
CF(∑j=1kpjeitj)n{\displaystyle \left(\sum _{j=1}^{k}p_{j}e^{it_{j}}\right)^{n}} wherei2=−1{\displaystyle i^{2}=-1}
PGF(∑i=1kpizi)n{\displaystyle \left(\sum _{i=1}^{k}p_{i}z_{i}\right)^{n}} for(z1,…,zk)∈Ck{\displaystyle (z_{1},\ldots ,z_{k})\in \mathbb {C} ^{k}}

Inprobability theory, themultinomial distribution is a generalization of thebinomial distribution. For example, it models the probability of counts for each side of ak-sided die rolledn times. Fornindependent trials each of which leads to a success for exactly one ofk categories, with each category having a given fixed success probability, the multinomial distribution gives the probability of any particular combination of numbers of successes for the various categories.

Whenk is 2 andn is 1, the multinomial distribution is theBernoulli distribution. Whenk is 2 andn is bigger than 1, it is thebinomial distribution. Whenk is bigger than 2 andn is 1, it is thecategorical distribution. The term "multinoulli" is sometimes used for the categorical distribution to emphasize this four-way relationship (son determines the suffix, andk the prefix).

TheBernoulli distribution models the outcome of a singleBernoulli trial. In other words, it models whether flipping a (possiblybiased) coin one time will result in either a success (obtaining a head) or failure (obtaining a tail). Thebinomial distribution generalizes this to the number of heads from performingn independent flips (Bernoulli trials) of the same coin. The multinomial distribution models the outcome ofn experiments, where the outcome of each trial has acategorical distribution, such as rolling a (possiblybiased)k-sided dien times.

Letk be a fixed finite number. Mathematically, we havek possible mutually exclusive outcomes, with corresponding probabilitiesp1, ...,pk, andn independent trials. Since thek outcomes are mutually exclusive and one must occur we havepi ≥ 0 fori = 1, ..., k and∑i=1kpi=1{\textstyle \sum _{i=1}^{k}p_{i}=1}. Then if the random variablesXi indicate the number of times outcome numberi is observed over then trials, the vectorX = (X1, ..., Xk) follows a multinomial distribution with parametersn andp, wherep = (p1, ..., pk). While the trials are independent, their outcomesXi are dependent because they must sum to n.

Definitions

[edit]

Probability mass function

[edit]

Suppose one does an experiment of extractingn balls ofk different colors from a bag, replacing the extracted balls after each draw. Balls of the same color are equivalent. Denote the variable which is the number of extracted balls of colori (i = 1, ...,k) asXi, and denote aspi the probability that a given extraction will be in colori. Theprobability mass function of this multinomial distribution is:

f(x1,…,xk;n,p1,…,pk)=Pr(X1=x1 and … and Xk=xk)={n!x1!⋯xk!p1x1×⋯×pkxk,when ∑i=1kxi=n0otherwise,{\displaystyle {\begin{aligned}f(x_{1},\ldots ,x_{k};n,p_{1},\ldots ,p_{k})&{}=\Pr(X_{1}=x_{1}{\text{ and }}\dots {\text{ and }}X_{k}=x_{k})\\[1ex]&{}={\begin{cases}{\displaystyle {n! \over x_{1}!\cdots x_{k}!}p_{1}^{x_{1}}\times \cdots \times p_{k}^{x_{k}}},\quad &{\text{when }}\sum _{i=1}^{k}x_{i}=n\\\\0&{\text{otherwise,}}\end{cases}}\end{aligned}}}

for non-negative integersx1, ...,xk.

The probability mass function can be expressed using thegamma function as:

f(x1,…,xk;p1,…,pk)=Γ(∑ixi+1)∏iΓ(xi+1)∏i=1kpixi.{\displaystyle f(x_{1},\dots ,x_{k};p_{1},\ldots ,p_{k})={\frac {\Gamma (\sum _{i}x_{i}+1)}{\prod _{i}\Gamma (x_{i}+1)}}\prod _{i=1}^{k}p_{i}^{x_{i}}.}

This form shows its resemblance to theDirichlet distribution, which is itsconjugate prior.

Example

[edit]

Suppose that in a three-way election for a large country, candidate A received 20% of the votes, candidate B received 30% of the votes, and candidate C received 50% of the votes. If six voters are selected randomly, what is the probability that there will be exactly one supporter for candidate A, two supporters for candidate B and three supporters for candidate C in the sample?

Note: Since we’re assuming that the voting population is large, it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample. Technically speaking this is sampling without replacement, so the correct distribution is themultivariate hypergeometric distribution, but the distributions converge as the population grows large in comparison to a fixed sample size.[1]

Pr(A=1,B=2,C=3)=6!1!2!3!(0.21)(0.32)(0.53)=0.135{\displaystyle \Pr(A{=}1,B{=}2,C{=}3)={\frac {6!}{1!2!3!}}\left(0.2^{1}\right)\left(0.3^{2}\right)\left(0.5^{3}\right)=0.135}

Properties

[edit]

Normalization

[edit]

The multinomial distribution is normalized according to:

∑∑j=1kxj=nf(x1,…,xk;n,p1,…,pk)=1{\displaystyle \sum _{\sum _{j=1}^{k}x_{j}=n}f(x_{1},\dots ,x_{k};n,p_{1},\dots ,p_{k})=1}

where the sum is over all permutations ofxj{\displaystyle x_{j}}such that∑j=1kxj=n{\textstyle \sum _{j=1}^{k}x_{j}=n}.

Expected value and variance

[edit]

Theexpected number of times the outcomei was observed overn trials is

E⁡(Xi)=npi.{\displaystyle \operatorname {E} (X_{i})=np_{i}.\,}

Thecovariance matrix is as follows. Each diagonal entry is thevariance of a binomially distributed random variable, and is therefore

Var⁡(Xi)=npi(1−pi).{\displaystyle \operatorname {Var} (X_{i})=np_{i}(1-p_{i}).\,}

The off-diagonal entries are thecovariances:

Cov⁡(Xi,Xj)=−npipj{\displaystyle \operatorname {Cov} (X_{i},X_{j})=-np_{i}p_{j}\,}

fori,j distinct.

All covariances are negative because for fixedn, an increase in one component of a multinomial vector requires a decrease in another component.

When these expressions are combined into a matrix withi, j elementcov⁡(Xi,Xj),{\displaystyle \operatorname {cov} (X_{i},X_{j}),} the result is ak ×kpositive-semidefinitecovariance matrix of rankk − 1. In the special case wherek = n and where thepi are all equal, the covariance matrix is thecentering matrix.

The entries of the correspondingcorrelation matrix are

ρ(Xi,Xi)=1ρ(Xi,Xj)=Cov⁡(Xi,Xj)Var⁡(Xi)Var⁡(Xj)=−pipjpi(1−pi)pj(1−pj)=−pipj(1−pi)(1−pj).{\displaystyle {\begin{aligned}\rho (X_{i},X_{i})&=1\\[1ex]\rho (X_{i},X_{j})&={\frac {\operatorname {Cov} (X_{i},X_{j})}{\sqrt {\operatorname {Var} (X_{i})\operatorname {Var} (X_{j})}}}\\&={\frac {-p_{i}p_{j}}{\sqrt {p_{i}(1-p_{i})p_{j}(1-p_{j})}}}\\&=-{\sqrt {\frac {p_{i}p_{j}}{(1-p_{i})(1-p_{j})}}}.\end{aligned}}}

Note that the number of trialsn drops out of this expression.

Each of thek components separately has a binomial distribution with parametersn andpi, for the appropriate value of the subscripti.

Thesupport of the multinomial distribution is the set

{(n1,…,nk)∈Nk∣n1+⋯+nk=n}.{\displaystyle \left\{(n_{1},\dots ,n_{k})\in \mathbb {N} ^{k}\mid n_{1}+\cdots +n_{k}=n\right\}.}

Its number of elements is

(n+k−1k−1).{\displaystyle {\binom {n+k-1}{k-1}}.}

Matrix notation

[edit]

In matrix notation,E⁡(X)=np,{\displaystyle \operatorname {E} (\mathbf {X} )=n\mathbf {p} ,\,}

andVar⁡(X)=n{diag⁡(p)−ppT},{\displaystyle \operatorname {Var} (\mathbf {X} )=n\lbrace \operatorname {diag} (\mathbf {p} )-\mathbf {p} \mathbf {p} ^{\rm {T}}\rbrace ,\,}

withpT = therow vector transpose of the column vectorp.

Visualization

[edit]

As slices of generalized Pascal's triangle

[edit]

Just like one can interpret thebinomial distribution as (normalized) one-dimensional (1D) slices ofPascal's triangle, so too can one interpret the multinomial distribution as 2D (triangular) slices ofPascal's pyramid, or 3D/4D/+ (pyramid-shaped) slices of higher-dimensional analogs of Pascal's triangle. This reveals an interpretation of therange of the distribution: discretized equilateral "pyramids" in arbitrary dimension—i.e. asimplex with a grid.[citation needed]

As polynomial coefficients

[edit]

Similarly, just like one can interpret thebinomial distribution as the polynomial coefficients of(p+q)n{\displaystyle (p+q)^{n}} when expanded, one can interpret the multinomial distribution as the coefficients of(p1+p2+p3+⋯+pk)n{\displaystyle (p_{1}+p_{2}+p_{3}+\cdots +p_{k})^{n}} when expanded, noting that just the coefficients must sum up to 1.

Large deviation theory

[edit]
See also:Sanov's theorem

Asymptotics

[edit]

ByStirling's formula, at the limit ofn,x1,…,xk→∞{\displaystyle n,x_{1},\dots ,x_{k}\to \infty }, we haveln⁡(nx1,…,xk)+∑i=1kxiln⁡pi=−nDKL(p^‖p)−k−12ln⁡(2πn)−12∑i=1kln⁡(p^i)+o(1){\displaystyle \ln {\binom {n}{x_{1},\dots ,x_{k}}}+\sum _{i=1}^{k}x_{i}\ln p_{i}=-nD_{\text{KL}}({\hat {p}}\|p)-{\frac {k-1}{2}}\ln(2\pi n)-{\frac {1}{2}}\sum _{i=1}^{k}\ln({\hat {p}}_{i})+o(1)}where relative frequenciesp^i=xi/n{\displaystyle {\hat {p}}_{i}=x_{i}/n} in the data can be interpreted as probabilities from the empirical distributionp^{\displaystyle {\hat {p}}}, andDKL{\displaystyle D_{\text{KL}}} is theKullback–Leibler divergence.

This formula can be interpreted as follows.

ConsiderΔk{\displaystyle \Delta _{k}}, the space of all possible distributions over the categories{1,2,…,k}{\displaystyle \{1,2,\dots ,k\}}. It is asimplex. Aftern{\displaystyle n} independent samples from the categorical distributionp{\displaystyle p} (which is how we construct the multinomial distribution), we obtain an empirical distributionp^{\displaystyle {\hat {p}}}.

By the asymptotic formula, the probability that empirical distributionp^{\displaystyle {\hat {p}}} deviates from the actual distributionp{\displaystyle p} decays exponentially as we sample more data, at a rate ofDKL(p^‖p){\displaystyle D_{\text{KL}}({\hat {p}}\|p)}. The more experiments and the more differentp^{\displaystyle {\hat {p}}} is fromp{\displaystyle p}, the less likely it is to see such an empirical distribution.

IfA{\displaystyle A} is a closed subset ofΔk{\displaystyle \Delta _{k}}, then by dividing upA{\displaystyle A} into pieces, and reasoning about the growth rate ofPr(p^∈Aϵ){\displaystyle Pr({\hat {p}}\in A_{\epsilon })} on each pieceAϵ{\displaystyle A_{\epsilon }}, we obtainSanov's theorem, which states thatlimn→∞1nln⁡Pr(p^∈A)=−infp^∈ADKL(p^‖p){\displaystyle \lim _{n\to \infty }{\frac {1}{n}}\ln \Pr({\hat {p}}\in A)=-\inf _{{\hat {p}}\in A}D_{\text{KL}}({\hat {p}}\|p)}

Concentration at largen

[edit]

Due to theexponential decay, at largen{\displaystyle n}, almost all the probability mass is concentrated in a small neighborhood ofp{\displaystyle p}. In this small neighborhood, we can take the first nonzero term in theTaylor expansion ofDKL{\displaystyle D_{KL}}, to obtainln⁡(nx1,⋯,xk)p1x1⋯pkxk≈−n2∑i=1k(p^i−pi)2pi=−12∑i=1k(xi−npi)2npi{\displaystyle {\begin{aligned}\ln {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}&\approx -{\frac {n}{2}}\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}\\&=-{\frac {1}{2}}\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}\end{aligned}}}This resembles the Gaussian distribution, which suggests the following theorem:

Theorem. At then→∞{\displaystyle n\to \infty } limit,n∑i=1k(p^i−pi)2pi=∑i=1k(xi−npi)2npi{\displaystyle n\sum _{i=1}^{k}{\frac {({\hat {p}}_{i}-p_{i})^{2}}{p_{i}}}=\sum _{i=1}^{k}{\frac {(x_{i}-np_{i})^{2}}{np_{i}}}}converges in distribution to thechi-squared distributionχ2(k−1){\displaystyle \chi ^{2}(k-1)}.

If we sample from the multinomial distributionMultinomial(n;0.2,0.3,0.5){\displaystyle \mathrm {Multinomial} (n;0.2,0.3,0.5)}, and plot the heatmap of the samples within the 2-dimensional simplex (here shown as a black triangle), we notice that asn→∞{\displaystyle n\to \infty }, the distribution converges to a Gaussian around the point(0.2,0.3,0.5){\displaystyle (0.2,0.3,0.5)}, with the contours converging in shape to ellipses, with radii converging as1/n{\displaystyle 1/{\sqrt {n}}}. Meanwhile, the separation between the discrete points converge as1/n{\displaystyle 1/n}, and so the discrete multinomial distribution converges to a continuous Gaussian distribution.
[Proof]

The space of all distributions over categories{1,2,…,k}{\displaystyle \{1,2,\ldots ,k\}} is asimplex:Δk={(y1,…,yk):y1,…,yk≥0,∑iyi=1}{\displaystyle \Delta _{k}=\left\{(y_{1},\ldots ,y_{k})\colon y_{1},\ldots ,y_{k}\geq 0,\sum _{i}y_{i}=1\right\}}, and the set of all possible empirical distributions aftern{\displaystyle n} experiments is a subset of the simplex:Δk,n={(x1/n,…,xk/n):x1,…,xk∈N,∑ixi=n}{\displaystyle \Delta _{k,n}=\left\{(x_{1}/n,\ldots ,x_{k}/n)\colon x_{1},\ldots ,x_{k}\in \mathbb {N} ,\sum _{i}x_{i}=n\right\}}. That is, it is the intersection betweenΔk{\displaystyle \Delta _{k}} and the lattice(Zk)/n{\displaystyle (\mathbb {Z} ^{k})/n}.

Asn{\displaystyle n} increases, most of the probability mass is concentrated in a subset ofΔk,n{\displaystyle \Delta _{k,n}} nearp{\displaystyle p}, and the probability distribution nearp{\displaystyle p} becomes well-approximated by(nx1,⋯,xk)p1x1⋯pkxk≈e−n2∑i(p^i−pi)2pi{\displaystyle {\binom {n}{x_{1},\cdots ,x_{k}}}p_{1}^{x_{1}}\cdots p_{k}^{x_{k}}\approx e^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}From this, we see that the subset upon which the mass is concentrated has radius on the order of1/n{\displaystyle 1/{\sqrt {n}}}, but the points in the subset are separated by distance on the order of1/n{\displaystyle 1/n}, so at largen{\displaystyle n}, the points merge into a continuum.To convert this from a discrete probability distribution to a continuous probability density, we need to multiply by the volume occupied by each point ofΔk,n{\displaystyle \Delta _{k,n}} inΔk{\displaystyle \Delta _{k}}. However, by symmetry, every point occupies exactly the same volume (except a negligible set on the boundary), so we obtain a probability densityρ(p^)=Ce−n2∑i(p^i−pi)2pi{\displaystyle \rho ({\hat {p}})=Ce^{-{\frac {n}{2}}\sum _{i}{\frac {\left({\hat {p}}_{i}-p_{i}\right)^{2}}{p_{i}}}}}, whereC{\displaystyle C} is a constant.

Finally, since the simplexΔk{\displaystyle \Delta _{k}} is not all ofRk{\displaystyle \mathbb {R} ^{k}}, but only within a(k−1){\displaystyle (k-1)}-dimensional plane, we obtain the desired result.

Conditional concentration at largen

[edit]

The above concentration phenomenon can be easily generalized to the case where we condition upon independent constraints. This is the theoretical justification forPearson's chi-squared test.

Theorem.

In the case that allp^i{\displaystyle {\hat {p}}_{i}} are equal, this reduces to the concentration of entropies around themaximum entropy.[2][3]

This theorem can be shown by starting with the previous case, then taking the conditional on the constraints.

Related distributions

[edit]

In some fields such asnatural language processing, categorical and multinomial distributions are synonymous and it is common to speak of a multinomial distribution when acategorical distribution is actually meant. This stems from the fact that it is sometimes convenient to express the outcome of a categorical distribution as a "1-of-k" vector (a vector with one element containing a 1 and all other elements containing a 0) rather than as an integer in the range1…k{\displaystyle 1\dots k}; in this form, a categorical distribution is equivalent to a multinomial distribution over a single trial.

Statistical inference

[edit]
[icon]
This sectionneeds expansion with: A new sub-section about simultaneous confidence intervals (with proper citations, e.g.:[1]).. You can help byadding to it.(March 2024)

Equivalence tests for multinomial distributions

[edit]

The goal of equivalence testing is to establish the agreement between a theoretical multinomial distribution and observed counting frequencies. The theoretical distribution may be a fully specified multinomial distribution or a parametric family of multinomial distributions.

Letq{\displaystyle q} denote a theoretical multinomial distribution and letp{\displaystyle p} be a true underlying distribution. The distributionsp{\displaystyle p} andq{\displaystyle q} are considered equivalent ifd(p,q)<ε{\displaystyle d(p,q)<\varepsilon } for a distanced{\displaystyle d} and a tolerance parameterε>0{\displaystyle \varepsilon >0}. The equivalence test problem isH0={d(p,q)≥ε}{\displaystyle H_{0}=\{d(p,q)\geq \varepsilon \}} versusH1={d(p,q)<ε}{\displaystyle H_{1}=\{d(p,q)<\varepsilon \}}. The true underlying distributionp{\displaystyle p} is unknown. Instead, the counting frequenciespn{\displaystyle p_{n}} are observed, wheren{\displaystyle n} is a sample size. An equivalence test usespn{\displaystyle p_{n}} to rejectH0{\displaystyle H_{0}}. IfH0{\displaystyle H_{0}} can be rejected then the equivalence betweenp{\displaystyle p} andq{\displaystyle q} is shown at a given significance level. The equivalence test for Euclidean distance can be found in text book of Wellek (2010).[4] The equivalence test for the total variation distance is developed in Ostrovski (2017).[5] The exact equivalence test for the specific cumulative distance is proposed in Frey (2009).[6]

The distance between the true underlying distributionp{\displaystyle p} and a family of the multinomial distributionsM{\displaystyle {\mathcal {M}}} is defined byd(p,M)=minh∈Md(p,h){\displaystyle d(p,{\mathcal {M}})=\min _{h\in {\mathcal {M}}}d(p,h)}. Then the equivalence test problem is given byH0={d(p,M)≥ε}{\displaystyle H_{0}=\{d(p,{\mathcal {M}})\geq \varepsilon \}} andH1={d(p,M)<ε}{\displaystyle H_{1}=\{d(p,{\mathcal {M}})<\varepsilon \}}. The distanced(p,M){\displaystyle d(p,{\mathcal {M}})} is usually computed using numerical optimization. The tests for this case are developed recently in Ostrovski (2018).[7]

Confidence intervals for the difference of two proportions

[edit]

In the setting of a multinomial distribution, constructing confidence intervals for the difference between the proportions of observations from two events,pi−pj{\displaystyle p_{i}-p_{j}}, requires the incorporation of the negative covariance between the sample estimatorsp^i=Xin{\displaystyle {\hat {p}}_{i}={\frac {X_{i}}{n}}} andp^j=Xjn{\displaystyle {\hat {p}}_{j}={\frac {X_{j}}{n}}}.

Some of the literature on the subject focused on the use-case of matched-pairs binary data, which requires careful attention when translating the formulas to the general case ofpi−pj{\displaystyle p_{i}-p_{j}} for any multinomial distribution. Formulas in the current section will be generalized, while formulas in the next section will focus on the matched-pairs binary data use-case.

Wald's standard error (SE) of the difference of proportion can be estimated using:[8]: 378 [9]

SE^(p^i−p^j)=(p^i+p^j)−(p^i−p^j)2n{\displaystyle {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}\right)-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}}{n}}}}

For a100(1−α)%{\displaystyle 100(1-\alpha )\%}approximate confidence interval, themargin of error may incorporate the appropriate quantile from thestandard normal distribution, as follows:

(p^i−p^j)±zα/2⋅SE^(p^i−p^j){\displaystyle ({\hat {p}}_{i}-{\hat {p}}_{j})\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})}

[Proof]

As the sample size (n{\displaystyle n}) increases, the sample proportions will approximately follow amultivariate normal distribution, thanks to themultidimensional central limit theorem (and it could also be shown using theCramér–Wold theorem). Therefore, their difference will also be approximately normal. Also, these estimators areweakly consistent and plugging them into the SE estimator makes it also weakly consistent. Hence, thanks toSlutsky's theorem, thepivotal quantity(p^i−p^j)−(pi−pj)SE⁡(p^i−p^j)^{\displaystyle {\frac {({\hat {p}}_{i}-{\hat {p}}_{j})-(p_{i}-p_{j})}{\widehat {\operatorname {SE} ({\hat {p}}_{i}-{\hat {p}}_{j})}}}} approximately follows thestandard normal distribution. And from that, the aboveapproximate confidence interval is directly derived.

The SE can be constructed using the calculus ofthe variance of the difference of two random variables:SE^(p^i−p^j)=p^i(1−p^i)n+p^j(1−p^j)n−2(−p^ip^jn)=1n(p^i+p^j−p^i2−p^j2+2p^ip^j)=(p^i+p^j)−(p^i−p^j)2n{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})&={\sqrt {{\frac {{\hat {p}}_{i}(1-{\hat {p}}_{i})}{n}}+{\frac {{\hat {p}}_{j}(1-{\hat {p}}_{j})}{n}}-2\left(-{\frac {{\hat {p}}_{i}{\hat {p}}_{j}}{n}}\right)}}\\&={\sqrt {{\frac {1}{n}}\left({\hat {p}}_{i}+{\hat {p}}_{j}-{\hat {p}}_{i}^{2}-{\hat {p}}_{j}^{2}+2{\hat {p}}_{i}{\hat {p}}_{j}\right)}}\\&={\sqrt {\frac {({\hat {p}}_{i}+{\hat {p}}_{j})-({\hat {p}}_{i}-{\hat {p}}_{j})^{2}}{n}}}\end{aligned}}}

A modification which includes acontinuity correction adds1n{\displaystyle {\frac {1}{n}}} to the margin of error as follows:[10]: 102–103 

(p^i−p^j)±(zα/2⋅SE^(p^i−p^j)+1n){\displaystyle ({\hat {p}}_{i}-{\hat {p}}_{j})\pm \left(z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})+{\frac {1}{n}}\right)}

Another alternative is to rely on a Bayesian estimator usingJeffreys prior which leads to using adirichlet distribution, with all parameters being equal to 0.5, as a prior. The posterior will be the calculations from above, but after adding 1/2 to each of thek elements, leading to an overall increase of the sample size byk2{\displaystyle {\frac {k}{2}}}. This was originally developed for a multinomial distribution with four events, and is known aswald+2, for analyzing matched pairs data (see the next section for more details).[11]

This leads to the following SE:

SE^(p^i−p^j)wald+k2=(p^i+p^j+1n)nn+k2−(p^i−p^j)2(nn+k2)2n+k2{\displaystyle {\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}}

[Proof]

SE^(p^i−p^j)wald+k2=(xi+1/2n+k2+xj+1/2n+k2)−(xi+1/2n+k2−xj+1/2n+k2)2n+k2=(xin+xjn+1n)nn+k2−(xin−xjn)2(nn+k2)2n+k2=(p^i+p^j+1n)nn+k2−(p^i−p^j)2(nn+k2)2n+k2{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}&={\sqrt {\frac {\left({\frac {x_{i}+1/2}{n+{\frac {k}{2}}}}+{\frac {x_{j}+1/2}{n+{\frac {k}{2}}}}\right)-\left({\frac {x_{i}+1/2}{n+{\frac {k}{2}}}}-{\frac {x_{j}+1/2}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\\&={\sqrt {\frac {\left({\frac {x_{i}}{n}}+{\frac {x_{j}}{n}}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\frac {x_{i}}{n}}-{\frac {x_{j}}{n}}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\\&={\sqrt {\frac {\left({\hat {p}}_{i}+{\hat {p}}_{j}+{\frac {1}{n}}\right){\frac {n}{n+{\frac {k}{2}}}}-\left({\hat {p}}_{i}-{\hat {p}}_{j}\right)^{2}\left({\frac {n}{n+{\frac {k}{2}}}}\right)^{2}}{n+{\frac {k}{2}}}}}\end{aligned}}}

Which can just be plugged into the original Wald formula as follows:

(pi−pj)nn+k2±zα/2⋅SE^(p^i−p^j)wald+k2{\displaystyle \left(p_{i}-p_{j}\right){\frac {n}{n+{\frac {k}{2}}}}\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}{({\hat {p}}_{i}-{\hat {p}}_{j})}_{wald+{\frac {k}{2}}}}

Occurrence and applications

[edit]

Confidence intervals for the difference in matched-pairs binary data (using multinomial withk=4)

[edit]

For the case of matched-pairs binary data, a common task is to build the confidence interval of the difference of the proportion of the matched events. For example, we might have a test for some disease, and we may want to check the results of it for some population at two points in time (1 and 2), to check if there was a change in the proportion of the positives for the disease during that time.

Such scenarios can be represented using a two-by-twocontingency table with the number of elements that had each of the combination of events. We can use smallf for sampling frequencies:f11,f10,f01,f00{\displaystyle f_{11},f_{10},f_{01},f_{00}}, and capitalF for population frequencies:F11,F10,F01,F00{\displaystyle F_{11},F_{10},F_{01},F_{00}}. These four combinations could be modeled as coming from a multinomial distribution (with four potential outcomes). The sizes of the sample and population can ben andN respectively. And in such a case, there is an interest in building a confidence interval for the difference of proportions from the marginals of the following (sampled) contingency table:

Test 2 positiveTest 2 negativeRow total
Test 1 positivef11{\displaystyle f_{11}}f10{\displaystyle f_{10}}f1∗=f11+f10{\displaystyle f_{1*}=f_{11}+f_{10}}
Test 1 negativef01{\displaystyle f_{01}}f00{\displaystyle f_{00}}f0∗=f01+f00{\displaystyle f_{0*}=f_{01}+f_{00}}
Column totalf∗1=f11+f01{\displaystyle f_{*1}=f_{11}+f_{01}}f∗0=f10+f00{\displaystyle f_{*0}=f_{10}+f_{00}}n{\displaystyle n}

In this case, checking the difference in marginal proportions means we are interested in using the following definitions:p1∗=F1∗N=F11+F10N{\displaystyle p_{1*}={\frac {F_{1*}}{N}}={\frac {F_{11}+F_{10}}{N}}},p∗1=F∗1N=F11+F01N{\displaystyle p_{*1}={\frac {F_{*1}}{N}}={\frac {F_{11}+F_{01}}{N}}}.And the difference we want to build confidence intervals for is:

p∗1−p1∗=F11+F01N−F11+F10N=F01N−F10N=p01−p10{\displaystyle p_{*1}-p_{1*}={\frac {F_{11}+F_{01}}{N}}-{\frac {F_{11}+F_{10}}{N}}={\frac {F_{01}}{N}}-{\frac {F_{10}}{N}}=p_{01}-p_{10}}

Hence, a confidence intervals for the marginal positive proportions (p∗1−p1∗{\displaystyle p_{*1}-p_{1*}}) is the same as building a confidence interval for the difference of the proportions from the secondary diagonal of the two-by-two contingency table (p01−p10{\displaystyle p_{01}-p_{10}}).

Calculating ap-value for such a difference is known asMcNemar's test. Building confidence interval around it can be constructed using methods described above forConfidence intervals for the difference of two proportions.

The Wald confidence intervals from the previous section can be applied to this setting, and appears in the literature using alternative notations. Specifically, the SE often presented is based on the contingency table frequencies instead of the sample proportions. For example, the Wald confidence intervals, provided above, can be written as:[10]: 102–103 

SE^(p∗1−p1∗)=SE^(p01−p10)=n(f10+f01)−(f10−f01)2nn{\displaystyle {\begin{aligned}{\widehat {\operatorname {SE} }}(p_{*1}-p_{1*})&={\widehat {\operatorname {SE} }}(p_{01}-p_{10})\\[1ex]&={\frac {\sqrt {n\left(f_{10}+f_{01}\right)-\left(f_{10}-f_{01}\right)^{2}}}{n{\sqrt {n}}}}\end{aligned}}}

Further research in the literature has identified several shortcomings in both the Wald and the Wald with continuity correction methods, and other methods have been proposed for practical application.[10]

One such modification includesAgresti and Min’s Wald+2 (similar to some of their other works[12]) in which each cell frequency had an extra12{\displaystyle {\frac {1}{2}}} added to it.[11] This leads to theWald+2 confidence intervals. In a Bayesian interpretation, this is like building the estimators taking as prior adirichlet distribution with all parameters being equal to 0.5 (which is, in fact, theJeffreys prior). The+2 in the namewald+2 can now be taken to mean that in the context of a two-by-two contingency table, which is a multinomial distribution with four possible events, then since we add 1/2 an observation to each of them, then this translates to an overall addition of 2 observations (due to the prior).

This leads to the following modified SE for the case of matched pairs data:

SE^(p∗1−p1∗)=(n+2)(f10+f01+1)−(f10−f01)2(n+2)n+2{\displaystyle {\widehat {\operatorname {SE} }}(p_{*1}-p_{1*})={\frac {\sqrt {\left(n+2\right)\left(f_{10}+f_{01}+1\right)-\left(f_{10}-f_{01}\right)^{2}}}{\left(n+2\right){\sqrt {n+2}}}}}

Which can just be plugged into the original Wald formula as follows:

(p∗1−p1∗)nn+2±zα/2⋅SE^(p^i−p^j)wald+2{\displaystyle \left(p_{*1}-p_{1*}\right){\frac {n}{n+2}}\pm z_{\alpha /2}\cdot {\widehat {\operatorname {SE} }}({\hat {p}}_{i}-{\hat {p}}_{j})_{wald+2}}

Other modifications includeBonett and Price’s Adjusted Wald, andNewcombe’s Score.

Computational methods

[edit]

Random variate generation

[edit]
Further information:Non-uniform random variate generation

First, reorder the parametersp1,…,pk{\displaystyle p_{1},\ldots ,p_{k}} such that they are sorted in descending order (this is only to speed up computation and not strictly necessary). Now, for each trial, draw an auxiliary variableX from a uniform (0, 1) distribution. The resulting outcome is the component

j=min{j′∈{1,…,k}:(∑i=1j′pi)−X≥0}.{\displaystyle j=\min \left\{j'\in \{1,\dots ,k\}\colon \left(\sum _{i=1}^{j'}p_{i}\right)-X\geq 0\right\}.}

{Xj = 1,Xk = 0 fork ≠ j } is one observation from the multinomial distribution withp1,…,pk{\displaystyle p_{1},\ldots ,p_{k}} andn = 1. A sum of independent repetitions of this experiment is an observation from a multinomial distribution withn equal to the number of such repetitions.

Sampling using repeated conditional binomial samples

[edit]

Given the parametersp1,p2,…,pk{\displaystyle p_{1},p_{2},\ldots ,p_{k}} and a total for the samplen{\displaystyle n} such that∑i=1kXi=n{\textstyle \sum _{i=1}^{k}X_{i}=n}, it is possible to sample sequentially for the number in an arbitrary stateXi{\displaystyle X_{i}}, by partitioning the state space intoi{\displaystyle i} and not-i{\displaystyle i}, conditioned on any prior samples already taken, repeatedly.

Algorithm: Sequential conditional binomial sampling

[edit]
S=nrho=1foriin[1,k-1]:ifrho!=0:X[i]~Binom(S,p[i]/rho)elseX[i]=0S=S-X[i]rho=rho-p[i]X[k]=S

Heuristically, each application of the binomial sample reduces the available number to sample from and the conditional probabilities are likewise updated to ensure logical consistency.[13]

Software implementations

[edit]
  • TheMultinomialCI R package allows the computation of simultaneous confidence intervals for the probabilities of a multinomial distribution given a set of observations.[14]

See also

[edit]

References

[edit]
  1. ^"probability - multinomial distribution sampling".Cross Validated. Retrieved2022-07-28.
  2. ^Loukas, Orestis; Chung, Ho Ryun (April 2022). "Categorical Distributions of Maximum Entropy under Marginal Constraints".arXiv:2204.03406 [hep-th].
  3. ^Loukas, Orestis; Chung, Ho Ryun (June 2022). "Entropy-based Characterization of Modeling Constraints".arXiv:2206.14105 [stat.ME].
  4. ^Wellek, Stefan (2010).Testing statistical hypotheses of equivalence and noninferiority. Chapman and Hall/CRC.ISBN 978-1439808184.
  5. ^Ostrovski, Vladimir (May 2017). "Testing equivalence of multinomial distributions".Statistics & Probability Letters.124:77–82.doi:10.1016/j.spl.2017.01.004.S2CID 126293429.Official web link (subscription required).Alternate, free web link.
  6. ^Frey, Jesse (March 2009). "An exact multinomial test for equivalence".The Canadian Journal of Statistics.37:47–59.doi:10.1002/cjs.10000.S2CID 122486567.Official web link (subscription required).
  7. ^Ostrovski, Vladimir (March 2018). "Testing equivalence to families of multinomial distributions with application to the independence model".Statistics & Probability Letters.139:61–66.doi:10.1016/j.spl.2018.03.014.S2CID 126261081.Official web link (subscription required).Alternate, free web link.
  8. ^Fleiss, Joseph L.; Levin, Bruce; Paik, Myunghee Cho (2003).Statistical Methods for Rates and Proportions (3rd ed.). Hoboken, N.J: J. Wiley. p. 760.ISBN 9780471526292.
  9. ^Newcombe, R. G. (1998). "Interval Estimation for the Difference Between Independent Proportions: Comparison of Eleven Methods".Statistics in Medicine.17 (8):873–890.doi:10.1002/(SICI)1097-0258(19980430)17:8<873::AID-SIM779>3.0.CO;2-I.PMID 9595617.
  10. ^abc"Confidence Intervals for the Difference Between Two Correlated Proportions"(PDF). NCSS. Retrieved2022-03-22.
  11. ^abAgresti, Alan; Min, Yongyi (2005)."Simple improved confidence intervals for comparing matched proportions"(PDF).Statistics in Medicine.24 (5):729–740.doi:10.1002/sim.1781.PMID 15696504.
  12. ^Agresti, A.; Caffo, B. (2000). "Simple and effective confidence intervals for proportions and difference of proportions result from adding two successes and two failures".The American Statistician.54 (4):280–288.doi:10.1080/00031305.2000.10474560.
  13. ^"11.5: The Multinomial Distribution".Statistics LibreTexts. 2020-05-05. Retrieved2023-09-13.
  14. ^"MultinomialCI - Confidence Intervals for Multinomial Proportions". CRAN. 11 May 2021. Retrieved2024-03-23.

Further reading

[edit]
Discrete
univariate
with finite
support
with infinite
support
Continuous
univariate
supported on a
bounded interval
supported on a
semi-infinite
interval
supported
on the whole
real line
with support
whose type varies
Mixed
univariate
continuous-
discrete
Multivariate
(joint)
Directional
Degenerate
andsingular
Degenerate
Dirac delta function
Singular
Cantor
Families
Authority control databasesEdit this at Wikidata
Retrieved from "https://en.wikipedia.org/w/index.php?title=Multinomial_distribution&oldid=1323018463"
Categories:
Hidden categories:
[8]ページ先頭
©2009-2025 Movatter.jp