Movatterモバイル変換

Jump to content

Joint probability distribution

From Wikipedia, the free encyclopedia

(Redirected fromJoint distribution function)

Type of probability distribution

X {\displaystyle X}

Y {\displaystyle Y}

p(X)

p(Y)

Many sample observations (black) are shown from a joint probability distribution. The marginal densities are shown as well (in blue and in red).

Probability theory
Part of a series onstatistics

Probability Axioms Determinism System Indeterminism Randomness
Probability space Sample space Event Collectively exhaustive events Elementary event Mutual exclusivity Outcome Singleton Experiment Bernoulli trial Probability distribution Bernoulli distribution Binomial distribution Exponential distribution Normal distribution Pareto distribution Poisson distribution Probability measure Random variable Bernoulli process Continuous or discrete Expected value Variance Markov chain Observed value Random walk Stochastic process
Complementary event Joint probability Marginal probability Conditional probability
Independence Conditional independence Law of total probability Law of large numbers Bayes' theorem Boole's inequality
Venn diagram Tree diagram
v t e

Givenrandom variables $X,Y,\ldots$ , that are defined on the same^[1]probability space, themultivariate orjoint probability distribution for $X,Y,\ldots$ is aprobability distribution that gives the probability that each of $X,Y,\ldots$ falls in any particular range or discrete set of values specified for that variable. In the case of only two random variables, this is called abivariate distribution, but the concept generalizes to any number of random variables.

The joint probability distribution can be expressed in terms of a jointcumulative distribution function and either in terms of a jointprobability density function (in the case ofcontinuous variables) or jointprobability mass function (in the case ofdiscrete variables). These in turn can be used to find two other types of distributions: themarginal distribution giving the probabilities for any one of the variables with no reference to any specific ranges of values for the other variables, and theconditional probability distribution giving the probabilities for any subset of the variables conditional on particular values of the remaining variables.

Examples

Draws from an urn

Each of two urns contains twice as many red balls as blue balls, and no others, and one ball is randomly selected from each urn, with the two draws independent of each other. Let $A {\displaystyle A}$ and $B {\displaystyle B}$ be discrete random variables associated with the outcomes of the draw from the first urn and second urn respectively. The probability of drawing a red ball from either of the urns is⁠2/3⁠, and the probability of drawing a blue ball is⁠1/3⁠. The joint probability distribution is presented in the following table:

	A=Red	A=Blue	P(B)
B=Red	(⁠2/3⁠)(⁠2/3⁠) =⁠4/9⁠	(⁠1/3⁠)(⁠2/3⁠) =⁠2/9⁠	⁠4/9⁠ +⁠2/9⁠ =⁠2/3⁠
B=Blue	(⁠2/3⁠)(⁠1/3⁠) =⁠2/9⁠	(⁠1/3⁠)(⁠1/3⁠) =⁠1/9⁠	⁠2/9⁠ +⁠1/9⁠ =⁠1/3⁠
P(A)	⁠4/9⁠ +⁠2/9⁠ =⁠2/3⁠	⁠2/9⁠ +⁠1/9⁠ =⁠1/3⁠

Each of the four inner cells shows the probability of a particular combination of results from the two draws; these probabilities are the joint distribution. In any one cell the probability of a particular combination occurring is (since the draws are independent) the product of the probability of the specified result for A and the probability of the specified result for B. The probabilities in these four cells sum to 1, as with all probability distributions.

Moreover, the final row and the final column give themarginal probability distribution for A and the marginal probability distribution for B respectively. For example, for A the first of these cells gives the sum of the probabilities for A being red, regardless of which possibility for B in the column above the cell occurs, as⁠2/3⁠. Thus the marginal probability distribution for $A {\displaystyle A}$ gives $A {\displaystyle A}$ 's probabilitiesunconditional on $B {\displaystyle B}$ , in a margin of the table.

Coin flips

Consider the flip of twofair coins; let $A {\displaystyle A}$ and $B {\displaystyle B}$ be discrete random variables associated with the outcomes of the first and second coin flips respectively. Each coin flip is aBernoulli trial and has aBernoulli distribution. If a coin displays "heads" then the associated random variable takes the value 1, and it takes the value 0 otherwise. The probability of each of these outcomes is⁠1/2⁠, so the marginal (unconditional) density functions are

${\begin{aligned}P(A)&=1/2\quad {\text{for}}\quad A\in \{0,1\};\\P(B)&=1/2\quad {\text{for}}\quad B\in \{0,1\}.\end{aligned}}$

The joint probability mass function of $A {\displaystyle A}$ and $B {\displaystyle B}$ defines probabilities for each pair of outcomes. All possible outcomes are $(A,B)\in \{(0,0),(0,1),(1,0),(1,1)\}.$ Since each outcome is equally likely the joint probability mass function becomes $P(A,B)=1/4\quad {\text{for}}\quad A,B\in \{0,1\}.$

Since the coin flips are independent, the joint probability mass function is the product of the marginals: $P(A,B)=P(A)P(B)\quad {\text{for}}\quad A,B\in \{0,1\}.$

Rolling a die

Consider the roll of a fairdie and let $A=1$ if the number is even (i.e. 2, 4, or 6) and $A=0$ otherwise. Furthermore, let $B=1$ if the number is prime (i.e. 2, 3, or 5) and $B=0$ otherwise.

	1	2	3	4	5	6
A	0	1	0	1	0	1
B	0	1	1	0	1	0

Then, the joint distribution of $A {\displaystyle A}$ and $B {\displaystyle B}$ , expressed as a probability mass function, is ${\begin{aligned}\mathrm {P} (A=0,B=0)&=P\{1\}={\frac {1}{6}},&\mathrm {P} (A=1,B=0)&=P\{4,6\}={\frac {2}{6}},\\\mathrm {P} (A=0,B=1)&=P\{3,5\}={\frac {2}{6}},&\mathrm {P} (A=1,B=1)&=P\{2\}={\frac {1}{6}}.\end{aligned}}$

These probabilities necessarily sum to 1, since the probability ofsome combination of $A {\displaystyle A}$ and $B {\displaystyle B}$ occurring is 1.

Marginal probability distribution

Main article:Marginal distribution

If more than one random variable is defined in a random experiment, it is important to distinguish between the joint probability distribution of X and Y and the probability distribution of each variable individually. The individual probability distribution of a random variable is referred to as its marginal probability distribution. In general, the marginal probability distribution of X can be determined from the joint probability distribution of X and other random variables.

If the joint probability density function of random variable X and Y is $f_{X,Y}(x,y)$ , the marginal probability density function of X and Y, which defines themarginal distribution, is given by:

${\begin{aligned}f_{X}(x)&=\int f_{X,Y}(x,y)\;dy\\f_{Y}(y)&=\int f_{X,Y}(x,y)\;dx\end{aligned}}$

where the first integral is over all points in the range of (X,Y) for which X=x and the second integral is over all points in the range of (X,Y) for which Y=y.^[2]

Joint cumulative distribution function

For a pair of random variables $X, Y {\displaystyle X,Y}$ , the joint cumulative distribution function (CDF) $F_{X,Y}$ is given by^[3]^: 89

$F_{X,Y}(x,y)=\operatorname {P} (X\leq x,Y\leq y)$ (Eq.1)

where the right-hand side represents theprobability that the random variable $X {\displaystyle X}$ takes on a value less than or equal to $x {\displaystyle x}$ and that $Y {\displaystyle Y}$ takes on a value less than or equal to $y {\displaystyle y}$ .

For $N {\displaystyle N}$ random variables $X_{1},\ldots ,X_{N}$ , the joint CDF $F_{X_{1},\ldots ,X_{N}}$ is given by

$F_{X_{1},\ldots ,X_{N}}(x_{1},\ldots ,x_{N})=\operatorname {P} (X_{1}\leq x_{1},\ldots ,X_{N}\leq x_{N})$ (Eq.2)

Interpreting the $N {\displaystyle N}$ random variables as arandom vector $\mathbf {X} =(X_{1},\ldots ,X_{N})^{T}$ yields a shorter notation:

$F_{\mathbf {X} }(\mathbf {x} )=\operatorname {P} (X_{1}\leq x_{1},\ldots ,X_{N}\leq x_{N})$

Joint density function or mass function

Discrete case

The jointprobability mass function of twodiscrete random variables $X, Y {\displaystyle X,Y}$ is:

$p_{X,Y}(x,y)=\mathrm {P} (X=x\ \mathrm {and} \ Y=y)$ (Eq.3)

or written in terms of conditional distributions $p_{X,Y}(x,y)=\mathrm {P} (Y=y\mid X=x)\cdot \mathrm {P} (X=x)=\mathrm {P} (X=x\mid Y=y)\cdot \mathrm {P} (Y=y)$ where $\mathrm {P} (Y=y\mid X=x)$ is theprobability of $Y=y$ given that $X=x$ .

The generalization of the preceding two-variable case is the joint probability distribution of $n {\displaystyle n}$ discrete random variables $X_{1},X_{2},\dots ,X_{n}$ which is:

$p_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=\mathrm {P} (X_{1}=x_{1}{\text{ and }}\dots {\text{ and }}X_{n}=x_{n})$ (Eq.4)

or equivalently

${\begin{aligned}p_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})={}&\mathrm {P} (X_{1}=x_{1})\\&\cdot \mathrm {P} (X_{2}=x_{2}\mid X_{1}=x_{1})\\&\cdot \mathrm {P} (X_{3}=x_{3}\mid X_{1}=x_{1},X_{2}=x_{2})\\&\cdots \\&\cdot \mathrm {P} (X_{n}=x_{n}\mid X_{1}=x_{1},X_{2}=x_{2},\dots ,X_{n-1}=x_{n-1}).\end{aligned}}$

This identity is known as thechain rule of probability.

Since these are probabilities, in the two-variable case

$\sum _{i}\sum _{j}\mathrm {P} (X=x_{i}\ \mathrm {and} \ Y=y_{j})=1,\,$ which generalizes for $n\,$ discrete random variables $X_{1},X_{2},\dots ,X_{n}$ to

$\sum _{i}\sum _{j}\dots \sum _{k}\mathrm {P} (X_{1}=x_{1i},X_{2}=x_{2j},\dots ,X_{n}=x_{nk})=1.\;$

Continuous case

Thejointprobability density function $f_{X,Y}(x,y)$ for twocontinuous random variables is defined as the derivative of the joint cumulative distribution function (seeEq.1):

$f_{X,Y}(x,y)={\frac {\partial ^{2}F_{X,Y}(x,y)}{\partial x\partial y}}$ (Eq.5)

This is equal to: $f_{X,Y}(x,y)=f_{Y\mid X}(y\mid x)f_{X}(x)=f_{X\mid Y}(x\mid y)f_{Y}(y)$

where $f_{Y\mid X}(y\mid x)$ and $f_{X\mid Y}(x\mid y)$ are theconditional distributions of $Y {\displaystyle Y}$ given $X=x$ and of $X {\displaystyle X}$ given $Y=y$ respectively, and $f_{X}(x)$ and $f_{Y}(y)$ are themarginal distributions for $X {\displaystyle X}$ and $Y {\displaystyle Y}$ respectively.

The definition extends naturally to more than two random variables:

$f_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})={\frac {\partial ^{n}F_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})}{\partial x_{1}\ldots \partial x_{n}}}$ (Eq.6)

Again, since these are probability distributions, one has $\int _{x}\int _{y}f_{X,Y}(x,y)\;dy\;dx=1$ respectively $\int _{x_{1}}\ldots \int _{x_{n}}f_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})\;dx_{n}\ldots \;dx_{1}=1$

Mixed case

The "mixed joint density" may be defined where one or more random variables are continuous and the other random variables are discrete. With one variable of each type $f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)\mathrm {P} (Y=y)=\mathrm {P} (Y=y\mid X=x)f_{X}(x).$ One example of a situation in which one may wish to find the cumulative distribution of one random variable which is continuous and another random variable which is discrete arises when one wishes to use alogistic regression in predicting the probability of a binary outcome Y conditional on the value of a continuously distributed outcome $X {\displaystyle X}$ . Onemust use the "mixed" joint density when finding the cumulative distribution of this binary outcome because the input variables $(X,Y)$ were initially defined in such a way that one could not collectively assign it either a probability density function or a probability mass function. Formally, $f_{X,Y}(x,y)$ is the probability density function of $(X,Y)$ with respect to theproduct measure on the respectivesupports of $X {\displaystyle X}$ and $Y {\displaystyle Y}$ . Either of these two decompositions can then be used to recover the joint cumulative distribution function: $F_{X,Y}(x,y)=\sum _{t\leq y}\int _{-\infty }^{x}f_{X,Y}(s,t)\;ds.$ The definition generalizes to a mixture of arbitrary numbers of discrete and continuous random variables.

Additional properties

Joint distribution for independent variables

In general two random variables $X {\displaystyle X}$ and $Y {\displaystyle Y}$ areindependent if and only if the joint cumulative distribution function satisfies $F_{X,Y}(x,y)=F_{X}(x)\cdot F_{Y}(y)$

Two discrete random variables $X {\displaystyle X}$ and $Y {\displaystyle Y}$ are independent if and only if the joint probability mass function satisfies $P(X=x\ {\text{and}}\ Y=y)=P(X=x)\cdot P(Y=y)$ for all $x {\displaystyle x}$ and $y {\displaystyle y}$ .

While the number of independent random events grows, the related joint probability value decreases rapidly to zero, according to a negative exponential law.

Similarly, two absolutely continuous random variables are independent if and only if $f_{X,Y}(x,y)=f_{X}(x)\cdot f_{Y}(y)$ for all $x {\displaystyle x}$ and $y {\displaystyle y}$ . This means that acquiring any information about the value of one or more of the random variables leads to a conditional distribution of any other variable that is identical to its unconditional (marginal) distribution; thus no variable provides any information about any other variable.

Joint distribution for conditionally dependent variables

If a subset $A {\displaystyle A}$ of the variables $X_{1},\cdots ,X_{n}$ isconditionally dependent given another subset $B {\displaystyle B}$ of these variables, then the probability mass function of the joint distribution is $\mathrm {P} (X_{1},\ldots ,X_{n})$ . $\mathrm {P} (X_{1},\ldots ,X_{n})$ is equal to $P(B)\cdot P(A\mid B)$ . Therefore, it can be efficiently represented by the lower-dimensional probability distributions $P(B)$ and $P(A\mid B)$ . Such conditional independence relations can be represented with aBayesian network orcopula functions.

Covariance

Main article:Covariance

When two or more random variables are defined on a probability space, it is useful to describe how they vary together; that is, it is useful to measure the relationship between the variables. A common measure of the relationship between two random variables is the covariance. Covariance is a measure of linear relationship between the random variables. If the relationship between the random variables is nonlinear, the covariance might not be sensitive to the relationship, which means, it does not relate the correlation between two variables.

The covariance between the random variables $X {\displaystyle X}$ and $Y {\displaystyle Y}$ is^[2] $\operatorname {cov} (X,Y)=\sigma _{XY}=\operatorname {E} \left[(X-\mu _{x})(Y-\mu _{y})\right]=\operatorname {E} (XY)-\mu _{x}\mu _{y}.$

Correlation

Main article:Correlation

There is another measure of the relationship between two random variables that is often easier to interpret than the covariance.

The correlation just scales the covariance by the product of the standard deviation of each variable. Consequently, the correlation is a dimensionless quantity that can be used to compare the linear relationships between pairs of variables in different units. If the points in the joint probability distribution of X and Y that receive positive probability tend to fall along a line of positive (or negative) slope, ρ_XY is near +1 (or −1). If ρ_XY equals +1 or −1, it can be shown that the points in the joint probability distribution that receive positive probability fall exactly along a straight line. Two random variables with nonzero correlation are said to be correlated. Similar to covariance, the correlation is a measure of the linear relationship between random variables.

The correlation coefficient between the random variables $X {\displaystyle X}$ and $Y {\displaystyle Y}$ is $\rho _{XY}={\frac {\operatorname {cov} (X,Y)}{\sqrt {V(X)V(Y)}}}={\frac {\sigma _{XY}}{\sigma _{X}\sigma _{Y}}}.$

Important named distributions

Named joint distributions that arise frequently in statistics include themultivariate normal distribution, themultivariate stable distribution, themultinomial distribution, thenegative multinomial distribution, themultivariate hypergeometric distribution, and theelliptical distribution.

See also

References

^Feller, William (1968).An Introduction to Probability Theory and its Applications. Vol. 1 (3rd ed.). pp. 217–218.ISBN 978-0471257080.
^^a ^bMontgomery, Douglas C.; Runger, George C. (19 November 2013).Applied Statistics and Probability for Engineers (Sixth ed.). Hoboken, NJ:Wiley.ISBN 978-1-118-53971-2.OCLC 861273897.
^Park, Kun Il (2018).Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer.ISBN 978-3-319-68074-3.

External links

"Joint distribution",Encyclopedia of Mathematics,EMS Press, 2001 [1994]
"Multi-dimensional distribution",Encyclopedia of Mathematics,EMS Press, 2001 [1994]
A modern introduction to probability and statistics : understanding why and how. Dekking, Michel, 1946-. London: Springer. 2005.ISBN 978-1-85233-896-1.OCLC 262680588.
"Joint continuous density function".PlanetMath.
Mathworld: Joint Distribution Function

v
t
e

Probability distributions (list)

Discrete
univariate

with finite support	Benford Bernoulli Beta-binomial Binomial Categorical Hypergeometric Negative Poisson binomial Rademacher Soliton Discrete uniform Zipf Zipf–Mandelbrot
with infinite support	Beta negative binomial Borel Conway–Maxwell–Poisson Discrete phase-type Delaporte Extended negative binomial Flory–Schulz Gauss–Kuzmin Geometric Logarithmic Mixed Poisson Negative binomial Panjer Parabolic fractal Poisson Skellam Yule–Simon Zeta

Continuous
univariate

supported on a bounded interval	Arcsine ARGUS Balding–Nichols Bates Beta Generalized Beta rectangular Continuous Bernoulli Irwin–Hall Kumaraswamy Logit-normal Noncentral beta PERT Power function Raised cosine Reciprocal Triangular U-quadratic Uniform Wigner semicircle
supported on a semi-infinite interval	Benini Benktander 1st kind Benktander 2nd kind Beta prime Burr Chi Chi-squared Noncentral Inverse Scaled Dagum Davis Erlang Hyper Exponential Hyperexponential Hypoexponential Logarithmic F Noncentral Folded normal Fréchet Gamma Generalized Inverse gamma/Gompertz Gompertz Shifted Half-logistic Half-normal Hotelling'sT-squared Hartman–Watson Inverse Gaussian Generalized Kolmogorov Lévy Log-Cauchy Log-Laplace Log-logistic Log-normal Log-t Lomax Matrix-exponential Maxwell–Boltzmann Maxwell–Jüttner Mittag-Leffler Nakagami Pareto Phase-type Poly-Weibull Rayleigh Relativistic Breit–Wigner Rice Truncated normal type-2 Gumbel Weibull Discrete Wilks's lambda
supported on the whole real line	Cauchy Exponential power Fisher'sz Kaniadakis κ-Gaussian Gaussianq Generalized hyperbolic Generalized logistic (logistic-beta) Generalized normal Geometric stable Gumbel Holtsmark Hyperbolic secant Johnson'sS_U Landau Laplace Asymmetric Logistic Noncentralt Normal (Gaussian) Normal-inverse Gaussian Skew normal Slash Stable Student'st Tracy–Widom Variance-gamma Voigt
with support whose type varies	Generalized chi-squared Generalized extreme value Generalized Pareto Marchenko–Pastur Kaniadakisκ-exponential Kaniadakisκ-Gamma Kaniadakisκ-Weibull Kaniadakisκ-Logistic Kaniadakisκ-Erlang q-exponential q-Gaussian q-Weibull Shifted log-logistic Tukey lambda

Mixed
univariate

continuous- discrete	Rectified Gaussian

Multivariate
(joint)

Discrete:
Ewens
Multinomial
- Dirichlet
- Negative
Continuous:
Dirichlet
- Generalized
Multivariate Laplace
Multivariate normal
Multivariate stable
Multivariatet
Normal-gamma
- Inverse
Matrix-valued:
LKJ
Matrix beta
Matrix normal
Matrixt
Matrix gamma
- Inverse
Wishart
Uniform distribution on a Stiefel manifold

Univariate (circular)directional: Circular uniform; Univariate von Mises; Wrapped normal; Wrapped Cauchy; Wrapped exponential; Wrapped asymmetric Laplace; Wrapped Lévy
Bivariate (spherical): Kent
Bivariate (toroidal): Bivariate von Mises
Multivariate: von Mises–Fisher; Bingham

Degenerate
andsingular

Degenerate: Dirac delta function
Singular: Cantor

Families

Retrieved from "https://en.wikipedia.org/w/index.php?title=Joint_probability_distribution&oldid=1323298240"

Hidden categories:

[8]ページ先頭

©2009-2025 Movatter.jp