Continuous Bernoulli distribution
Probability density function
Parameters	${\displaystyle \lambda =1/(1+e^{-\theta })\in (0,1)}$	${\displaystyle \theta \in \mathbb{R}}$,natural parameter
Support	${\displaystyle x\in [0,1]}$	${\displaystyle x\in [0,1]}$
PDF	${\displaystyle C(\lambda ){\lambda }^x(1-\lambda {)}^{1-x}}$ where
CDF
Mean
Variance

Inprobability theory,statistics, andmachine learning, thecontinuous Bernoulli distribution^[1][2][3] is a family of continuousprobability distributions parameterized by a singleshape parameter${\displaystyle \lambda \in (0,1)}$, defined on the unit interval${\displaystyle x\in [0,1]}$, by:

${\displaystyle p(x|\lambda )\propto {\lambda }^x(1-\lambda {)}^{1-x}.}$

The continuous Bernoulli distribution arises indeep learning andcomputer vision, specifically in the context ofvariational autoencoders,^[4][5] for modeling the pixel intensities of natural images. As such, it defines a proper probabilistic counterpart for the commonly used binarycross entropy loss, which is often applied to continuous,${\displaystyle [0,1]}$-valued data.^[6][7][8][9] This practice amounts to ignoring the normalizing constant of the continuous Bernoulli distribution, since the binary cross entropy loss only defines a true log-likelihood for discrete,${\displaystyle \{0,1\}}$-valued data.

The continuous Bernoulli also defines anexponential family of distributions. Writing${\displaystyle \theta =\log \left(\lambda /(1-\lambda )\right)}$ for thenatural parameter, the density can be rewritten in canonical form:${\displaystyle p(x|\theta )\propto \exp (\theta x)}$.^[10]

Given an independent sample of${\displaystyle n}$ points${\displaystyle x_1,\dots ,x_n}$ with${\displaystyle x_i\in [0,1]\mathrm{\forall }i}$ from continuous Bernoulli, the log-likelihood of the natural parameter${\displaystyle \theta }$ is

${\displaystyle \mathcal{L}(\theta )=\theta \sum _{i=1}^n{x}_i-n\log \{(e^{\theta }-1)/\theta \}}$

and themaximum likelihood estimator of the natural parameter${\displaystyle \theta }$ is the solution of${\displaystyle {\mathcal{L}}^{\prime }(\theta )=0}$, that is,${\displaystyle \hat{\theta }}$ satisfies

${\displaystyle \frac{e^{\hat{\theta }}}{e^{\hat{\theta }}-1}-\frac{1}{\hat{\theta }}=\frac{1}{n}\sum _{i=1}^n{x}_i}$

where the left hand side${\displaystyle e^{\hat{\theta }}/(e^{\hat{\theta }}-1)-{\hat{\theta }}^{-1}}$ is the expected value of continuous Bernoulli with parameter${\displaystyle \hat{\theta }}$. Although${\displaystyle \hat{\theta }}$ does not admit a closed-form expression, it can be easily calculated with numerical inversion.

The entropy of a continuous Bernoulli distribution is

The continuous Bernoulli can be thought of as a continuous relaxation of theBernoulli distribution, which is defined on the discrete set${\displaystyle \{0,1\}}$ by theprobability mass function:

${\displaystyle p(x)=p^x(1-p{)}^{1-x},}$

where${\displaystyle p}$ is a scalar parameter between 0 and 1. Applying this same functional form on the continuous interval${\displaystyle [0,1]}$ results in the continuous Bernoulliprobability density function, up to a normalizing constant.

TheUniform distribution between the unit interval [0,1] is a special case of continuous Bernoulli when${\displaystyle \lambda =1/2}$ or${\displaystyle \theta =0}$.

Anexponential distribution with rate${\displaystyle \mathrm{\Lambda }}$ restricted to the unit interval [0,1] corresponds to a continuous Bernoulli distribution with natural parameter.

The multivariate generalization of the continuous Bernoulli is called thecontinuous-categorical.^[11]

• Continuous distributions
• Exponential family distributions

• Webarchive template wayback links
• Articles with short description
• Short description matches Wikidata