Inprobability theory andcomputer science, alog probability is simply alogarithm of aprobability.^[1] The use of log probabilities means representing probabilities on alogarithmic scale${\displaystyle (-\mathrm{\infty },0]}$, instead of the standard${\displaystyle [0,1]}$unit interval.

Since the probabilities ofindependentevents multiply, and logarithms convert multiplication to addition, log probabilities of independent events add. Log probabilities are thus practical for computations, and have an intuitive interpretation in terms ofinformation theory: the negativeexpected value of the log probabilities is theinformation entropy of an event. Similarly,likelihoods are often transformed to the log scale, and the correspondinglog-likelihood can be interpreted as the degree to which an event supports astatistical model. The log probability is widely used in implementations of computations with probability, and is studied as a concept in its own right in some applications of information theory, such asnatural language processing.

Representing probabilities in this way has several practical advantages:

1. Speed. Since multiplication is moreexpensive than addition, taking the product of a high number of probabilities is often faster if they are represented in log form. (The conversion to log form is expensive, but is only incurred once.) Multiplication arises from calculating the probability that multiple independent events occur: the probability that all independent events of interest occur is the product of all these events' probabilities.
2. Accuracy. The use of log probabilities improvesnumerical stability, when the probabilities are very small, because of the way in which computersapproximate real numbers.^[1]
3. Simplicity. Manyprobability distributions have an exponential form. Taking the log of these distributions eliminates the exponential function, unwrapping the exponent. For example, the log probability of the normal distribution'sprobability density function is${\displaystyle -((x-m_x)/{\sigma }_m{)}^2+C}$ instead of${\displaystyle C_2\exp \left(-((x-m_x)/{\sigma }_m{)}^2\right)}$. Log probabilities make some mathematical manipulations easier to perform.
4. Optimization. Since most commonprobability distributions—notably theexponential family—are onlylogarithmically concave,^[2][3] andconcavity of theobjective function plays a key role in themaximization of a function such as probability, optimizers work better with log probabilities.

The logarithm function is not defined for zero, so log probabilities can only represent non-zero probabilities. Since the logarithm of a number in${\displaystyle (0,1)}$ interval is negative, often the negative log probabilities are used. In that case the log probabilities in the following formulas would beinverted.

Any base can be selected for the logarithm.

In this section we would name probabilities in logarithmic space${\displaystyle x^{\prime }}$ and${\displaystyle y^{\prime }}$ for short:

${\displaystyle x^{\prime }=\log (x)\in \mathbb{R}}$

${\displaystyle y^{\prime }=\log (y)\in \mathbb{R}}$

The product of probabilities${\displaystyle x\cdot y}$ corresponds to addition in logarithmic space.

${\displaystyle \log (x\cdot y)=\log (x)+\log (y)=x^{\prime }+y^{\prime }.}$

Thesum of probabilities${\displaystyle x+y}$ is a bit more involved to compute in logarithmic space, requiring the computation of one exponent and one logarithm.

However, in many applications a multiplication of probabilities (giving the probability of all independent events occurring) is used more often than their addition (giving the probability of at least one of mutually exclusive events occurring). Additionally, the cost of computing the addition can be avoided in some situations by simply using the highest probability as an approximation. Since probabilities are non-negative this gives a lower bound. This approximation is used in reverse to get acontinuous approximation of the max function.

The formula above is more accurate than${\displaystyle \log \left(e^{x^{\prime }}+e^{y^{\prime }}\right)}$, provided one takes advantage of the asymmetry in the addition formula.${\displaystyle x^{\prime }}$ should be the larger (least negative) of the two operands. This also produces the correct behavior if one of the operands isfloating-pointnegative infinity, which corresponds to a probability of zero.

${\displaystyle -\mathrm{\infty }+\log \left(1+\exp \left(y^{\prime }-(-\mathrm{\infty })\right)\right)=-\mathrm{\infty }+\mathrm{\infty }}$ This quantity isindeterminate, and will result inNaN.

${\displaystyle x^{\prime }+\log \left(1+\exp \left(-\mathrm{\infty }-x^{\prime }\right)\right)=x^{\prime }+0}$ This is the desired answer.

The above formula alone will incorrectly produce an indeterminate result in the case where both arguments are${\displaystyle -\mathrm{\infty }}$. This should be checked for separately to return${\displaystyle -\mathrm{\infty }}$.

For numerical reasons, one should use a function that computes${\displaystyle \log (1+x)}$ (log1p) directly.

• Information content
• Log-likelihood

• Logarithms
• Mathematics of computing

• Articles with short description
• Short description matches Wikidata