Instatistical classification,Bayes error rate is the lowest possible error rate for any classifier of a random outcome (into, for example, one of two categories) and is analogous to the irreducible error.^[1][2]

A number of approaches to the estimation of the Bayes error rate exist. One method seeks to obtain analytical bounds which are inherently dependent on distribution parameters, and hence difficult to estimate. Another approach focuses on class densities, while yet another method combines and compares various classifiers.^[2]

The Bayes error rate finds important use in the study of patterns andmachine learning techniques.^[3]

Mohri, Rostamizadeh and Talwalkar define it as

Given a distribution${\displaystyle \mathcal{D}}$ over${\displaystyle \mathcal{X}\times \mathcal{Y}}$, the Bayes error${\displaystyle R^{\ast }}$ is defined as the infimum of the errors achieved by measurable functions${\displaystyle h:\mathcal{X}\to \mathcal{Y}}$:

$${\displaystyle R^{\ast }=\underset{h:h\text{ measurable}}{inf}R(h)}$$

A hypothesis h with R(h) = R^* is called a Bayes hypothesis orBayes classifier.

In terms of machine learning and pattern classification, the labels of a set of random observations can be divided into 2 or more classes. Each observation is called aninstance and the class it belongs to is thelabel.The Bayes error rate of the data distribution is the probability an instance is misclassified by a classifier that knows the true class probabilities given the predictors.

For amulticlass classifier, the expected prediction error may be calculated as follows:^[3]

$${\displaystyle EPE=E_x[\sum _{k=1}^{K}L(C_k,\hat{C}(x))P(C_k|x)]}$$

where${\displaystyle x}$ is the instance,${\displaystyle E[]}$ the expectation value,${\displaystyle C_k}$ is a class into which an instance is classified,${\displaystyle P(C_k|x)}$ is theconditional probability of label${\displaystyle k}$ for instance${\displaystyle x}$, and${\displaystyle L()}$ is the 0–1loss function:

where${\displaystyle {\delta }_{x,y}}$ is theKronecker delta.

When the learner knows the conditional probability, then one solution is:

$${\displaystyle {\hat{C}}_B(x)=\arg \underset{k\in \{\mathrm{1...}K\}}{max}P(C_k|X=x)}$$

This solution is known as the Bayes classifier.

The corresponding expected Prediction Error is called the Bayes error rate:

$${\displaystyle BE=E_x[\sum _{k=1}^{K}L(C_k,{\hat{C}}_B(x))P(C_k|x)]=E_x[\sum _{k=1,C_k\ne {\hat{C}}_B(x)}^{K}P(C_k|x)]=E_x[1-P({\hat{C}}_B(x)|x)],}$$

where the sum can be omitted in the last step due to considering the counter event.By the definition of the Bayes classifier, it maximizes${\displaystyle P({\hat{C}}_B(x)|x)}$ and, therefore, minimizes the Bayes error BE.

The Bayes error is non-zero if the classification labels are not deterministic, i.e., there is a non-zero probability of a given instance belonging to more than one class.^[4] In a regression context with squared error, the Bayes error is equal to the noise variance.^[3]

Proof that the Bayes error rate is indeed the minimum possible and that the Bayes classifier is therefore optimal, may be found together on the Wikipedia pageBayes classifier.

A plug-in rule uses an estimate of the posterior probability${\displaystyle \eta }$ to form a classification rule. Given an estimate${\displaystyle \tilde{\eta }}$, the excess Bayes error rate of the associated classifier is bounded above by:

$${\displaystyle 2\mathbb{E}[|\eta (X)-\tilde{\eta }(X)|].}$$

To see this, note that the excess Bayes error is equal to 0 where the classifiers agree, and equal to${\displaystyle 2|\eta (X)-1/2|}$ where they disagree. To form the bound, notice that${\displaystyle \tilde{\eta }}$ is at least as far as${\displaystyle 1/2}$ when the classifiers disagree.

• Naive Bayes classifier

Thisstatistics-related article is astub. You can help Wikipedia byadding missing information.

• v
• t
• e

• Statistical classification
• Bayesian statistics
• Statistics stubs

• Articles with short description
• Short description is different from Wikidata
• All stub articles