Movatterモバイル変換

[0]ホーム

Jump to content

False positive rate

Edit links

From Wikipedia, the free encyclopedia

Chance of wrongly rejecting the null hypothesis

This articlerelies largely or entirely on asingle source. Relevant discussion may be found on thetalk page. Please helpimprove this article byintroducing citations to additional sources.
Find sources: "False positive rate" – news ·newspapers ·books ·scholar ·JSTOR(July 2016)

Instatistics, when performingmultiple comparisons, afalse positive ratio (also known asfall-out orfalse alarm rate^[1] ) is theprobability of falsely rejecting thenull hypothesis for a particulartest. The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events (regardless of classification).

The false positiverate (or "false alarm rate") usually refers to theexpectancy of the false positiveratio.

Definition

[edit]

The false positive rate (false alarm rate) is ${\boldsymbol {\mathrm {FPR} }}={\frac {\mathrm {FP} }{\mathrm {FP} +\mathrm {TN} }}$ ^[1]

where $\mathrm {FP}$ is the number of false positives, $\mathrm {TN}$ is the number of true negatives and $N=\mathrm {FP} +\mathrm {TN}$ is the total number of ground truth negatives.

Thesignificance level used to test each hypothesis is set based on the form of inference (simultaneous inference vs. selective inference) and its supporting criteria (for exampleFWER orFDR), that were pre-determined by the researcher.

When performingmultiple comparisons in astatistical framework such as above, thefalse positive ratio (also known as thefalse alarm rate, as opposed false alarmratio - FAR ) usually refers to the probability of falsely rejecting thenull hypothesis for a particulartest. Using the terminology suggested here, it is simply $V/m_{0}$ .

SinceV is arandom variable and $m_{0}$ is a constant ( $V\leq m_{0}$ ), the false positiveratio is also a random variable, ranging between 0–1.
Thefalse positive rate (or "false alarm rate") usually refers to theexpectancy of the false positive ratio, expressed by $E(V/m_{0})$ .

It is worth noticing that the two definitions ("false positive ratio" / "false positive rate") are somewhat interchangeable. For example, in the referenced article^[2] $V/m_{0}$ serves as the false positive "rate" rather than as its "ratio".

Classification of multiple hypothesis tests

[edit]

Main article:Classification of multiple hypothesis tests

The following table defines the possible outcomes when testing multiple null hypotheses. Suppose we have a numberm of null hypotheses, denoted by:H₁, H₂, ..., H_m.Using astatistical test, we reject the null hypothesis if the test is declared significant. We do not reject the null hypothesis if the test is non-significant.Summing each type of outcome over allH_i yields the following random variables:

	Null hypothesis is true (H₀)	Alternative hypothesis is true (H_A)	Total
Test is declared significant	V	S	R
Test is declared non-significant	U	T	$m-R$
Total	$m_{0}$	$m-m_{0}$	m

m is the total number hypotheses tested
$m_{0}$ is the number of truenull hypotheses, an unknown parameter
$m-m_{0}$ is the number of truealternative hypotheses
V is the number offalse positives (Type I error) (also called "false discoveries")
S is the number oftrue positives (also called "true discoveries")
T is the number offalse negatives (Type II error)
U is the number oftrue negatives
$R=V+S$ is the number of rejected null hypotheses (also called "discoveries", either true or false)

Inm hypothesis tests of which $m_{0}$ are true null hypotheses,R is an observable random variable, andS,T,U, andV are unobservablerandom variables.

Comparison with other error rates

[edit]

This sectionpossibly containsoriginal research. Pleaseimprove it byverifying the claims made and addinginline citations. Statements consisting only of original research should be removed.(February 2013) (Learn how and when to remove this message)

While the false positive rate is mathematically equal to thetype I error rate, it is viewed as a separate term for the following reasons:^{[citation needed]}

The type I error rate is often associated with thea-priori setting of the significance level by the researcher: the significance level represents an acceptable error rateconsidering that all null hypotheses are true (the "global null" hypothesis). The choice of a significance level may thus be somewhat arbitrary (i.e. setting 10% (0.1), 5% (0.05), 1% (0.01) etc.)

As opposed to that, the false positive rate is associated with apost-prior result, which is the expected number of false positives divided by the total number of hypotheses under thereal combination of true and non-true null hypotheses (disregarding the "global null" hypothesis). Since the false positive rate is a parameter that is not controlled by the researcher, it cannot be identified with the significance level.

Moreover, false positive rate is usually used regarding a medical test or diagnostic device (i.e. "the false positive rate of a certain diagnostic device is 1%"), while type I error is a term associated with statistical tests, where the meaning of the word "positive" is not as clear (i.e. "the type I error of a test is 1%").

The false positive rate should also not be confused with thefamily-wise error rate, which is defined as ${\boldsymbol {\mathrm {FWER} }}=\Pr(V\geq 1)\,$ . As the number of tests grows, the familywise error rate usually converges to 1 while the false positive rate remains fixed.

Lastly, it is important to note the profound difference between the false positive rate and thefalse discovery rate: while the first is defined as $E(V/m_{0})$ , the second is defined as $E(V/R)$ .