Tukey's range test, also known asTukey's test,Tukey method,Tukey's honest significance test, orTukey's HSD (honestly significant difference)test,^[1]is a single-stepmultiple comparison procedure andstatistical test. It can be used to correctly interpret thestatistical significance of the difference between means that have been selected for comparison because of their extreme values.

The method was initially developed and introduced byJohn Tukey for use inAnalysis of Variance (ANOVA), and usually has only been taught in connection with ANOVA. However, thestudentized range distribution used to determine the level of significance of the differences considered in Tukey's test has vastly broader application: It is useful for researchers who have searched their collected data for remarkable differences between groups, but then cannot validly determinehow significant their discovered stand-out difference is using standard statistical distributions used for other conventional statistical tests, for which the data must have been selected at random. Since when stand-out data is compared it was by definitionnot selected at random, but rather specifically chosen because it was extreme, it needs a different, stricter interpretation provided by the likely frequency and size of thestudentized range; the modern practice of "data mining" is an example where it is used.

The test was devised byJohn Tukey,^[2]it compares all possible pairs ofmeans, and is based on astudentized range distribution (q) (this distribution is similar to the distribution oft from thet-test. See below).^{[3][full citation needed]}

Tukey's test compares the means of every treatment to the means of every other treatment; that is, it applies simultaneously to the set of all pairwise comparisons

${\displaystyle {\mu }_i-{\mu }_j,}$

and identifies any difference between two means that is greater than the expectedstandard error. Theconfidence coefficient for theset, when all sample sizes are equal, is exactly${\displaystyle 1-\alpha }$ for any${\displaystyle \alpha :0\le \alpha \le 1.}$ For unequal sample sizes, the confidence coefficient is greater than${\displaystyle 1-\alpha .}$ In other words, the Tukey method is conservative when there areunequal sample sizes.

This test is often followed by theCompact Letter Display (CLD) statistical procedure to render the output of this test more transparent to non-statistician audiences.

1. The observations being tested areindependent within and among the groups.^{[citation needed]}
2. The subgroups associated with each mean in the test arenormally distributed.^{[citation needed]}
3. There is equal within-subgroup variance across the subgroups associated with each mean in the test (homogeneity of variance).^{[citation needed]}

Tukey's test is based on a formula very similar to that of thet-test. In fact, Tukey's test is essentially at-test, except that it corrects forfamily-wise error rate.

The formula for Tukey's test is

${\displaystyle q_{\mathsf{s}}=\frac{\left|Y_{\mathsf{A}}-Y_{\mathsf{B}}\right|}{\mathsf{S}\mathsf{E}},}$

whereY_A andY_B are the two means being compared, and SE is thestandard error for the sum of the means. The valueq_s is the sample's test statistic. (The notation |x| means theabsolute value ofx; the magnitude ofx with the sign set to+, regardless of the original sign ofx.)

Thisq_s test statistic can then be compared to aq value for the chosen significance levelα from a table of thestudentized range distribution. If theq_s value islarger than the critical valueq_α obtained from the distribution, the two means are said to be significantly different at level${\displaystyle \alpha :0\le \alpha \le 1.}$^[3]

Since thenull hypothesis for Tukey's test states that all means being compared are from the same population (i.e.μ₁ =μ₂ =μ₃ = ... =μ_k), the means should be normally distributed (according to thecentral limit theorem) with the same modelstandard deviationσ, estimated by the mergedstandard error,${\displaystyle \mathsf{S}\mathsf{E},}$ for all the samples; its calculation is discussed in the following sections. This gives rise to the normality assumption of Tukey's test.

The Tukey method uses thestudentized range distribution. Suppose that we take a sample of sizen from each ofk populations with the samenormal distributionN(μ,σ²) and suppose that${\displaystyle {\overline{y}}_{\mathsf{m}\mathsf{i}\mathsf{n}}}$ is the smallest of these sample means and${\displaystyle {\overline{y}}_{\mathsf{m}\mathsf{a}\mathsf{x}}}$ is the largest of these sample means, and supposeS² is thepooled sample variance from these samples. Then the following random variable has a Studentized range distribution:

${\displaystyle q\equiv \frac{{\overline{y}}_{\mathsf{m}\mathsf{a}\mathsf{x}}-{\overline{y}}_{\mathsf{m}\mathsf{i}\mathsf{n}}}{S\sqrt{2/n}}}$

This definition of the statisticq given above is the basis of the critically significant value forq_α discussed below, and is based on these three factors:

${\displaystyle \alpha }$ theType I error rate, or the probability of rejecting a true null hypothesis;

${\displaystyle k}$ the number of sub-populations being compared;

${\displaystyle \mathsf{d}\mathsf{f}}$ the number of degrees of freedom for each mean

( df =N −k ) whereN is the total number of observations.)

The distribution ofq has been tabulated and appears in many textbooks on statistics. In some tables the distribution ofq has been tabulated without the${\displaystyle \sqrt{2}}$ factor. To understand which table it is, we can compute the result fork = 2 and compare it to the result of theStudent's t-distribution with the same degrees of freedom and the sameα .In addition,R offers acumulative distribution function (ptukey) and aquantile function (qtukey)forq .

The Tukeyconfidence limits for all pairwise comparisons with confidence coefficient of at least  1 −α  are

${\displaystyle {\overline{y}}_{i\bullet }-{\overline{y}}_{j\bullet }\pm \frac{q_{\alpha ;k;N-k}}{\sqrt{2}}{\hat{\sigma }}_{\epsilon }\sqrt{\frac{2}{n}}:i,j=1,\dots ,ki\ne j.}$

Notice that the point estimator and the estimated variance are the same as those for a single pairwise comparison. The only difference between the confidence limits for simultaneous comparisons and those for a single comparison is the multiple of the estimated standard deviation.

Also note that the sample sizes must be equal when using the studentized range approach.${\displaystyle {\hat{\sigma }}_{\epsilon }}$ is the standard deviation of the entire design, not just that of the two groups being compared. It is possible to work with unequal sample sizes. In this case, one has to calculate the estimated standard deviation for each pairwise comparison as formalized byClyde Kramer in 1956, so the procedure for unequal sample sizes is sometimes referred to as theTukey–Kramer method which is as follows:

${\displaystyle {\overline{y}}_{i\bullet }-{\overline{y}}_{j\bullet }\pm \frac{q_{\alpha ;k;N-k}}{\sqrt{2}}{\hat{\sigma }}_{\epsilon }\sqrt{\frac{1}{n_i}+\frac{1}{n_j}}}$

wheren_i andn_j are the sizes of groupsi andj respectively. The degrees of freedom for the whole design is also applied.

Both ANOVA and Tukey–Kramer tests are based on the same assumptions. However, these two tests fork groups (i.e.μ₁ =μ₂ = ... =μ_k) may result in logical contradictions whenk > 2 , even if the assumptions do hold.

It is possible to generate a set of pseudorandom samples of strictly negative measure such that hypothesisμ₁ =μ₂ is rejected at significance level${\displaystyle 1-\alpha >0.95}$ whileμ₁ =μ₂ =μ₃ is not rejected even at${\displaystyle 1-\alpha =0.975.}$^[4]

• Family-wise error rate
• Newman–Keuls method

• Montgomery, Douglas C. (2013).Design and Analysis of Experiments (8th ed.). Wiley. Section 3.5.7.

• "Tukey's method". e-Handbook of Statistical Methods.itl.nist.gov/div898/handbook. SEMATECH.National Institute of Standards and Technology / U.S.Department of Commerce.

• Analysis of variance
• Statistical tests
• Multiple comparisons

• Articles with short description
• Short description matches Wikidata
• Use dmy dates from March 2024
• All articles with incomplete citations
• Articles with incomplete citations from March 2024
• All articles with unsourced statements
• Articles with unsourced statements from January 2022