


Instatistics,Spearman's rank correlation coefficient orSpearman'sρ is a number ranging from -1 to 1 that indicates how strongly two sets of ranks are correlated. It could be used in a situation where one only has ranked data, such as a tally of gold, silver, and bronze medals. If a statistician wanted to know whether people who are high ranking in sprinting are also high ranking in long-distance running, they would use a Spearman rank correlation coefficient.
The coefficient is named afterCharles Spearman[1] and often denoted by the Greek letter (rho) or as. It is anonparametric measure ofrank correlation (statistical dependence between therankings of twovariables). It assesses how well the relationship between two variables can be described using amonotonic function.
The Spearman correlation between two variables is equal to thePearson correlation between the rank values of those two variables; while Pearson's correlation assesses linear relationships, Spearman's correlation assesses monotonic relationships (whether linear or not). If there are no repeated data values, a perfect Spearman correlation of +1 or −1 occurs when each of the variables is a perfect monotone function of the other.
Intuitively, the Spearman correlation between two variables will be high when observations have a similar (or identical for a correlation of 1)rank (i.e. relative position label of the observations within the variable: 1st, 2nd, 3rd, etc.) between the two variables, and low when observations have a dissimilar (or fully opposed for a correlation of −1) rank between the two variables.
Spearman's coefficient is appropriate for bothcontinuous and discreteordinal variables.[2][3] Both Spearman's andKendall's can be formulated as special cases of a moregeneral correlation coefficient.
The coefficient can be used to determine how well data fits a model,[4] like when determining the similarity of text documents.[5]
The Spearman correlation coefficient is defined as thePearson correlation coefficient between therank variables.[6]
For a sample of size the pairs ofraw scores are converted to ranks and is computed as
where
Only when all ranks aredistinct integers (no ties), it can be computed using the popular formula
where
Consider a bivariate sample with corresponding rank pairsThen the Spearman correlation coefficient of is
where, as usual,
and
We shall show that can be expressed purely in terms of provided we assume that there be no ties within each sample.
Under this assumption, we have that can be viewed as random variables distributed like a uniformly distributed discrete random variable on Henceandwhere
and thus
(These sums can be computed using the formulas for thetriangular numbers andsquare pyramidal numbers, or basicsummation results fromumbral calculus.)
Observe now that
Putting this all together thus yields
Identical values are usually[7] each assignedfractional ranks equal to the average of their positions in the ascending order of the values, which is equivalent to averaging over all possible permutations.
If ties are present in the data set, the simplified formula above yields incorrect results: Only if in both variables all ranks are distinct, then (calculated according to biased variance).The first equation — normalizing by the standard deviation — may be used even when ranks are normalized to [0, 1] ("relative ranks") because it is insensitive both to translation and linear scaling.
The simplified method should also not be used in cases where the data set is truncated; that is, when the Spearman's correlation coefficient is desired for the topX records (whether by pre-change rank or post-change rank, or both), the user should use the Pearson correlation coefficient formula given above.[8]
There are several other numerical measures that quantify the extent ofstatistical dependence between pairs of observations. The most common of these is thePearson product-moment correlation coefficient, which is a similar correlation method to Spearman's rank, that measures the "linear" relationships between the raw numbers rather than between their ranks.
An alternative name for the Spearmanrank correlation is the "grade correlation";[9] in this, the "rank" of an observation is replaced by the "grade". In continuous distributions, the grade of an observation is, by convention, always one half less than the rank, and hence the grade and rank correlations are the same in this case. More generally, the "grade" of an observation is proportional to an estimate of the fraction of a population less than a given value, with the half-observation adjustment at observed values. Thus this corresponds to one possible treatment of tied ranks. While unusual, the term "grade correlation" is still in use.[10]
The sign of the Spearman correlation indicates the direction of association betweenX (the independent variable) andY (the dependent variable). IfY tends to increase whenX increases, the Spearman correlation coefficient is positive. IfY tends to decrease whenX increases, the Spearman correlation coefficient is negative. A Spearman correlation of zero indicates that there is no tendency forY to either increase or decrease whenX increases. The Spearman correlation increases in magnitude asX andY become closer to being perfectly monotonic functions of each other. WhenX andY are perfectly monotonically related, the Spearman correlation coefficient becomes 1. A perfectly monotonic increasing relationship implies that for any two pairs of data valuesXi,Yi andXj,Yj, thatXi −Xj andYi −Yj always have the same sign. A perfectly monotonic decreasing relationship implies that these differences always have opposite signs.
The Spearman correlation coefficient is often described as being "nonparametric". This can have two meanings. First, a perfect Spearman correlation results whenX andY are related by anymonotonic function. Contrast this with the Pearson correlation, which only gives a perfect value whenX andY are related by alinear function. The other sense in which the Spearman correlation is nonparametric is that its exact sampling distribution can be obtained without requiring knowledge (i.e., knowing the parameters) of thejoint probability distribution ofX andY.
The strength of correlations also are an important consideration for interpretation of correlation analyses but descriptions of the strength of correlation are not universally accepted. A small correlation coefficient calculated for a very large sample may be statistically significant without providing a quantitative relation between variables. Similarly, a large correlation coefficient calculated with a small sample size may indicate that a quantitative equation between variables may be possible even though the correlation coefficient is not statistically significant. Akoglu (2018)[11] noted the need for consistent descriptions of strength and provides different correlation-strength descriptors for psychology, politics, and medicine that have different descriptors and thresholds. Because there is a general lack of consensus on correlation strength, Granato (2014)[12] defined operational definitions for the absolute values of correlation coefficients as weak (less than 0.5), moderate (greater than or equal to 0.5 and less than 0.75), semi-strong (greater than or equal to 0.75 and less than 0.85), and strong (greater than or equal to 0.85) for use in hydrologic analyses of stormwater treatment statistics. Similarly, Schober and others (2018)[13] note that "...cutoff points are arbitrary and inconsistent and should be used judiciously..." Despite their warning Schober and others (2018) provide a table indicating that correlations less than or equal to 0.1 are negligible; correlations greater than 0.1 and less than or equal to 0.39 are weak; correlations greater than 0.39 and less than or equal to 0.69 are moderate; correlations greater than 0.69 and less than or equal to 0.89 are strong; and correlations greater than 0.89 are very strong. Descriptions of strength indicate the potential to develop quantitative relations among variables but these descriptors, as with the correlation coefficients themselves, do not indicate causation.
In this example, the arbitrary raw data in the table below is used to calculate the correlation between theIQ of a person with the number of hours spent in front ofTV per week [fictitious values used].
| IQ, | Hours ofTV per week, |
|---|---|
| 106 | 7 |
| 100 | 27 |
| 86 | 2 |
| 101 | 50 |
| 99 | 28 |
| 103 | 29 |
| 97 | 20 |
| 113 | 12 |
| 112 | 6 |
| 110 | 17 |
Firstly, evaluate. To do so use the following steps, reflected in the table below.
| IQ, | Hours ofTV per week, | rank | rank | ||
|---|---|---|---|---|---|
| 86 | 2 | 1 | 1 | 0 | 0 |
| 97 | 20 | 2 | 6 | −4 | 16 |
| 99 | 28 | 3 | 8 | −5 | 25 |
| 100 | 27 | 4 | 7 | −3 | 9 |
| 101 | 50 | 5 | 10 | −5 | 25 |
| 103 | 29 | 6 | 9 | −3 | 9 |
| 106 | 7 | 7 | 3 | 4 | 16 |
| 110 | 17 | 8 | 5 | 3 | 9 |
| 112 | 6 | 9 | 2 | 7 | 49 |
| 113 | 12 | 10 | 4 | 6 | 36 |
With found, add them to find. The value ofn is 10. These values can now be substituted back into the equation
to give
which evaluates toρ = −29/165 = −0.175757575... with ap-value = 0.627188 (using thet-distribution).

That the value is close to zero shows that the correlation between IQ and hours spent watching TV is very low, although the negative value suggests that the longer the time spent watching television the lower the IQ. In the case of ties in the original values, this formula should not be used; instead, the Pearson correlation coefficient should be calculated on the ranks (where ties are given ranks, as described above).
Confidence intervals for Spearman'sρ can be easily obtained using the Jackknife Euclidean likelihood approach in de Carvalho and Marques (2012).[14] The confidence interval with level is based on a Wilks' theorem given in the latter paper, and is given by
where is the quantile of a chi-square distribution with one degree of freedom, and the are jackknife pseudo-values. This approach is implemented in the R packagespearmanCI.
One approach to test whether an observed value ofρ is significantly different from zero (r will always maintain−1 ≤r ≤ 1) is to calculate the probability that it would be greater than or equal to the observedr, given thenull hypothesis, by using apermutation test. An advantage of this approach is that it automatically takes into account the number of tied data values in the sample and the way they are treated in computing the rank correlation.
Another approach parallels the use of theFisher transformation in the case of the Pearson product-moment correlation coefficient. That is,confidence intervals andhypothesis tests relating to the population valueρ can be carried out using the Fisher transformation:
IfF(r) is the Fisher transformation ofr, the sample Spearman rank correlation coefficient, andn is the sample size, then
is az-score forr, which approximately follows a standardnormal distribution under thenull hypothesis ofstatistical independence (ρ = 0).[15][16]
One can also test for significance using
which is distributed approximately asStudent'st-distribution withn − 2 degrees of freedom under thenull hypothesis.[17] A justification for this result relies on a permutation argument.[18]
A generalization of the Spearman coefficient is useful in the situation where there are three or more conditions, a number of subjects are all observed in each of them, and it is predicted that the observations will have a particular order. For example, a number of subjects might each be given three trials at the same task, and it is predicted that performance will improve from trial to trial. A test of the significance of the trend between conditions in this situation was developed by E. B. Page[19] and is usually referred to asPage's trend test for ordered alternatives.
Classiccorrespondence analysis is a statistical method that gives a score to every value of two nominal variables. In this way the Pearsoncorrelation coefficient between them is maximized.
There exists an equivalent of this method, calledgrade correspondence analysis, which maximizes Spearman'sρ orKendall's τ.[20]
There are two existing approaches to approximating the Spearman's rank correlation coefficient from streaming data.[21][22] The first approach[21]involves coarsening the joint distribution of. For continuous values: cutpoints are selected for and respectively, discretizingthese random variables. Default cutpoints are added at and. A count matrix of size, denoted, is then constructed where stores the number of observations thatfall into the two-dimensional cell indexed by. For streaming data, when a new observation arrives, the appropriate element is incremented. The Spearman's rankcorrelation can then be computed, based on the count matrix, using linear algebra operations (Algorithm 2[21]). Note that for discrete randomvariables, no discretization procedure is necessary. This method is applicable to stationary streaming data as well as large data sets. For non-stationary streaming data, where the Spearman's rank correlation coefficient may change over time, the same procedure can be applied, but to a moving window of observations. When using a moving window, memory requirements grow linearly with chosen window size.
The second approach to approximating the Spearman's rank correlation coefficient from streaming data involves the use of Hermite series based estimators.[22] These estimators, based onHermite polynomials,allow sequential estimation of the probability density function and cumulative distribution function in univariate and bivariate cases. Bivariate Hermite series densityestimators and univariate Hermite series based cumulative distribution function estimators are plugged into a large sample version of theSpearman's rank correlation coefficient estimator, to give a sequential Spearman's correlation estimator. This estimator is phrased interms of linear algebra operations for computational efficiency (equation (8) and algorithm 1 and 2[22]). These algorithms are only applicable to continuous random variable data, but havecertain advantages over the count matrix approach in this setting. The first advantage is improved accuracy when applied to large numbers of observations. The second advantage is that the Spearman's rank correlation coefficient can becomputed on non-stationary streams without relying on a moving window. Instead, the Hermite series based estimator uses an exponential weighting scheme to track time-varying Spearman's rank correlation from streaming data,which has constant memory requirements with respect to "effective" moving window size. A software implementation of these Hermite series based algorithms exists[23] and is discussed in Software implementations.
cor.test(x, y, method = "spearman") in its "stats" package (alsocor(x, y, method = "spearman") will work). The packagespearmanCI computes confidence intervals. The packagehermiter[23] computes fast batch estimates of the Spearman correlation along with sequential estimates (i.e. estimates that are updated in an online/incremental manner as new observations are incorporated). spearmanvarlist calculates all pairwise correlation coefficients for all variables invarlist.[r,p] = corr(x,y,'Type','Spearman') wherer is the Spearman's rank correlation coefficient,p is the p-value, andx andy are vectors.[24]scipy.stats module, as well as with theDataFrame.corr(method='spearman') method from thepandas library, and thecorr(x, y, method='spearman') function from the statistical packagepingouin.