Several sets of (x, y) points, with the correlation coefficient ofx andy for each set. The correlation reflects the strength and direction of a linear relationship (top row), but not the slope of that relationship (middle row), nor many aspects of nonlinear relationships (bottom row). N.B.: the figure in the center has a slope of 0 but in that case the correlation coefficient is undefined because the variance ofY is zero.
Instatistics, thePearson correlation coefficient (PCC)[a] is acorrelation coefficient that measureslinear correlation between two sets of data. It is the ratio between thecovariance of two variables and the product of theirstandard deviations; thus, it is essentially a normalized measurement of the covariance, such that the result always has a value between −1 and 1.[3] A key difference is that unlike covariance, this correlation coefficient does not haveunits, allowing comparison of the strength of the joint association between different pairs of random variables that do not necessarily have the same units.[4] As with covariance itself, the measure can only reflect a linearcorrelation of variables, and ignores many other types of relationships or correlations. As a simple example, one would expect the age and height of a sample of children from a school to have a Pearson correlation coefficient significantly greater than 0, but less than 1 (as 1 would represent an unrealistically perfect correlation).
The correlation coefficient can be derived by considering the cosine of the angle between two points representing the two sets of x and y co-ordinate data.[12] This expression is therefore a number between -1 and 1 and is equal to unity when all the points lie on a straight line.
Pearson's correlation coefficient is thecovariance of the two variables divided by the product of their standard deviations. The form of the definition involves a "product moment", that is, the mean (the firstmoment about the origin) of the product of the mean-adjusted random variables; hence the modifierproduct-moment in the name.[verification needed]
Pearson's correlation coefficient, when applied to apopulation, is commonly represented by the Greek letterρ (rho) and may be referred to as thepopulation correlation coefficient or thepopulation Pearson correlation coefficient. Given a pair of random variables (for example, Height and Weight), the formula forρ[13] is[14]
Pearson's correlation coefficient, when applied to asample, is commonly represented by and may be referred to as thesample correlation coefficient or thesample Pearson correlation coefficient. We can obtain a formula for by substituting estimates of the covariances and variances based on a sample into the formula above. Given paired data consisting of pairs, is defined as
This formula suggests a convenient single-pass algorithm for calculating sample correlations, though depending on the numbers involved, it can sometimes benumerically unstable.
An equivalent expression gives the formula for as the mean of the products of thestandard scores as follows:
where
are defined as above, and are defined below
is the standard score (and analogously for the standard score of).
Alternative formulae for are also available. For example, one can use the following formula for:
Under heavy noise conditions, extracting the correlation coefficient between two sets ofstochastic variables is nontrivial, in particular whereCanonical Correlation Analysis reports degraded correlation values due to the heavy noise contributions. A generalization of the approach is given elsewhere.[15]
The values of both the sample and population Pearson correlation coefficients are on or between −1 and 1. Correlations equal to +1 or −1 correspond to data points lying exactly on a line (in the case of the sample correlation), or to a bivariate distribution entirelysupported on a line (in the case of the population correlation). The Pearson correlation coefficient is symmetric: corr(X,Y) = corr(Y,X).
A key mathematical property of the Pearson correlation coefficient is that it isinvariant under separate changes in location and scale in the two variables. That is, we may transformX toa +bX and transformY toc +dY, wherea,b,c, andd are constants withb,d > 0, without changing the correlation coefficient. (This holds for both the population and sample Pearson correlation coefficients.) More general linear transformations do change the correlation: see§ Decorrelation of n random variables for an application of this. In particular, it might be useful to notice that corr(-X,Y) = -corr(X,Y)
The correlation coefficient ranges from −1 to 1. An absolute value of exactly 1 implies that a linear equation describes the relationship betweenX andY perfectly, with all data points lying on aline. The correlation sign is determined by theregression slope: a value of +1 implies that all data points lie on a line for whichY increases asX increases, whereas a value of -1 implies a line whereY increases whileX decreases.[17] A value of 0 implies that there is no linear dependency between the variables.[18]
More generally,(Xi −X)(Yi −Y) is positive if and only ifXi andYi lie on the same side of their respective means. Thus the correlation coefficient is positive ifXi andYi tend to be simultaneously greater than, or simultaneously less than, their respective means. The correlation coefficient is negative (anti-correlation) ifXi andYi tend to lie on opposite sides of their respective means. Moreover, the stronger either tendency is, the larger is theabsolute value of the correlation coefficient.
Rodgers and Nicewander[19] cataloged thirteen ways of interpreting correlation or simple functions of it:
Function of raw scores and means
Standardized covariance
Standardized slope of the regression line
Geometric mean of the two regression slopes
Square root of the ratio of two variances
Mean cross-product of standardized variables
Function of the angle between two standardized regression lines
Function of the angle between two variable vectors
Rescaled variance of the difference between standardized scores
Estimated from the balloon rule
Related to the bivariate ellipses of isoconcentration
Function of test statistics from designed experiments
For uncentered data, there is a relation between the correlation coefficient and the angleφ between the two regression lines,y =gX(x) andx =gY(y), obtained by regressingy onx andx ony respectively. (Here,φ is measured counterclockwise within the first quadrant formed around the lines' intersection point ifr > 0, or counterclockwise from the fourth to the second quadrant ifr < 0.) One can show[20] that if the standard deviations are equal, thenr = secφ − tanφ, where sec and tan aretrigonometric functions.
For centered data (i.e., data which have been shifted by the sample means of their respective variables so as to have an average of zero for each variable), the correlation coefficient can also be viewed as thecosine of theangleθ between the two observedvectors inN-dimensional space (forN observations of each variable).[21]
Both the uncentered (non-Pearson-compliant) and centered correlation coefficients can be determined for a dataset. As an example, suppose five countries are found to have gross national products of 1, 2, 3, 5, and 8 billion dollars, respectively. Suppose these same five countries (in the same order) are found to have 11%, 12%, 13%, 15%, and 18% poverty. Then letx andy be ordered 5-element vectors containing the above data:x = (1, 2, 3, 5, 8) andy = (0.11, 0.12, 0.13, 0.15, 0.18).
By the usual procedure for finding the angleθ between two vectors (seedot product), theuncentered correlation coefficient is
This uncentered correlation coefficient is identical with thecosine similarity. The above data were deliberately chosen to be perfectly correlated:y = 0.10 + 0.01x. The Pearson correlation coefficient must therefore be exactly one. Centering the data (shiftingx byℰ(x) = 3.8 andy byℰ(y) = 0.138) yieldsx = (−2.8, −1.8, −0.8, 1.2, 4.2) andy = (−0.028, −0.018, −0.008, 0.012, 0.042), from which
This figure gives a sense of how the usefulness of a Pearson correlation for predicting values varies with its magnitude. Given jointly normalX,Y with correlationρ, (plotted here as a function ofρ) is the factor by which a givenprediction interval forY may be reduced given the corresponding value ofX. For example, ifρ = 0.5, then the 95% prediction interval ofY|X will be about 13% smaller than the 95% prediction interval ofY.
Several authors have offered guidelines for the interpretation of a correlation coefficient.[22][23] However, all such criteria are in some ways arbitrary.[23] The interpretation of a correlation coefficient depends on the context and purposes. A correlation of 0.8 may be very low if one is verifying a physical law using high-quality instruments, but may be regarded as very high in the social sciences, where there may be a greater contribution from complicating factors.
Permutation tests provide a direct approach to performing hypothesis tests and constructing confidence intervals. A permutation test for Pearson's correlation coefficient involves the following two steps:
Using the original paired data (xi, yi), randomly redefine the pairs to create a new data set (xi, yi′), where thei′ are apermutation of the set {1,...,n}. The permutationi′ is selected randomly, with equal probabilities placed on alln! possible permutations. This is equivalent to drawing thei′ randomly without replacement from the set {1, ...,n}. Inbootstrapping, a closely related approach, thei and thei′ are equal and drawn with replacement from {1, ...,n};
Construct a correlation coefficientr from the randomized data.
To perform the permutation test, repeat steps (1) and (2) a large number of times. Thep-value for the permutation test is the proportion of ther values generated in step (2) that are larger than the Pearson correlation coefficient that was calculated from the original data. Here "larger" can mean either that the value is larger in magnitude, or larger in signed value, depending on whether atwo-sided orone-sided test is desired.
Thebootstrap can be used to construct confidence intervals for Pearson's correlation coefficient. In the "non-parametric" bootstrap,n pairs (xi, yi) are resampled "with replacement" from the observed set ofn pairs, and the correlation coefficientr is calculated based on the resampled data. This process is repeated a large number of times, and the empirical distribution of the resampledr values are used to approximate thesampling distribution of the statistic. A 95%confidence interval forρ can be defined as the interval spanning from the 2.5th to the 97.5thpercentile of the resampledr values.
If and are random variables, with a simple linear relationship between them with an additive normal noise (i.e., y= a + bx + e), then astandard error associated to the correlation is
where is the correlation and the sample size.[24][25]
has a student'st-distribution in the null case (zero correlation).[26] This holds approximately in case of non-normal observed values if sample sizes are large enough.[27] For determining the critical values forr the inverse function is needed:
Alternatively, large sample, asymptotic approaches can be used.
Another early paper[28] provides graphs and tables for general values ofρ, for small sample sizes, and discusses computational approaches.
In the case where the underlying variables are not normal, the sampling distribution of Pearson's correlation coefficient follows a Student'st-distribution, but the degrees of freedom are reduced.[29]
In the special case when (zero population correlation), the exact density functionf(r) can be written as
where is thebeta function, which is one way of writing the density of a Student's t-distribution for astudentized sample correlation coefficient, as above.
To obtain a confidence interval for ρ, we first compute a confidence interval forF():
The inverse Fisher transformation brings the interval back to the correlation scale.
For example, suppose we observer = 0.7 with a sample size ofn=50, and we wish to obtain a 95% confidence interval for ρ. The transformed value is, so the confidence interval on the transformed scale is, or (0.5814, 1.1532). Converting back to the correlation scale yields (0.5237, 0.8188).
The square of the sample correlation coefficient is typically denotedr2 and is a special case of thecoefficient of determination. In this case, it estimates the fraction of the variance inY that is explained byX in asimple linear regression. So if we have the observed dataset and the fitted dataset then as a starting point the total variation in theYi around their average value can be decomposed as follows
where the are the fitted values from the regression analysis. This can be rearranged to give
The two summands above are the fraction of variance inY that is explained byX (right) and that is unexplained byX (left).
Next, we apply a property ofleast squares regression models, that the sample covariance between and is zero. Thus, the sample correlation coefficient between the observed and fitted response values in the regression can be written (calculation is under expectation, assumes Gaussian statistics)
Thus
where is the proportion of variance inY explained by a linear function ofX.
In the derivation above, the fact that
can be proved by noticing that the partial derivatives of theresidual sum of squares (RSS) overβ0 andβ1 are equal to 0 in the least squares model, where
The population Pearson correlation coefficient is defined in terms ofmoments, and therefore exists for any bivariateprobability distribution for which thepopulationcovariance is defined and themarginalpopulation variances are defined and are non-zero. Some probability distributions, such as theCauchy distribution, have undefined variance and hence ρ is not defined ifX orY follows such a distribution. In some practical applications, such as those involving data suspected to follow aheavy-tailed distribution, this is an important consideration. However, the existence of the correlation coefficient is usually not a concern; for instance, if the range of the distribution is bounded, ρ is always defined.
If the sample size is moderate or large and the population is normal, then, in the case of the bivariatenormal distribution, the sample correlation coefficient is themaximum likelihood estimate of the population correlation coefficient, and isasymptoticallyunbiased andefficient, which roughly means that it is impossible to construct a more accurate estimate than the sample correlation coefficient.
If the sample size is large and the population is not normal, then the sample correlation coefficient remains approximately unbiased, but may not be efficient.
If the sample size is large, then the sample correlation coefficient is aconsistent estimator of the population correlation coefficient as long as the sample means, variances, and covariance are consistent (which is guaranteed when thelaw of large numbers can be applied).
If the sample size is small, then the sample correlation coefficientr is not an unbiased estimate ofρ.[13] The adjusted correlation coefficient must be used instead: see elsewhere in this article for the definition.
Correlations can be different for imbalanceddichotomous data when there is variance error in sample.[33]
Like many commonly used statistics, the samplestatisticr is notrobust,[34] so its value can be misleading ifoutliers are present.[35][36] Specifically, the PMCC is neither distributionally robust,[37] nor outlier resistant[34] (seeRobust statistics § Definition). Inspection of thescatterplot betweenX andY will typically reveal a situation where lack of robustness might be an issue, and in such cases it may be advisable to use a robust measure of association. Note however that while most robust estimators of association measurestatistical dependence in some way, they are generally not interpretable on the same scale as the Pearson correlation coefficient.
Statistical inference for Pearson's correlation coefficient is sensitive to the data distribution. Exact tests, and asymptotic tests based on theFisher transformation can be applied if the data are approximately normally distributed, but may be misleading otherwise. In some situations, thebootstrap can be applied to construct confidence intervals, andpermutation tests can be applied to carry out hypothesis tests. Thesenon-parametric approaches may give more meaningful results in some situations where bivariate normality does not hold. However the standard versions of these approaches rely onexchangeability of the data, meaning that there is no ordering or grouping of the data pairs being analyzed that might affect the behavior of the correlation estimate.
A stratified analysis is one way to either accommodate a lack of bivariate normality, or to isolate the correlation resulting from one factor while controlling for another. IfW represents cluster membership or another factor that it is desirable to control, we canstratify the data based on the value ofW, then calculate a correlation coefficient within each stratum. The stratum-level estimates can then be combined to estimate the overall correlation while controlling forW.[38]
The sample correlation coefficientr is not an unbiased estimate ofρ. For data that follows abivariate normal distribution, the expectationE[r] for the sample correlation coefficientr of a normal bivariate is[39]
thereforer is a biased estimator of
The unique minimum variance unbiased estimatorradj is given by[40]
Suppose observations to be correlated have differing degrees of importance that can be expressed with a weight vectorw. To calculate the correlation between vectorsx andy with the weight vectorw (all of length n),[41][42]
The reflective correlation is a variant of Pearson's correlation in which the data are not centered around their mean values.[citation needed] The population reflective correlation is
The reflective correlation is symmetric, but it is not invariant under translation:
The sample reflective correlation is equivalent tocosine similarity:
The weighted version of the sample reflective correlation is
Scaled correlation is a variant of Pearson's correlation in which the range of the data is restricted intentionally and in a controlled manner to reveal correlations between fast components intime series.[43] Scaled correlation is defined as average correlation across short segments of data.
Let be the number of segments that can fit into the total length of the signal for a given scale:
The scaled correlation across the entire signals is then computed as
where is Pearson's coefficient of correlation for segment.
By choosing the parameter, the range of values is reduced and the correlations on long time scale are filtered out, only the correlations on short time scales being revealed. Thus, the contributions of slow components are removed and those of fast components are retained.
A distance metric for two variablesX andY known asPearson's distance can be defined from their correlation coefficient as[44]
Considering that the Pearson correlation coefficient falls between [−1, +1], the Pearson distance lies in [0, 2]. The Pearson distance has been used incluster analysis and data detection for communications and storage with unknown gain and offset.[45]
The Pearson "distance" defined this way assigns distance greater than 1 to negative correlations. In reality, both strong positive correlation and negative correlations are meaningful, so care must be taken when Pearson "distance" is used for nearest neighbor algorithm as such algorithm will only include neighbors with positive correlation and exclude neighbors with negative correlation. Alternatively, an absolute valued distance,, can be applied, which will take both positive and negative correlations into consideration. The information on positive and negative association can be extracted separately, later.
For variablesX = {x1,...,xn} andY = {y1,...,yn} that are defined on the unit circle[0, 2π), it is possible to define a circular analog of Pearson's coefficient.[46] This is done by transforming data points inX andY with asine function such that the correlation coefficient is given as:
where and are thecircular means ofX and Y. This measure can be useful in fields like meteorology where the angular direction of data is important.
If a population or data-set is characterized by more than two variables, apartial correlation coefficient measures the strength of dependence between a pair of variables that is not accounted for by the way in which they both change in response to variations in a selected subset of the other variables.
Pearson correlation coefficient in quantum systems
It is always possible to remove the correlations between all pairs of an arbitrary number of random variables by using a data transformation, even if the relationship between the variables is nonlinear. A presentation of this result for population distributions is given by Cox & Hinkley.[49]
A corresponding result exists for reducing the sample correlations to zero. Suppose a vector ofn random variables is observedm times. LetX be a matrix where is thejth variable of observationi. Let be anm bym square matrix with every element 1. ThenD is the data transformed so every random variable has zero mean, andT is the data transformed so all variables have zero mean and zero correlation with all other variables – the samplecorrelation matrix ofT will be the identity matrix. This has to be further divided by the standard deviation to get unit variance. The transformed variables will be uncorrelated, even though they may not beindependent.
where an exponent of−+1⁄2 represents thematrix square root of theinverse of a matrix. The correlation matrix ofT will be the identity matrix. If a new data observationx is a row vector ofn elements, then the same transform can be applied tox to get the transformed vectorsd andt:
ThePandas and Polars Python libraries implement the Pearson correlation coefficient calculation as the default option for the methodspandas.DataFrame.corr andpolars.corr, respectively.
^Also known asPearson'sr, thePearson product-moment correlation coefficient (PPMCC), thebivariate correlation,[1] or simply the unqualifiedcorrelation coefficient[2]
^As early as 1877, Galton was using the term "reversion" and the symbol "r" for what would become "regression".[5][6][7]
^Garren, Steven T. (15 June 1998). "Maximum likelihood estimation of the correlation coefficient in a bivariate normal model, with missing data".Statistics & Probability Letters.38 (3):281–288.doi:10.1016/S0167-7152(98)00035-2.
^Schmid, John Jr. (December 1947). "The relationship between the coefficient of correlation and the angle included between regression lines".The Journal of Educational Research.41 (4):311–313.doi:10.1080/00220671.1947.10881608.JSTOR27528906.
^Rummel, R.J. (1976)."Understanding Correlation". ch. 5 (as illustrated for a special case in the next paragraph).
^Buda, Andrzej; Jarynowski, Andrzej (December 2010).Life Time of Correlations and its Applications. Wydawnictwo Niezależne. pp. 5–21.ISBN978-83-915272-9-0.
^abCohen, J. (1988).Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
^Bowley, A. L. (1928). "The Standard Deviation of the Correlation Coefficient".Journal of the American Statistical Association.23 (161):31–34.doi:10.2307/2277400.ISSN0162-1459.JSTOR2277400.
^Hotelling, Harold (1953). "New Light on the Correlation Coefficient and its Transforms".Journal of the Royal Statistical Society. Series B (Methodological).15 (2):193–232.doi:10.1111/j.2517-6161.1953.tb00135.x.JSTOR2983768.
^Kenney, J.F.; Keeping, E.S. (1951).Mathematics of Statistics. Vol. Part 2 (2nd ed.). Princeton, NJ: Van Nostrand.
^Katz., Mitchell H. (2006)Multivariable Analysis – A Practical Guide for Clinicians. 2nd Edition. Cambridge University Press.ISBN978-0-521-54985-1.ISBN0-521-54985-X
^Hotelling, H. (1953). "New Light on the Correlation Coefficient and its Transforms".Journal of the Royal Statistical Society. Series B (Methodological).15 (2):193–232.doi:10.1111/j.2517-6161.1953.tb00135.x.JSTOR2983768.
"cocor".comparingcorrelations.org. – A free web interface and R package for the statistical comparison of two dependent or independent correlations with overlapping or non-overlapping variables.
"Correlation".nagysandor.eu. Archived fromthe original on 17 May 2021. Retrieved30 January 2013. – an interactive Flash simulation on the correlation of two normally distributed variables.
"Guess the Correlation". – A game where players guess how correlated two variables in a scatter plot are, in order to gain a better understanding of the concept of correlation.
