This article has multiple issues. Please helpimprove it or discuss these issues on thetalk page.(Learn how and when to remove these messages) This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Studentized residual" – news ·newspapers ·books ·scholar ·JSTOR(May 2015) (Learn how and when to remove this message) This article'sfactual accuracy isdisputed. Relevant discussion may be found on thetalk page. Please help to ensure that disputed statements arereliably sourced.(February 2014) (Learn how and when to remove this message) (Learn how and when to remove this message)

Instatistics, astudentized residual is thedimensionless ratio resulting from the division of aresidual by anestimate of itsstandard deviation, both expressed in the sameunits. It is a form of aStudent'st-statistic, with the estimate of error varying between points.

This is an important technique in the detection ofoutliers. It is among several named in honor ofWilliam Sealey Gosset, who wrote under the pseudonym "Student" (e.g.,Student's distribution). Dividing a statistic by asample standard deviation is calledstudentizing, in analogy withstandardizing andnormalizing.

The key reason for studentizing is that, inregression analysis of amultivariate distribution, the variances of theresiduals at different input variable values may differ, even if the variances of theerrors at these different input variable values are equal. The issue is the difference betweenerrors and residuals in statistics, particularly the behavior of residuals in regressions.

Consider thesimple linear regression model

${\displaystyle Y={\alpha }_0+{\alpha }_{1}X+\epsilon .}$

Given a random sample (X_i, Y_i),i = 1, ..., n, each pair (X_i, Y_i) satisfies

${\displaystyle Y_i={\alpha }_0+{\alpha }_1{X}_i+{\epsilon }_i,}$

where theerrors${\displaystyle {\epsilon }_i}$, areindependent and all have the same variance${\displaystyle {\sigma }^2}$. Theresiduals are not the true errors, butestimates, based on the observable data. When themethod of least squares is used to estimate${\displaystyle {\alpha }_0}$ and${\displaystyle {\alpha }_1}$, then the residuals${\displaystyle \hat{\epsilon }}$, unlike the errors${\displaystyle \epsilon }$, cannot be independent since they satisfy the two constraints

${\displaystyle \sum _{i=1}^n{\hat{\epsilon }}_i=0}$

and

${\displaystyle \sum _{i=1}^n{\hat{\epsilon }}_i{x}_i=0.}$

(Hereε_i is theith error, and${\displaystyle {\hat{\epsilon }}_i}$ is theith residual.)

The residuals, unlike the errors,do not all have the same variance: the variance decreases as the correspondingx-value gets farther from the averagex-value. This is not a feature of the data itself, but of the regression better fitting values at the ends of the domain. It is also reflected in theinfluence functions of various data points on theregression coefficients: endpoints have more influence. This can also be seen because the residuals at endpoints depend greatly on the slope of a fitted line, while the residuals at the middle are relatively insensitive to the slope. The fact thatthe variances of the residuals differ, even thoughthe variances of the true errors are all equal to each other, is theprincipal reason for the need for studentization.

It is not simply a matter of the population parameters (mean and standard deviation) being unknown – it is thatregressions yielddifferent residual distributions atdifferent data points, unlikepointestimators ofunivariate distributions, which share acommon distribution for residuals.

For this simple model, thedesign matrix is

and thehat matrixH is the matrix of theorthogonal projection onto the column space of the design matrix:

${\displaystyle H=X(X^{T}X{)}^{-1}{X}^T.}$

Theleverageh_ii is theith diagonal entry in the hat matrix. The variance of theith residual is

${\displaystyle \mathrm{var}({\hat{\epsilon }}_i)={\sigma }^2(1-h_{ii}).}$

In case the design matrixX has only two columns (as in the example above), this is equal to

${\displaystyle \mathrm{var}({\hat{\epsilon }}_i)={\sigma }^2\left(1-\frac{1}{n}-\frac{(x_i-\overline{x}{)}^2}{\sum _{j=1}^n(x_j-\overline{x}{)}^2}\right).}$

In the case of anarithmetic mean, the design matrixX has only one column (avector of ones), and this is simply:

${\displaystyle \mathrm{var}({\hat{\epsilon }}_i)={\sigma }^2\left(1-\frac{1}{n}\right).}$

Given the definitions above, theStudentized residual is then

${\displaystyle t_i=\frac{{\hat{\epsilon }}_i}{\hat{\sigma }\sqrt{1-h_{ii}}}}$

whereh_ii is theleverage, and${\displaystyle \hat{\sigma }}$ is an appropriate estimate ofσ (see below).

In the case of a mean, this is equal to:

${\displaystyle t_i=\frac{{\hat{\epsilon }}_i}{\hat{\sigma }\sqrt{(n-1)/n}}}$

The usual estimate ofσ² is theinternally studentized residual

${\displaystyle {\hat{\sigma }}^2=\frac{1}{n-m}\sum _{j=1}^n{\hat{\epsilon }}_j^2.}$

wherem is the number of parameters in the model (2 in our example).

But if thei th case is suspected of being improbably large, then it would also not be normally distributed. Hence it is prudent to exclude thei th observation from the process of estimating the variance when one is considering whether thei th case may be an outlier, and instead use theexternally studentized residual, which is

${\displaystyle {\hat{\sigma }}_{(i)}^2=\frac{1}{n-m-1}\sum _{\begin{array}{c}j=1\\ j\ne i\end{array}}^n{\hat{\epsilon }}_j^2,}$

based on all the residualsexcept the suspecti th residual. Here is to emphasize that${\displaystyle {\hat{\epsilon }}_j^2(j\ne i)}$ for suspecti are computed withi th case excluded.

If the estimateσ²includes thei th case, then it is called theinternally studentized residual,${\displaystyle t_i}$ (also known as thestandardized residual^[1]).If the estimate${\displaystyle {\hat{\sigma }}_{(i)}^2}$ is used instead,excluding thei th case, then it is called theexternally studentized,${\displaystyle t_{i(i)}}$.

If the errors are independent andnormally distributed withexpected value 0 and varianceσ², then theprobability distribution of theith externally studentized residual${\displaystyle t_{i(i)}}$ is aStudent's t-distribution withn − m − 1degrees of freedom, and can range from${\displaystyle -\mathrm{\infty }}$ to${\displaystyle +\mathrm{\infty }}$.

On the other hand, the internally studentized residuals are in the range${\displaystyle 0\pm \sqrt{\nu }}$, whereν =n − m is the number of residual degrees of freedom. Ift_i represents the internally studentized residual, and again assuming that the errors are independent identically distributed Gaussian variables, then:^[2]

${\displaystyle t_i\sim \sqrt{\nu }\frac{t}{\sqrt{t^2+\nu -1}}}$

wheret is a random variable distributed asStudent's t-distribution withν − 1 degrees of freedom. In fact, this implies thatt_i² /ν follows thebeta distributionB(1/2,(ν − 1)/2).The distribution above is sometimes referred to as thetau distribution;^[2] it was first derived by Thompson in 1935.^[3]

Whenν = 3, the internally studentized residuals areuniformly distributed between${\displaystyle -\sqrt{3}}$ and${\displaystyle +\sqrt{3}}$.If there is only one residual degree of freedom, the above formula for the distribution of internally studentized residuals doesn't apply. In this case, thet_i are all either +1 or −1, with 50% chance for each.

The standard deviation of the distribution of internally studentized residuals is always 1, but this does not imply that the standard deviation of all thet_i of a particular experiment is 1.For instance, the internally studentized residuals when fitting a straight line going through (0, 0) to the points (1, 4), (2, −1), (2, −1) are${\displaystyle \sqrt{2},-\sqrt{5}/5,-\sqrt{5}/5}$, and the standard deviation of these is not 1.

Note that any pair of studentized residualt_i andt_j (where${\displaystyle i\ne j}$), are NOT i.i.d. They have the same distribution, but are not independent due to constraints on the residuals having to sum to 0 and to have them be orthogonal to the design matrix.

Many programs and statistics packages, such asR,Python, etc., include implementations of Studentized residual.

• Cook's distance – a measure of changes in regression coefficients when an observation is deleted
• Grubbs's test
• Normalization (statistics)
• Samuelson's inequality
• Standard score
• William Sealy Gosset

• Cook, R. Dennis; Weisberg, Sanford (1982).Residuals and Influence in Regression (Repr. ed.). New York:Chapman and Hall.ISBN 041224280X. Retrieved23 February 2013.

• Statistical outliers
• Statistical deviation and dispersion
• Errors and residuals
• Statistical ratios
• Regression diagnostics

• Articles with short description
• Short description matches Wikidata
• Articles needing additional references from May 2015
• All articles needing additional references
• Accuracy disputes from February 2014
• All accuracy disputes
• Articles with multiple maintenance issues