Instatistics, asequence ofrandom variables ishomoscedastic (/ˌhoʊmoʊskəˈdæstɪk/) if all its random variables have the same finitevariance; this is also known ashomogeneity of variance. The complementary notion is calledheteroscedasticity, also known asheterogeneity of variance.[a] The term originates from theAncient Greekσκεδάννυμιskedánnymi, 'to scatter'.[1][2][3]
Assuming a variable is homoscedastic when in reality it is heteroscedastic (/ˌhɛtəroʊskəˈdæstɪk/) results inunbiased butinefficientpoint estimates and in biased estimates ofstandard errors, and may result in overestimating thegoodness of fit as measured by thePearson coefficient.
The existence of heteroscedasticity is a major concern inregression analysis and theanalysis of variance, as it invalidatesstatistical tests of significance that assume that themodelling errors all have the same variance. While theordinary least squares (OLS) estimator is still unbiased in the presence of heteroscedasticity, it is inefficient and inference based on the assumption of homoskedasticity is misleading. In that case,generalized least squares (GLS) was frequently used in the past.[4][5] Nowadays, standard practice in econometrics is to includeHeteroskedasticity-consistent standard errors instead of using GLS, as GLS can exhibit strong bias in small samples if the actualskedastic function is unknown.[6]
Because heteroscedasticity concernsexpectations of the secondmoment of the errors, its presence is referred to asmisspecification of the second order.[7]
TheeconometricianRobert Engle was awarded the 2003Nobel Memorial Prize for Economics for his studies onregression analysis in the presence of heteroscedasticity, which led to his formulation of theautoregressive conditional heteroscedasticity (ARCH) modeling technique.[8]
Consider thelinear regression equation where the dependent random variable equals the deterministic variable times coefficient plus a random disturbance term that has mean zero. The disturbances are homoscedastic if the variance of is a constant; otherwise, they are heteroscedastic. In particular, the disturbances are heteroscedastic if the variance of depends on or on the value of. One way they might be heteroscedastic is if (an example of ascedastic function), so the variance is proportional to the value of.
More generally, if the variance-covariance matrix of disturbance across has a nonconstant diagonal, the disturbance is heteroscedastic.[9] The matrices below are covariances when there are just three observations across time. The disturbance in matrix A is homoscedastic; this is the simple case where OLS is the best linear unbiased estimator. The disturbances in matrices B and C are heteroscedastic. In matrix B, the variance is time-varying, increasing steadily across time; in matrix C, the variance depends on the value of. The disturbance in matrix D is homoscedastic because the diagonal variances are constant, even though the off-diagonal covariances are non-zero and ordinary least squares is inefficient for a different reason: serial correlation.
Heteroscedasticity often occurs when there is a large difference among the sizes of the observations.
A classic example of heteroscedasticity is that of income versus expenditure on meals. A wealthy person may eat inexpensive food sometimes and expensive food at other times. A poor person will almost always eat inexpensive food. Therefore, people with higher incomes exhibit greater variability in expenditures on food.
At a rocket launch, an observer measures the distance traveled by the rocket once per second. In the first couple of seconds, the measurements may be accurate to the nearest centimeter. After five minutes, the accuracy of the measurements may be good only to 100 m, because of the increased distance, atmospheric distortion, and a variety of other factors. So the measurements of distance may exhibit heteroscedasticity.
One of the assumptions of the classical linear regression model is that there is no heteroscedasticity. Breaking this assumption means that theGauss–Markov theorem does not apply, meaning thatOLS estimators are not theBest Linear Unbiased Estimators (BLUE) and their variance is not the lowest of all other unbiased estimators.Heteroscedasticity doesnot cause ordinary least squares coefficient estimates to be biased, although it can cause ordinary least squares estimates of the variance (and, thus, standard errors) of the coefficients to be biased, possibly above or below the true of population variance. Thus, regression analysis using heteroscedastic data will still provide an unbiased estimate for the relationship between the predictor variable and the outcome, but standard errors and therefore inferences obtained from data analysis are suspect. Biased standard errors lead to biased inference, so results of hypothesis tests are possibly wrong. For example, if OLS is performed on a heteroscedastic data set, yielding biased standard error estimation, a researcher might fail to reject a null hypothesis at a givensignificance level, when that null hypothesis was actually uncharacteristic of the actual population (making atype II error).
Under certain assumptions, the OLS estimator has a normalasymptotic distribution when properly normalized and centered (even when the data does not come from anormal distribution). This result is used to justify using a normal distribution, or achi square distribution (depending on how thetest statistic is calculated), when conducting ahypothesis test. This holds even under heteroscedasticity. More precisely, the OLS estimator in the presence of heteroscedasticity is asymptotically normal, when properly normalized and centered, with a variance-covariancematrix that differs from the case of homoscedasticity. In 1980, White proposed aconsistent estimator for the variance-covariance matrix of the asymptotic distribution of the OLS estimator.[2] This validates the use of hypothesis testing using OLS estimators and White's variance-covariance estimator under heteroscedasticity.
Heteroscedasticity is also a major practical issue encountered inANOVA problems.[10]TheF test can still be used in some circumstances.[11]
However, it has been said that students ineconometrics should not overreact to heteroscedasticity.[3] One author wrote, "unequal error variance is worth correcting only when the problem is severe."[12] In addition, another word of caution was in the form, "heteroscedasticity has never been a reason to throw out an otherwise good model."[3][13] With the advent ofheteroscedasticity-consistent standard errors allowing for inference without specifying the conditional second moment of error term, testing conditional homoscedasticity is not as important as in the past.[6]
For any non-linear model (for instanceLogit andProbit models), however, heteroscedasticity has more severe consequences: themaximum likelihood estimates (MLE) of the parameters will usually be biased, as well as inconsistent (unless the likelihood function is modified to correctly take into account the precise form of heteroscedasticity or the distribution is a member of thelinear exponential family and the conditional expectation function is correctly specified).[14][15] Yet, in the context of binary choice models (Logit orProbit), heteroscedasticity will only result in a positive scaling effect on the asymptotic mean of the misspecified MLE (i.e. the model that ignores heteroscedasticity).[16] As a result, the predictions which are based on the misspecified MLE will remain correct. In addition, the misspecified Probit and Logit MLE will be asymptotically normally distributed which allows performing the usual significance tests (with the appropriate variance-covariance matrix). However, regarding the general hypothesis testing, as pointed out byGreene, "simply computing a robust covariance matrix for an otherwise inconsistent estimator does not give it redemption. Consequently, the virtue of a robust covariance matrix in this setting is unclear."[17]
There are several common corrections for heteroscedasticity. They are:
Residuals can be tested for homoscedasticity using theBreusch–Pagan test,[20] which performs an auxiliary regression of the squared residuals on the independent variables. From this auxiliary regression, the explained sum of squares is retained, divided by two, and then becomes the test statistic for a chi-squared distribution with the degrees of freedom equal to the number of independent variables.[21] The null hypothesis of this chi-squared test is homoscedasticity, and the alternative hypothesis would indicate heteroscedasticity. Since the Breusch–Pagan test is sensitive to departures from normality or small sample sizes, the Koenker–Bassett or 'generalized Breusch–Pagan' test is commonly used instead.[22][additional citation(s) needed] From the auxiliary regression, it retains the R-squared value which is then multiplied by the sample size, and then becomes the test statistic for a chi-squared distribution (and uses the same degrees of freedom). Although it is not necessary for the Koenker–Bassett test, the Breusch–Pagan test requires that the squared residuals also be divided by the residual sum of squares divided by the sample size.[22] Testing for groupwise heteroscedasticity can be done with theGoldfeld–Quandt test.[23]
Due to the standard use ofheteroskedasticity-consistent Standard Errors and the problem ofPre-test, econometricians nowadays rarely use tests for conditional heteroskedasticity.[6]
Although tests for heteroscedasticity between groups can formally be considered as a special case of testing within regression models, some tests have structures specific to this case.
Two or morenormal distributions, are both homoscedastic and lackserial correlation if they share the same diagonals in theircovariance matrix, and their non-diagonal entries are zero. Homoscedastic distributions are especially useful to derive statisticalpattern recognition andmachine learning algorithms. One popular example of an algorithm that assumes homoscedasticity is Fisher'slinear discriminant analysis.The concept of homoscedasticity can be applied to distributions on spheres.[27]
The study of homescedasticity and heteroscedasticity has been generalized to the multivariate case, which deals with the covariances of vector observations instead of the variance of scalar observations. One version of this is to use covariance matrices as the multivariate measure of dispersion. Several authors have considered tests in this context, for both regression and grouped-data situations.[28][29]Bartlett's test for heteroscedasticity between grouped data, used most commonly in the univariate case, has also been extended for the multivariate case, but a tractable solution only exists for 2 groups.[30] Approximations exist for more than two groups, and they are both calledBox's M test.
{{cite book}}:ISBN / Date incompatibility (help)Most statistics textbooks will include at least some material on homoscedasticity and heteroscedasticity. Some examples are:
