Instatistics,polynomial regression is a form ofregression analysis in which the relationship between theindependent variablex and thedependent variabley is modeled as apolynomial inx. Polynomial regression fits a nonlinear relationship between the value ofx and the correspondingconditional mean ofy, denoted E(y |x). Although polynomial regression fits a nonlinear model to the data, as astatistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknownparameters that are estimated from thedata. Thus, polynomial regression is a special case ofmultiple linear regression.

The explanatory (independent) variables resulting from the polynomial expansion of the "baseline" variables are known as higher-degree terms. Such variables are also used inclassification settings.^[1]

Polynomial regression models are usually fit using the method ofleast squares. The least-squares method minimizes thevariance of theunbiasedestimators of the coefficients, under the conditions of theGauss–Markov theorem. The least-squares method was published in 1805 byLegendre and in 1809 byGauss. The firstdesign of anexperiment for polynomial regression appeared in an 1815 paper ofGergonne.^[2][3] In the twentieth century, polynomial regression played an important role in the development ofregression analysis, with a greater emphasis on issues ofdesign andinference.^[4] More recently, the use of polynomial models has been complemented by other methods, with non-polynomial models having advantages for some classes of problems.^{[citation needed]}

The goal of regression analysis is to model the expected value of a dependent variabley in terms of the value of an independent variable (or vector of independent variables)x. In simple linear regression, the model

${\displaystyle y={\beta }_0+{\beta }_{1}x+\epsilon ,}$

is used, where ε is an unobserved random error with mean zero conditioned on ascalar variablex. In this model, for each unit increase in the value ofx, the conditional expectation ofy increases byβ₁ units.

In many settings, such a linear relationship may not hold. For example, if we are modeling the yield of a chemical synthesis in terms of the temperature at which the synthesis takes place, we may find that the yield improves by a different amount for each unit increase in temperature. Or we may find that the yielddecreases withincreasing temperature (but only for a certain range of temperatures) andincreases withincreasing temperature in a different range of temperatures. In this case, we might propose a quadratic model of the form

${\displaystyle y={\beta }_0+{\beta }_{1}x+{\beta }_2{x}^2+\epsilon .}$

In this model, when the temperature is increased fromx tox + 1 units, the expected yield changes by${\displaystyle {\beta }_1+{\beta }_2(2x+1).}$ (This can be seen by obtaining the derivative with respect to x of the regression formula.) Forinfinitesimal changes inx, the effect ony is given by thetotal derivative with respect tox:${\displaystyle {\beta }_1+2{\beta }_{2}x.}$ The fact that the change in yield depends onx is what makes the relationship betweenx andy nonlinear even though the model is linear in the parameters to be estimated.

In general, we can model the expected value ofy as ann^th degree polynomial, yielding the general polynomial regression model

${\displaystyle y={\beta }_0+{\beta }_{1}x+{\beta }_2{x}^2+{\beta }_3{x}^3+\cdots +{\beta }_n{x}^n+\epsilon .}$

Conveniently, these models are all linear from the point of view ofestimation, since the regression function is linear in terms of the unknown parametersβ₀,β₁, .... Therefore, forleast squares analysis, the computational and inferential problems of polynomial regression can be completely addressed using the techniques ofmultiple regression. This is done by treatingx, x², ... as being distinct independent variables in a multiple regression model.

The polynomial regression model

${\displaystyle y_i={\beta }_0+{\beta }_1{x}_i+{\beta }_2{x}_i^2+\cdots +{\beta }_m{x}_i^m+{\epsilon }_i(i=1,2,\dots ,n)}$

can be expressed in matrix form in terms of adesign matrix${\displaystyle \mathbf{X}}$, a response vector${\displaystyle \overrightarrow{y}}$, a parameter vector${\displaystyle \overrightarrow{\beta }}$, and a vector${\displaystyle \overrightarrow{\epsilon }}$ of random errors. Thei-th row of${\displaystyle \mathbf{X}}$ and${\displaystyle \overrightarrow{y}}$ will contain thex andy value for thei-th data sample. Then the model can be written as asystem of linear equations:

which when using pure matrix notation is written as

${\displaystyle \overrightarrow{y}=\mathbf{X}\overrightarrow{\beta }+\overrightarrow{\epsilon }.}$

The vector of estimated polynomial regression coefficients (usingordinary least squaresestimation) is

${\displaystyle \hat{\overrightarrow{\beta }}=({\mathbf{X}}^{\mathsf{T}}\mathbf{X}{)}^{-1}{\mathbf{X}}^{\mathsf{T}}\overrightarrow{y},}$

assumingm <n which is required for the matrix to be invertible; then since${\displaystyle \mathbf{X}}$ is aVandermonde matrix, the invertibility condition is guaranteed to hold if all the${\displaystyle x_i}$ values are distinct. This is the unique least-squares solution.

The above matrix equations explain the behavior of polynomial regression well. However, to physically implement polynomial regression for a set of xy point pairs, more detail is useful. The below matrix equations for polynomial coefficients are expanded from regression theory without derivation and easily implemented.^[5][6][7]

After solving the abovesystem of linear equations for${\displaystyle {\beta }_0\text{ through }{\beta }_m}$, the regression polynomial may be constructed as follows:

Although polynomial regression is technically a special case of multiple linear regression, the interpretation of a fitted polynomial regression model requires a somewhat different perspective. It is often difficult to interpret the individual coefficients in a polynomial regression fit, since the underlying monomials can be highly correlated. For example,x andx² have correlation around 0.97 when x isuniformly distributed on the interval (0, 1). Although the correlation can be reduced by usingorthogonal polynomials, it is generally more informative to consider the fitted regression function as a whole. Point-wise or simultaneousconfidence bands can then be used to provide a sense of the uncertainty in the estimate of the regression function.

Polynomial regression is one example of regression analysis usingbasis functions to model a functional relationship between two quantities. More specifically, it replaces${\displaystyle x\in {\mathbb{R}}^{d_x}}$ in linear regression with polynomial basis${\displaystyle \phi (x)\in {\mathbb{R}}^{d_{\phi }}}$, e.g.${\displaystyle [1,x]\stackrel{\phi }{\to }[1,x,x^2,\dots ,x^d]}$. A drawback of polynomial bases is that the basis functions are "non-local", meaning that the fitted value ofy at a given valuex = x₀ depends strongly on data values withx far fromx₀.^[8] In modern statistics, polynomial basis-functions are used along with newbasis functions, such assplines,radial basis functions, andwavelets. These families of basis functions offer a more parsimonious fit for many types of data.

The goal of polynomial regression is to model a non-linear relationship between the independent and dependent variables (technically, between the independent variable and the conditional mean of the dependent variable). This is similar to the goal ofnonparametric regression, which aims to capture non-linear regression relationships. Therefore, non-parametric regression approaches such assmoothing can be useful alternatives to polynomial regression. Some of these methods make use of a localized form of classical polynomial regression.^[9] An advantage of traditional polynomial regression is that the inferential framework of multiple regression can be used (this also holds when using other families of basis functions such as splines).

A final alternative is to usekernelized models such assupport vector regression with apolynomial kernel.

Ifresiduals haveunequal variance, aweighted least squares estimator may be used to account for that.^[10]

• Curve fitting
• Line regression
• Local polynomial regression
• Polynomial and rational function modeling
• Polynomial interpolation
• Response surface methodology
• Smoothing spline

• Microsoft Excel makes use of polynomial regression when fitting a trendline to data points on an X Y scatter plot.^[11]

• Curve Fitting,PhET Interactive simulations, University of Colorado at Boulder

• Regression analysis

• Articles with short description
• Short description matches Wikidata
• All articles with unsourced statements
• Articles with unsourced statements from March 2018