Instatistics andnumerical analysis,isotonic regression ormonotonic regression is the technique of fitting a free-form line to a sequence of observations such that the fitted line isnon-decreasing (or non-increasing) everywhere, and lies as close to the observations as possible.

Isotonic regression has applications instatistical inference. For example, one might use it to fit an isotonic curve to the means of some set of experimental results when an increase in those means according to some particular ordering is expected. A benefit of isotonic regression is that it is not constrained by any functional form, such as the linearity imposed bylinear regression, as long as the function is monotonic increasing.

Another application is nonmetricmultidimensional scaling,^[1] where a low-dimensionalembedding for data points is sought such that order of distances between points in the embedding matchesorder of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order.

Isotonic regression is also used inprobabilistic classification to calibrate the predicted probabilities ofsupervised machine learning models.^[2]

Isotonic regression for the simply ordered case with univariate${\displaystyle x,y}$ has been applied to estimating continuous dose-response relationships in fields such as anesthesiology and toxicology. Narrowly speaking, isotonic regression only provides point estimates at observed values of${\displaystyle x.}$ Estimation of the complete dose-response curve without any additional assumptions is usually done via linear interpolation between the point estimates.^[3]

Software for computing isotone (monotonic) regression has been developed forR,^[4][5][6]Stata, andPython.^[7]

Let${\displaystyle (x_1,y_1),\dots ,(x_n,y_n)}$ be a given set of observations, where the${\displaystyle y_i\in \mathbb{R}}$ and the${\displaystyle x_i}$ fall in somepartially ordered set. For generality, each observation${\displaystyle (x_i,y_i)}$ may be given a weight${\displaystyle w_i\ge 0}$, although commonly${\displaystyle w_i=1}$ for all${\displaystyle i}$.

Isotonic regression seeks a weightedleast-squares fit${\displaystyle {\hat{y}}_i\approx y_i}$ for all${\displaystyle i}$, subject to the constraint that${\displaystyle {\hat{y}}_i\le {\hat{y}}_j}$ whenever${\displaystyle x_i\le x_j}$. This gives the followingquadratic program (QP) in the variables${\displaystyle {\hat{y}}_1,\dots ,{\hat{y}}_n}$:

${\displaystyle min\sum _{i=1}^n{w}_i({\hat{y}}_i-y_i{)}^2}$ subject to${\displaystyle {\hat{y}}_i\le {\hat{y}}_j\text{ for all }(i,j)\in E}$

where${\displaystyle E=\{(i,j):x_i\le x_j\}}$ specifies the partial ordering of the observed inputs${\displaystyle x_i}$ (and may be regarded as the set of edges of somedirected acyclic graph (dag) with vertices${\displaystyle 1,2,\dots n}$). Problems of this form may be solved by generic quadratic programming techniques.

In the usual setting where the${\displaystyle x_i}$ values fall in atotally ordered set such as${\displaystyle \mathbb{R}}$, we may assumeWLOG that the observations have been sorted so that${\displaystyle x_1\le x_2\le \cdots \le x_n}$, and take. In this case, a simpleiterative algorithm for solving the quadratic program is thepool adjacent violators algorithm. Conversely, Best and Chakravarti^[8] studied the problem as anactive set identification problem, and proposed a primal algorithm. These two algorithms can be seen as each other's dual, and both have acomputational complexity of${\displaystyle O(n)}$ on already sorted data.^[8]

To complete the isotonic regression task, we may then choose any non-decreasing function${\displaystyle f(x)}$ such that${\displaystyle f(x_i)={\hat{y}}_i}$ for all i. Any such function obviously solves

${\displaystyle \underset{f}{min}\sum _{i=1}^n{w}_i(f(x_i)-y_i{)}^2}$ subject to${\displaystyle f}$ being nondecreasing

and can be used to predict the${\displaystyle y}$ values for new values of${\displaystyle x}$. A common choice when${\displaystyle x_i\in \mathbb{R}}$ would be to interpolate linearly between the points${\displaystyle (x_i,{\hat{y}}_i)}$, as illustrated in the figure, yielding a continuous piecewise linear function:

As this article's first figure shows, in the presence of monotonicity violations the resulting interpolated curve will have flat (constant) intervals. In dose-response applications it is usually known that${\displaystyle f(x)}$ is not only monotone but alsosmooth. The flat intervals are incompatible with${\displaystyle f(x)}$'s assumed shape, and can be shown to be biased. A simple improvement for such applications, named centered isotonic regression (CIR), was developed by Oron and Flournoy and shown to substantially reduce estimation error for both dose-response and dose-finding applications.^[9] Both CIR and the standard isotonic regression for the univariate, simply ordered case, are implemented in the R package "cir".^[4] This package also provides analytical confidence-interval estimates.

Wikibooks has a book on the topic of:Isotonic regression

• Robertson, T.; Wright, F. T.; Dykstra, R. L. (1988).Order restricted statistical inference. New York: Wiley.ISBN 978-0-471-91787-8.
• Barlow, R. E.; Bartholomew, D. J.; Bremner, J. M.; Brunk, H. D. (1972).Statistical inference under order restrictions; the theory and application of isotonic regression. New York: Wiley.ISBN 978-0-471-04970-8.
• Shively, T.S., Sager, T.W., Walker, S.G. (2009). "A Bayesian approach to non-parametric monotone function estimation".Journal of the Royal Statistical Society, Series B.71 (1):159–175.CiteSeerX 10.1.1.338.3846.doi:10.1111/j.1467-9868.2008.00677.x.S2CID 119761196.{{cite journal}}: CS1 maint: multiple names: authors list (link)
• Wu, W. B.;Woodroofe, M.; Mentz, G. (2001). "Isotonic regression: Another look at the changepoint problem".Biometrika.88 (3):793–804.doi:10.1093/biomet/88.3.793.

• Nonparametric regression
• Nonparametric Bayesian statistics
• Numerical analysis

• Articles with short description
• Short description matches Wikidata
• CS1 maint: multiple names: authors list