**Isotonic regression** is a method for obtaining a monotonic fit for 1-dimensional data. Let’s say we have data such that . (We assume no ties among the ‘s for simplicity.) Informally, isotonic regression looks for such that the ‘s approximate the ‘s well while being monotonically non-decreasing. Formally, the ‘s are the solution to the optimization problem

(**Note:** There is a corresponding solution for a monotonically non-increasing fit. Sometimes, the above is referred to as linear ordering isotonic regression, with isotonic regression referring to a more general version of the problem above. For more general versions, see Reference 1.)

Isotonic regression is useful for enforcing a monotonic fit to the data. Sometimes you might know that your trend is monotonic but the output from your model is not monotonic. In that situation you can use isotonic regression as a smoother for the data or a post-processing step to force your model prediction to be monotonic.

A commonly used algorithm to obtain the isotonic regression solution is the **pool-adjacent-violators algorithm (PAVA)**. It runs in linear time and linear memory. At a high level it works like this: go from left to right and set . If setting as such causes a violation of monotonicity (i.e. ), replace both and with the mean . This move may result in earlier violations (i.e. the new may be less than ): if that happens, we need to go back and fix it via averaging.

The animation below works through an example of how PAVA works. (Essentially I wrote a homebrew version of PAVA but kept a record of the intermediate fits.) Note that there are a number of ways to implement PAVA: what you see below may not be the most efficient. Click here for the R code; you can amend the dataset there to get your own version of the animation below.

If you want to do isotonic regression in R, **DON’T** use my homebrew version, use the `gpava`

function in the `isotone`

package instead.

References:

- Mair, P., Hornik, M., and de Leeuw, J. (2009). Isotone optimization in R: Pool-adjacent-violators algorithm (PAVA) and active set methods.

### Like this:

Like Loading...

*Related*

Dear Editor

Please can you details stepes of application about isotonic Regression

Using R code

Best regards

Salwa

LikeLike

Hi Salwa, there was a link to the R code I used to generate the plots in the post, here it is again: https://github.com/kjytay/misc/blob/master/blog/2020-05-24%20isotonic%20regression.R

If you want to do isotonic regression in R I suggest using the gpava function in the isotone package and not my homebrew version.

LikeLike

Hi Kjytay,

The animation did a really good job of presenting isotonic regression. Thank you for this.

I am interested in reading the first reference (stat wiki), but that link appears dead. Can you please update with a working link?

LikeLike

Hmm.. it looks like stat wiki has closed down. I didn’t keep a copy… If you are interested in learning more, I would recommend the second reference, as well as the works cited in its intro section.

(Update: I’ve removed the stat wiki link.)

LikeLike