A variable is said to be ordinal if it takes on values or categories which have a natural ordering to them but the distances between the categories is not known. Ordinal variables are very common: some examples include T-shirt sizes, the Likert scale used in surveys (“for each of the following statements, choose along the scale of strongly disagree to strongly agree”), income brackets and ranked variables.
In this post, I introduce 3 commonly used methods for fitting regression models with an ordinal response. In what follows, I assume that the response variable is ordinal, taking on category values . (As is ordinal, we should think of these values as category numbers instead of actual quantitative values.) We want to fit a model where we can predict based on a set of features or covariates .
Cumulative logit model with proportional odds
The cumulative logit model with proportional odds assumes the following relationship between and :
This is a natural extension of logistic regression: there, we have . Note also that for fixed , the above looks like logistic regression where we have made the response binary (above vs. below or equal to ).
Note that in this model, each value of gets its own intercept but the other coefficients are fixed across . Because of this, the model satisfies the proportional odds property: for all ,
With this model, the estimated probability of being less than or equal to is
Hence, the estimated probability of being equal to is
Adjacent-category logit model
The adjacent-category logit model assumes the relationship
Here the odds ratio uses only adjacent categories, whereas in the cumulative logit model it uses the entire response scale. Hence, the interpretation of the model is in terms of local odds ratios as opposed to cumulative odds ratios.
Having fit the model, the estimated probability of being equal to is
This is quite a different form from that for the cumulative logit model.
Continuation-ratio logit model
The continuation-ratio logit model assumes the relationship
Having fit the model, we have the estimated conditional probabilities
With a bit of algebra we can convert them to unconditional probability estimates:
- Agresti, A. (2010). Modeling Ordinal Categorical Data.