What is Harrell’s C-index?

Harrell’s C-index (also known as the concordance index) introduced in Harrell et al. 1982, is a goodness of fit measure for models which produce risk scores. It is commonly used to evaluate risk models in survival analysis, where data may be censored.

Survival data

For concreteness, let’s imagine that we are in the clinical setting. For a given patient, we are interested in predicting his/her “time-to-disease”, i.e. the length of time until he/she develops the disease of interest, given covariates (e.g. demographic information) that we have on hand today. To train a model to make these predictions, we have n patients with their covariate information X_1, \dots, X_p and a “time-to-event” response T.

Survival analysis is different from typical regression/classification because of the nature of the response T. For a given disease, not all patients are going to develop the disease. In clinical settings, it’s also possible that the patient never comes for a follow-up visit, so we won’t know if the patient developed the disease or not. Hence, for the ith patient, the “time-to-event” response T_i is either

  1. The actual time-to-disease if we get to observe it, or
  2. The last time at which we know that the patient did not have the disease.

To know which of the two cases happened, we have an auxiliary variable d such that d_i = 1 if we got to see the disease, d_i = 0 otherwise. If d_i = 0, patient i‘s time-to-event is said to be “censored”.

Harrell’s C-index

The intuition behind Harrell’s C-index is as follows. For patient i, our risk model assigns a risk score \eta_i. If our risk model is any good, patients who had shorter times-to-disease should have higher risk scores. Boiling this intuition down to two patients: the patient with the higher risk score should have a shorter time-to-disease.

We can compute the C-index in the following way: For every pair of patients i and j (with i \neq j), look at their risk scores and times-to-event.

  1. If both T_i and T_j are not censored, then we can observe when both patients got the disease. We say that the pair (i, j) is a concordant pair if \eta_i > \eta_j and T_i < T_j, and it is a discordant pair if \eta_i > \eta_j and T_i > T_j.
  2. If both T_i and T_j are censored, then we don’t know who got the disease first (if at all), so we don’t consider this pair in the computation.
  3. If one of T_i and T_j is censored, we only observe one disease. Let’s say we observe patient i getting disease at time T_i, and that T_j is censored. (The same logic holds for the reverse situation.)
    1. If T_j < T_i, then we don’t know for sure who got the disease first, so we don’t consider this pair in the computation.
    2. If T_j > T_i, then we know for sure that patient i got the disease first. Hence, (i, j) is a concordant pair if \eta_i > \eta_j, and is a discordant pair if \eta_i < \eta_j.

Harrell’s C-index is simply

\begin{aligned} c = \dfrac{\# \text{ concordant pairs}}{\# \text{ concordant pairs} + \# \text{ discordant pairs}}. \end{aligned}

The logic above can be expressed succinctly in a formula (taken from Reference 2):

\begin{aligned} c = \dfrac{\sum_{i \neq j} 1 \{ \eta_i < \eta_j \} 1 \{ T_i > T_j \} d_j }{\sum_{i \neq j} 1 \{ T_i > T_j \} d_j }. \end{aligned}

Values of c near 0.5 indicate that the risk score predictions are no better than a coin flip in determining which patient will live longer. Values near 1 indicate that the risk scores are good at determining which of two patients will have the disease first. Values near 0 means that the risk scores are worse than a coin flip: you might be better off concluding the opposite of what the risk scores tell you.

Harrell’s C-index for continuous data

Of course, one can compute the C-index if none of the data is censored. In that case, all pairs such that T_i \neq T_j will be included in the computation.

Harrell’s C-index for binary data

As Reference 3 suggests, the concept of the C-index can be easily ported over to binary data. In this setting, a high risk score prediction means more likely to be 1 than a 0. We only consider pairs where subject i‘s response is a 1 and subject j‘s response is a one. The pair is concordant if \eta_i > \eta_j, and discordant if \eta_i < \eta_j.

References:

  1. Harrell Jr, F. E. et al. (1982). Evaluating the yield of medical tests.
  2. Schmid, M. et al. (2016). On the use of Harrell’s C for clinical risk prediction via random survival forests.
  3. Statistics How To. What is a C-Statistic?

4 thoughts on “What is Harrell’s C-index?

    • The risk scores for the individuals are whatever your risk model assigns. Harrell’s C-index is a way to evaluate how good the risk model is. In R, the glmnet package as a function called Cindex() you can use. It is a wrapper for the survival package’s concordance() function. (I can’t comment on SAS code as I am not a SAS user.)

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s