equicorrelation | Statistical Odds & Ends

In this previous post, I introduced the concept of an equicorrelation matrix, i.e. an $\mathbb{R}^{n \times n}$ matrix where the entries on the diagonal are all equal to $1$ and all off-diagonal entries are equal to some parameter $\rho$ which lies in $[-1, 1]$ :

${\bf \Sigma} = \begin{pmatrix} 1 & \rho & \rho & \dots & \rho \\ \rho & 1 & \rho & \dots & \rho \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \rho & \rho & \rho &\dots & 1 \end{pmatrix}$

There, I noted that for the matrix to be positive definite, we need $-\frac{1}{n-1} < \rho < 1$ . (Alternatively, $-\frac{1}{n-1} \leq \rho \leq 1$ is equivalent to $\bf \Sigma$ being positive semidefinite.)

It might seem a little strange that $\rho$ has a non-trivial lower bound but no non-trivial upper bound. A quick direct proof of the lower bound follows from the fact that the determinant of a positive semidefinite matrix must be non-negative. We can explicitly compute the determinant of the equicorrelation matrix, so

$\begin{aligned} 0 &\leq \text{det} ({\bf \Sigma}) = (1-\rho)^{n-1}[1 + (n-1)\rho], \\ 1 + (n-1) \rho &\geq 0, \\ \rho &\geq -\frac{1}{n-1}. \end{aligned}$

Here’s another proof of the lower bound that might be slightly more illuminating. Recall that $\bf \Sigma$ being (symmetic) positive semidefinite is equivalent to the existence of random variables $X_1, \dots, X_n$ such that the random vector $\begin{pmatrix} X_1, \dots, X_n \end{pmatrix}^\top$ has covariance matrix $\bf \Sigma$ . Hence,

$\begin{aligned} 0 &\leq \text{Var}(X_1 + \dots + X_n) \\ &= \sum_{i=1}^n \text{Var}(X_i) + \sum_{i \neq j} \text{Cov}(X_i, X_j) \\ &= n + n(n-1)\rho, \\ \rho &\geq -\frac{1}{n-1}. \end{aligned}$

Now for some intuition. When $n = 2$ , the lower bound is the trivial $-1$ , which can be achieved when $X_2 = -X_1$ . Now, imagine that $n = 3$ : how can we add $X_3$ so that the pairwise correlations are minimized (i.e. be most negative)? There is an inherent tension: the more negatively correlated we make $X_3$ with $X_1$ , the more positively correlated $X_3$ becomes with $X_2$ . So there is no way for $\text{Cov}(X_i, X_j) = -1$ for every pair $(i, j)$ .

One analogy that could be helpful is to think of each $X_i$ as a magnet that only repels, and we are trying to squeeze all the magnets into one box. As we add more and more magnets, the magnets get closer and closer together (even though they are still trying to repel each other). We don’t get the same problem with magnets that only attract: they all simply stick to each other at a single point, no matter how many magnets we add.

The equicorrelation matrix is the $\mathbb{R}^{n \times n}$ matrix where the entries on the diagonal are all equal to $1$ and all off-diagonal entries are equal to some parameter $\rho$ which lies in $[-1, 1]$ . If we were to write out the matrix, it would look something like this:

${\bf \Sigma} = \begin{pmatrix} 1 & \rho & \rho & \dots & \rho \\ \rho & 1 & \rho & \dots & \rho \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \rho & \rho & \rho &\dots & 1 \end{pmatrix}$

Alternatively, we can write it as ${\bf \Sigma} = \rho {\bf 11}^T + (1-\rho){\bf I}$ , where ${\bf 1} \in \mathbb{R}^n$ is the column vector with all entries being 1 and $\bf I$ is the identity matrix.

Here are some useful properties of the equicorrelation matrix:

It is a Toeplitz matrix, and hence has the properties that all Toeplitz matrices has (see e.g. this link).
It has two eigenvalues. The first eigenvalue is $1 + (n-1)\rho$ , with associated eigenvector $v_1 = {\bf 1}$ . The second eigenvalue is $1 - \rho$ , with $n-1$ associated eigenvectors $v_2, \dots, v_n$ , where the entries of $v_k$ are $\begin{aligned} (v_k)_j = \begin{cases} 1 &\text{if } j = 1, \\ -1 &\text{if } j = k, \\ 0 &\text{otherwise.} \end{cases} \end{aligned}$ This can be verified directly by doing the matrix multiplication.
$\text{det} ({\bf \Sigma}) = (1-\rho)^{n-1}[1 + (n-1)\rho]$ . This is because the determinant of a square matrix is equal to the product of its eigenvalues.
$\bf \Sigma$ is positive definite if and only if $-\frac{1}{n-1} < \rho < 1$ . A sketch of the proof can be found in the answer here. It boils down to proving some properties of the determinant expression in the previous point.
${\bf \Sigma}^{-1} = \dfrac{1}{1-\rho} \left( {\bf I} - \dfrac{\rho}{1 + (n-1)\rho} {\bf 11}^T \right)$ . This can be verified directly by matrix multiplication. It can also be derived using the Sherman-Morrison formula.