# Toeplitz covariance structure

When someone says that their model has Toeplitz covariance (or correlation) structure, what do they mean?

In linear algebra, a Toeplitz matrix is also known as a diagonal-constant matrix: i.e. “each descending diagonal from left to right is constant”: $A = \begin{bmatrix} a_0 & a_{-1} & \dots & \dots & a_{-(n-1)} \\ a_1 & a_0 & a_{-1} &\ddots & \vdots \\ \vdots & \ddots & \ddots & \ddots & \vdots \\ \vdots & \ddots & a_1 & a_0 & a_{-1} \\ a_{n-1} & \dots & \dots & a_1 & a_0 \end{bmatrix}$.

A model is said to have Toeplitz covariance (correlation resp.) structure if the covariance  (correlation resp.) matrix is a Toeplitz matrix. Here are 2 places where we see such structures pop up:

• We have time series data at equally-spaced times $1, 2, \dots$, denoted by $X_1, X_2, \dots$. This model has Toeplitz covariance (correlation resp.) structure if for any $n$, the covariance (correlation resp.) matrix of $X_1, X_2, \dots, X_n$ is Toeplitz. The AR(1) model, commonly used in econometrics, is an example of this, since it has $\text{Cor}(X_i, X_j) = \rho^{|i-j|}$ for some $\rho$.
• We have a continuous-time stochastic process $\{ X_t \}$ which is weakly stationary, i.e. for any 2 times $t$ and $s$, $\text{Cov}(X_t, X_s)$ depends only on $t - s$. In this setting, for any equally-spaced times $t_1, \dots, t_n$, the covariance matrix of $X_{t_1}, \dots, X_{t_n}$ will be Toeplitz.

Why work with Toeplitz covariance structure? Other than the fact that they arise naturally in certain situations (like the two above), operations with Toeplitz matrices are fast, and a matrix inverse always exists.

References:

1. Toeplitz matrix, Wikipedia.
2. Guidelines for Selecting the Covariance Structure in Mixed Model Analysis, Chuck Kincaid.
3. Toeplitz Covariance Matrix Estimation for Adaptive Beamforming and Ultrasound Imaging, Michael Pettersson.