What do we mean by isotropic/anisotropic covariance?

Update (2019-10-24): I totally messed up the definitions the first time I posted this! I’ve fixed it now. Many thanks to commenter szcfweiya to pointing this out!

Let \{ X_t \}_{t \in \mathbb{I}} be a stochastic process, where \mathbb{I} is the index set for the stochastic process. (Most often we have \mathbb{I} = [0, \infty) to index time or \mathbb{I} = \mathbb{R}^d to index space). The stochastic process has an associated covariance function K: \mathbb{I} \times \mathbb{I} \mapsto \mathbb{R} such that for any s, t \in \mathbb{I}, \text{Cov}(X_s, X_t) = K(s, t).

In general, a covariance function must satisfy two properties:

  1. It is symmetric, i.e. K(x, x') = K(x', x) for all x, x' \in \mathbb{I}, and
  2. It is positive semi-definite, i.e. for all n \in \mathbb{N}, x_1, x_2, \dots, x_n \in \mathbb{I}, a_1, \dots, a_n \in \mathbb{R}, \begin{aligned} \sum_{i=1}^n \sum_{j=1}^n a_i K(x_i, x_j) a_j \geq 0 \end{aligned}.

A covariance is isotropic if K(x, x') depends only on the distance \| x - x' \|.  A covariance is said to be anisotropic if it is not isotropic. That is, K(x, x') either does not depend on the distance \| x - x' \|, or it depends on \| x - x' \| as well as some other functions of x and x'.

Clearly the class of isotropic covariances is much smaller than that of anisotropic covariances. To model anisotropic covariances, one usually has to make an assumption on how K(x, x') depends on x and x'.

In my little googling around, anisotropic covariance modeling seems to be popular in geostatistics and more broadly, spatial statistics. One popular example of anisotropic covariance is called geometric anisotropy. In that setting, the index set of the stochastic process is \mathbb{R}^n (typically with n = 1 or n = 2). Isotropic covariance in this setting would have the form K(x, x') = \rho (d (x, x')), where \rho is some function and d is the Euclidean metric. (See this earlier post for some examples. Not all kernels there are isotropic but it should be obvious which are.) Geometric anisotropy refers to a covariance of the form K(x, x') = \rho (d' (x, x')), where d' is some other distance metric. Some examples (from Reference 1) are the Mahalanobis distance and the Minkowski distance.

A special case of using the Mahalanobis distance is where d'(x, x') = \sqrt{(x-x')D(x-x')} for some diagonal matrix D. This corresponds to giving each axis a different scale before computing the Euclidean distance.

References:

  1. Haskard, K. A. (2007) An anisotropic Matérn spatial covariance model: REML estimation and properties.

2 thoughts on “What do we mean by isotropic/anisotropic covariance?

  1. Hi kjytay, is there some typo? It seems that you didn’t distinguish isotropic with anisotropic in your example since you said “depend on the distance” for both of them.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s