Laplace distribution as a mixture of normals

Posted on by

Previously on this blog we showed that the t-distribution can be expressed as a continuous mixture of normal distributions. Today, I learned from this paper that the Laplace distribution can be viewed as a continuous mixture of normal distributions as well.

The Laplace distribution with mean \mu \in \mathbb{R} and scale b > 0 has the probability density function

\begin{aligned} f(x) = \frac{1}{2b} \exp \left(-\frac{|x-\mu|}{b} \right). \end{aligned}

(The Laplace distribution is sometimes known as the double exponential distribution, since each tail corresponds to that for an exponential random variable.) Note that if we can show that the Laplace distribution with mean 0 is a mixture of normals, then by shifting all these normals by \mu, it follows that the Laplace distribution with mean \mu is also a mixture of normals.

Fix b > 0. Let W \sim \text{Exp}(\frac{1}{2b^2}), i.e. f_W(w) = \dfrac{1}{2b^2} \exp \left(-\dfrac{x}{2b^2} \right) for w \geq 0, and let X \mid W = w \sim \mathcal{N}(0, w). We claim that X has the Laplace distribution with mean 0 and scale b. This is equivalent to showing that for any x,

\begin{aligned} f_X(x) &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right), \\ \Leftrightarrow \quad \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right). \end{aligned}

The key to showing this identity is noticing that the integrand on the LHS looks very much like the probability density function (PDF) of the inverse Gaussian distribution. Letting \lambda = |x|^2, \mu = |x|b and Z be an inverse Gaussian random variable with mean \mu and shape \lambda,

\begin{aligned} \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{x^2}{2w} \right) \frac{1}{2b^2} \exp \left( - \frac{w}{2b^2} \right) dw \\ &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{w^2 + |x|^2b^2}{2b^2 w} \right) dw \\ &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{(w - |x|b)^2}{2b^2 w} - \frac{2w|x|b}{2b^2w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{|x|^2(w - |x|b)^2}{2|x|^2b^2 w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{\sqrt{\lambda}} \int_0^\infty w \frac{\sqrt{\lambda}}{\sqrt{2\pi w^3}} \exp \left( - \frac{\lambda(w - \mu)^2}{2\mu^2 w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{|x|} \mathbb{E}[Z] \\ &= \frac{1}{2|x|b^2} e^{-|x|/b} \mu \\ &= \frac{1}{2b} e^{-|x|/b}. \end{aligned}

4 thoughts on “Laplace distribution as a mixture of normals”

  1. Pingback: The Bayesian lasso | Statistical Odds & Ends

Leave a comment