Laplace distribution as a mixture of normals

Previously on this blog we showed that the t-distribution can be expressed as a continuous mixture of normal distributions. Today, I learned from this paper that the Laplace distribution can be viewed as a continuous mixture of normal distributions as well.

The Laplace distribution with mean \mu \in \mathbb{R} and scale b > 0 has the probability density function

\begin{aligned} f(x) = \frac{1}{2b} \exp \left(-\frac{|x-\mu|}{b} \right). \end{aligned}

(The Laplace distribution is sometimes known as the double exponential distribution, since each tail corresponds to that for an exponential random variable.) Note that if we can show that the Laplace distribution with mean 0 is a mixture of normals, then by shifting all these normals by \mu, it follows that the Laplace distribution with mean \mu is also a mixture of normals.

Fix b > 0. Let W \sim \text{Exp}(\frac{1}{2b^2}), i.e. f_W(w) = \dfrac{1}{2b^2} \exp \left(-\dfrac{x}{2b^2} \right) for w \geq 0, and let X \mid W = w \sim \mathcal{N}(0, w). We claim that X has the Laplace distribution with mean 0 and scale b. This is equivalent to showing that for any x,

\begin{aligned} f_X(x) &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right), \\  \Leftrightarrow \quad \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right). \end{aligned}

The key to showing this identity is noticing that the integrand on the LHS looks very much like the probability density function (PDF) of the inverse Gaussian distribution. Letting \lambda = |x|^2, \mu = |x|b and Z be an inverse Gaussian random variable with mean \mu and shape \lambda,

\begin{aligned} \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{x^2}{2w} \right) \frac{1}{2b^2} \exp \left( - \frac{w}{2b^2} \right) dw \\  &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{w^2 + |x|^2b^2}{2b^2 w} \right) dw \\  &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{(w - |x|b)^2}{2b^2 w} - \frac{2w|x|b}{2b^2w} \right) dw \\  &= \frac{1}{2b^2} e^{-|x|/b} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{|x|^2(w - |x|b)^2}{2|x|^2b^2 w} \right) dw \\  &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{\sqrt{\lambda}} \int_0^\infty w \frac{\sqrt{\lambda}}{\sqrt{2\pi w^3}} \exp \left( - \frac{\lambda(w - \mu)^2}{2\mu^2 w} \right) dw \\  &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{|x|} \mathbb{E}[Z] \\  &= \frac{1}{2|x|b^2} e^{-|x|/b} \mu \\  &= \frac{1}{2b} e^{-|x|/b}. \end{aligned}

Advertisements

3 thoughts on “Laplace distribution as a mixture of normals

  1. Pingback: The Bayesian lasso | Statistical Odds & Ends

Leave a Reply to kjytay Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s