Laplace distribution as a mixture of normals

Previously on this blog we showed that the t-distribution can be expressed as a continuous mixture of normal distributions. Today, I learned from this paper that the Laplace distribution can be viewed as a continuous mixture of normal distributions as well.

The Laplace distribution with mean $\mu \in \mathbb{R}$ and scale $b > 0$ has the probability density function

$\begin{aligned} f(x) = \frac{1}{2b} \exp \left(-\frac{|x-\mu|}{b} \right). \end{aligned}$

(The Laplace distribution is sometimes known as the double exponential distribution, since each tail corresponds to that for an exponential random variable.) Note that if we can show that the Laplace distribution with mean $0$ is a mixture of normals, then by shifting all these normals by $\mu$ , it follows that the Laplace distribution with mean $\mu$ is also a mixture of normals.

Fix $b > 0$ . Let $W \sim \text{Exp}(\frac{1}{2b^2})$ , i.e. $f_W(w) = \dfrac{1}{2b^2} \exp \left(-\dfrac{x}{2b^2} \right)$ for $w \geq 0$ , and let $X \mid W = w \sim \mathcal{N}(0, w)$ . We claim that $X$ has the Laplace distribution with mean $0$ and scale $b$ . This is equivalent to showing that for any $x$ ,

$\begin{aligned} f_X(x) &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right), \\ \Leftrightarrow \quad \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \frac{1}{2b} \exp \left(-\frac{|x|}{b} \right). \end{aligned}$

The key to showing this identity is noticing that the integrand on the LHS looks very much like the probability density function (PDF) of the inverse Gaussian distribution. Letting $\lambda = |x|^2$ , $\mu = |x|b$ and $Z$ be an inverse Gaussian random variable with mean $\mu$ and shape $\lambda$ ,

$\begin{aligned} \int_0^\infty f_{X \mid W = w}(x) f_W(w) dw &= \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{x^2}{2w} \right) \frac{1}{2b^2} \exp \left( - \frac{w}{2b^2} \right) dw \\ &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{w^2 + |x|^2b^2}{2b^2 w} \right) dw \\ &= \frac{1}{2b^2} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{(w - |x|b)^2}{2b^2 w} - \frac{2w|x|b}{2b^2w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \int_0^\infty \frac{1}{\sqrt{2\pi w}} \exp \left( - \frac{|x|^2(w - |x|b)^2}{2|x|^2b^2 w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{\sqrt{\lambda}} \int_0^\infty w \frac{\sqrt{\lambda}}{\sqrt{2\pi w^3}} \exp \left( - \frac{\lambda(w - \mu)^2}{2\mu^2 w} \right) dw \\ &= \frac{1}{2b^2} e^{-|x|/b} \frac{1}{|x|} \mathbb{E}[Z] \\ &= \frac{1}{2|x|b^2} e^{-|x|/b} \mu \\ &= \frac{1}{2b} e^{-|x|/b}. \end{aligned}$

3 thoughts on “Laplace distribution as a mixture of normals”

Pingback: The Bayesian lasso | Statistical Odds & Ends
ya wei on January 26, 2019 at 2:59 pm said:

Hi, there is a small typo in the formula $W\sim \mathrm{Exp}(\frac{1}{2b^2})$ where you forget the square of `b`.

LikeLike

Reply ↓
- kjytay on January 26, 2019 at 3:18 pm said:
  
  Thanks, I’ve fixed it!
  
  LikeLike
  
  Reply ↓

Share this:

Like this:

Related

3 thoughts on “Laplace distribution as a mixture of normals”

Leave a Reply Cancel reply