Generating random draws from the Dir(1, 1, …, 1) distribution using the uniform distribution

Fix an integer K \geq 2 and parameters \alpha_1, \dots, \alpha_K > 0, commonly written as a vector \boldsymbol{\alpha} = (\alpha_1, \dots, \alpha_K). \boldsymbol{X} = (X_1, \dots, X_K) has the Dirichlet distribution (we write \boldsymbol{X} \sim \text{Dir}(\boldsymbol{\alpha})) if it has support on the probability simplex \left\{ (x_1, \dots, x_K) \in \mathbb{R}^K: x_i \geq 0 \text{ and } \sum_{i=1}^K  = 1 \right\} and its probability density function (PDF) satisfies

\begin{aligned} f(x_1, \dots, x_K) &\propto \prod_{i=1}^K x_i^{\alpha_i - 1}. \end{aligned}

The \text{Dir}(1, 1, \dots, 1) distribution is special because its PDF is constant over its support:

\begin{aligned} f(x_1, \dots, x_K) \propto 1. \end{aligned}

The following lemma demonstrates one way to generate random draws from the \text{Dir}(1, 1, \dots, 1) distribution:

Lemma. Let U_1, \dots, U_{K-1} \sim \text{Unif}[0,1], and let U_{(1)}, \dots, U_{(K-1)} denote the order statistics for U_1, \dots, U_{K-1}. Then

\begin{aligned} \left( U_{(1)} - 0, U_{(2)} - U_{(1)}, \dots, 1 - U_{(K-1)} \right)  \sim \text{Dir}(1, 1, \dots, 1). \end{aligned}

(See Section 2 of Reference 1 for various ways to generate samples from the Dirichlet distribution for arbitrary \boldsymbol{\alpha}.)

The proof of the lemma starts with the following theorem, which gives the joint density of all the order statistics:

Theorem. Let X_1, \dots, X_n be i.i.d. random variables with PDF f. Then the joint density of the order statistics X_{(1)}, \dots, X_{(n)} is

\begin{aligned} f_{X_{(1)}, \dots, X_{(n)}}(y_1, \dots, y_n) = n! f(y_1) \dots f(y_n) 1 \{ y_1 < y_2 < \dots < y_n \}. \end{aligned}

(This is Theorem 6.1 of Reference 2; you can find a proof for this theorem there.) Applying this theorem to U_1, \dots, U_n, we have

\begin{aligned} f_{U_{(1)}, \dots, U_{(K-1)}}(y_1, \dots, y_{K-1}) = (K-1)! 1 \{ 0 \leq y_1 < y_2 < \dots < y_n \leq 1 \}. \end{aligned}

The lemma then follows by applying a change of variables to the PDF (see this link for the steps one needs to carry out for the change of variables).

References:

  1. Frigyik, B. A., et. al. (2010). Introduction to the Dirichlet Distribution and Related Processes.
  2. DasGupta, A. Finite Sample Theory of Order Statistics and Extremes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s