Kolmogorov’s strong law of large numbers

The strong law of large numbers (SLLN) is usually stated in the following way:

Theorem: For $X_1, X_2, \dots$ such that the $X_i$ ‘s are independent and identically distributed (i.i.d.) with finite mean $\mu$ , as $n \rightarrow \infty$ ,

$\begin{aligned} \frac{X_1 + X_2 + \dots + X_n}{n} \stackrel{a.s.}{\rightarrow} \mu. \end{aligned}$

What if the $X_i$ ‘s are independent but not identically distributed? Can we say anything in that setting? We can if we add a condition on the sum of the variances of the $X_i$ ‘s. This is sometimes known as Kolmogorov’s strong law of large numbers or the Kolmogorov criterion.

Theorem: Assume that $X_1, X_2, \dots$ are independent with means $\mu_1, \mu_2, \dots$ and variances $\sigma_1^2, \sigma_2^2, \dots$ such that $\displaystyle\sum_{k=1}^\infty \dfrac{\sigma_k^2}{k^2} < \infty$ . Then

$\begin{aligned} \frac{X_1 + X_2 + \dots + X_n - (\mu_1 + \mu_2 + \dots + \mu_n)}{n} \stackrel{a.s.}{\rightarrow} 0. \end{aligned}$