Kolmogorov’s strong law of large numbers

The strong law of large numbers (SLLN) is usually stated in the following way:

Theorem: For $X_1, X_2, \dots$ such that the $X_i$‘s are independent and identically distributed (i.i.d.) with finite mean $\mu$, as $n \rightarrow \infty$,

\begin{aligned} \frac{X_1 + X_2 + \dots + X_n}{n} \stackrel{a.s.}{\rightarrow} \mu. \end{aligned}

What if the $X_i$‘s are independent but not identically distributed? Can we say anything in that setting? We can if we add a condition on the sum of the variances of the $X_i$‘s. This is sometimes known as Kolmogorov’s strong law of large numbers or the Kolmogorov criterion.

Theorem: Assume that $X_1, X_2, \dots$ are independent with means $\mu_1, \mu_2, \dots$ and variances $\sigma_1^2, \sigma_2^2, \dots$ such that $\displaystyle\sum_{k=1}^\infty \dfrac{\sigma_k^2}{k^2} < \infty$. Then

\begin{aligned} \frac{X_1 + X_2 + \dots + X_n - (\mu_1 + \mu_2 + \dots + \mu_n)}{n} \stackrel{a.s.}{\rightarrow} 0. \end{aligned}

References:

1. WolframMathWorld. Strong Law of Large Numbers.