I recently learnt of the “noise outsourcing” theorem from Teh Yee Whye‘s IMS Medallion Lecture at the recent Joint Statistical Meetings (JSM). Here is the statement of the theorem from the talk:
Theorem (Noise Outsourcing). If
and
are random variables in “nice” (e.g. Borel) spaces
and
, then there is a random variable
which is independent of
such that
In particular, if there is a statistic
with
, then
The arXiv paper tells us how we can interpret this theorem: for any two random variables in “nice” spaces, there is a generative functional representation of the conditional distribution 
Apparently noise outsourcing is a “standard technical tool from measure theoretic probability”, and it appears in Kallenberg’s Foundations of Modern Probability, a commonly used graduate text in probability.
References:
- Teh, Y. W. On Statistical Thinking in Deep Learning (Slide 34).
- Bloem-Reddy, B., and Teh, Y. W. (2019). Probabilistic symmetry and invariant neural networks (Section 3.1).
