I introduced measure theory in my last post, Measure Theory in Two Definitions. With that out of the way, we can move on to probability measures and random variables.
A probability measure is a measure taking values in the closed interval and mapping the unit to 1, . It follows that .
In a probability measure, the sets are called events and we write for .
The sample space for a probability measure is .
Those Pesky Random Variables
Given a probability measure , a random variable is a real-valued function over the measure’s sample space such that (so that it has a measure) for all .
The cumulative density function for is defined by
The probability that random variable is less than a fixed value is
The probability that the variable falls in the interval is defined by
When is differentiable, the density function is
When the density is defined,
End of Story
This still seems like a lot of work just to get going with random variables. We haven’t even defined multivariate densities or conditional and joint probabilities.
In applied work, we define integrable multivariate density functions explicitly and reason through integration. For instance, in the hierarchical batting ability model I discussed as an example of Bayesian inference, the model is fully described by the density
Technically, we could construct the measure from the density function as follows. First, construct the sample space from which the parameters are drawn. Continuing the baseball example, we draw our parameter assignment from the sample space
We then take the Lebesgue measurable events with measure defined by (multivariate) Lebesgue-Stieltjes integration,
where, of course, . If is a simple hypercube in , this works out to the usual sum/integral: