Measure Theory in Two Definitions

Measure theory generalizes the notion of length, area and volume to a very general topological setting. It also forms the basis of probability theory.

Definition 1

A $\sigma$ -algebra is a set $\mathcal S$ of sets which

is closed under difference, $A - B \in {\mathcal S}$ if $A, B \in \mathcal S$ ,
is closed under countable unions, $\bigcup_{n = 0}^{\infty} A_n \in \mathcal S$ if $A_n \in \mathcal S$ , and
has a unit $\mbox{I} \in \mathcal S$ such that $A \cap \mbox{I} = A$ for all $A \in \mathcal S$ .

Definition 2

A $\sigma$ -additive measure is a function $\mu$ over a $\sigma$ -algebra $\mathcal S$ taking non-negative values such that

$\mu(\bigcup_{n=0}^{\infty} A_n) = \sum_{n=0}^{\infty} \mu(A_n)$ for pairwise disjoint sets $A_n \in \mathcal S$ .

Immediate Consequences

The closure of a $\sigma$ -algebra under countable intersections follows from de Morgan’s law,

$\bigcap_{n=0}^{\infty} A_n = \mbox{I} - \bigcup_{n = 0}^{\infty} (\mbox{I} - A_n)$ .

The empty set $\emptyset = \mbox{I} - \mbox{I} \in \mathcal S$ is the unique zero element such that $\emptyset \cap A = \emptyset$ for all $A \in \mathcal S$ . Further, $\emptyset \cup A = A$ and $\mbox{I} \cup A = \mbox{I}$ for all $A \in \mathcal S$ .

A $\sigma$ -algebra forms a ring, with $\cap$ as addition, $\cup$ as multiplication, $\emptyset$ as the multiplicative identity, $\mbox{I}$ as the additive identity, and $-A = \mbox{I} - A$ as additive difference.

Constructing Lebesgue Measures

It’s not entirely trivial to show that interesting measures exist, even working over the real numbers. The easiest useful measure to construct is the Lebesgue measure over the unit interval $(0,1)$ .

Start by defining the measure of a subinterval to be its length. Extend the definition to countable unions of subintervals by summation, following the definition of measure.

It remains to tidy up the some boundary cases producing zero-measure results, which isn’t entirely trivial (analysis is built on counterexamples and edge cases), but the basic idea pretty much works as is.

This entry was posted on December 10, 2009 at 8:43 pm and is filed under Carp's Blog, Statistics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

9 Responses to “Measure Theory in Two Definitions”

John Says:
December 10, 2009 at 11:29 pm | Reply
Were you planning on this being the first in a series of blog posts on measure theory? If so, I think that would be interesting. It’s easy to find technical books on measure theory, but it’s not so easy to find informal expository articles like your post.

It would be interesting for someone to write about how elementary and advanced probability notions relate. They can look so different, for example in the definitions of conditional expectation, that it’s not obvious they’re talking about the same thing.
- lingpipe Says:
  December 11, 2009 at 12:30 am | Reply
  I have one more on random variables. Which is how this all got started. I was just trying to track down a precise definition of random variables for an introduction to Bayesian classifiers I’m putting together.
  
  Little did I realize it’d lead back to Lebesgue integrals! The bane of my undergrad math degree. I was terrible at analysis.
  
  But I loved topology and set theory. So it’s actually more comfortable to me to crank up the abstraction level beyond the real numbers.
  
  I almost always write what I think I would’ve wanted to have read before I started digging. But I’m afraid this might not be so helpful for anyone who doesn’t already know the material pretty well. Sort of like unix man pages.
  - John Says:
    December 11, 2009 at 7:19 am
    I had a solid understanding of measure theory and probability before I took measure-theoretic probability. I thought it would be a breeze since it was simply combining two things I already understood. But the synthesis was harder than I expected. I could put on my measure theory hat and think “this is just analysis with different vocabulary” and prove theorems, but then I’d start to wonder “but what does this say about randomness?” It was like some quantum effect where I could think like a wave or a particle but I couldn’t hold both in my head at the same time.
Q Says:
December 11, 2009 at 4:31 am | Reply
Why not just read Billingsley’s ‘Probability and Measure’ ? Its lengthy, but really quite clear if enough time is spent.
- lingpipe Says:
  December 11, 2009 at 11:56 am | Reply
  I think the main reason is that the audience I’d like to write an intro for isn’t going to sit down and read a measure theory book. I really liked Kolmogorov and Folmin’s Introductory Real Analysis.
doug Says:
December 13, 2009 at 1:03 pm | Reply
Why the odd definition of a sigma algebra, with no mention of the underlying space? The last condition implies that every member of the sigma algebra is a subset of I, so your conditions are equivalent to the usual ones (a nonempty collection of subsets of I that is closed under complements and countable unions).
- lingpipe Says:
  December 14, 2009 at 12:05 am | Reply
  Would you believe I just didn’t think about it? I was following Kolmogorov and Fomin’s definitions. When they abstract from measures on the plane to general measures, the underlying space is implicit.
  
  I think there’s a deeper issue about just how explicit all the definitions should be in terms of domains and ranges of things, and how much should be implicit. Kolmogorov and Fomin leave almost everything implicit. I really like that style, which surprised me, because usually I find laying everything out explicitly makes me happer.
  
  I wound up introducing the underlying points from which the sets are drawn in order to talk about sample spaces in the next post.
CD Says:
December 25, 2009 at 5:42 pm | Reply
Why not just read Billingsley’s ‘Probability and Measure’ ? Its lengthy, but really quite clear if enough time is spent.
- lingpipe Says:
  December 28, 2009 at 12:12 pm | Reply
  Thanks for the recommendation. I’ve found all of the discussions of measure theory I’ve read pretty clear, and I really liked Kolmogorov’s (because it was so algebraic and topological).
  
  My real problem is that I’m trying to define random variables in an intro to Bayesian categorical data analysis for readers who most likely don’t know analysis.
  
  Even in intro math stats texts (e.g. Larsen and Marx or DeGroot and Schervish), you don’t get a proper definition of measure theory, just some examples.

LingPipe Blog