1. Do we then use for event probs?

2. And what about mixtures, like spike and slab or Dirichlet process which are part discrete and part continuous?

3. The objection to the cap/lowercase convention is about matrices and to a lesser extent Greek letters. If we also want to capitalize matrices or bold them, we run into conflicting conventions. Not all the Greek letters have easily distinguishable caps.

3. While writing may clear up some confusions, it also runs head-on into the notation used for events, where is shorthand for the event . And obviously if we’re talking events and is continuous.

4. I don’t see how using conditionals cleans up the distinction between and , which would be written in probability theory as and . This notation gets cumbersome when we have a dozen parameters.

I’ve never heard anyone say that the problem is purely Bayesian or frequentist — this is just about probability theory, about which everyone is in agreement. The frequentist/Bayesian debate is about what can be the object of a probability distribution, not how the laws of probability work.

It’s basis is simple: p(x), or P(x), is an object which does not exist. The correct tool in probabilities is conditional probabilities: one should always specify the preliminary knowledge that conditions the state of uncertainty about a quantity x. Therefore, p(X | c) is the distribution about variable X under the assumption that c holds. If preliminary knowledge is different, say c’, then p(X | c’) can be another mathematical distribution. Function p(. | .), in this case, is not overloaded (contrary to what many reviewers of my papers asserted, but that’s not the point. :) ).

Complement this with a few conventions, for instance, use p(. | .) in the continuous case, P(. | .) in the discrete case. Use capitalized symbols when referring to variables (i.e., domains), and small-capped symbols for values. So that P([X=x] | c) (or P(X=x | c), when you’re lazy) is a probability value, whereas P(X | c) is a probability distribution. And you should be all set.

The use of right-hand side symbols to make probability distributions non-ambiguous does not even need to be tied to “subjectivist” stances about the meaning of probabilities as states of knowledge of agents. Indeed, it is also the basis of (purely Bayesian statistics or machine learning inspired) methods of model selection: both P(X | c) and P(X | c’) being defined, a new variable C can then be introduced, with domain C={c, c’}, and the model P(C | D) P(X | C D) can then be introduced to carry out model selection by computing P(C | X D).

I have read way too many papers where, for instance, Normal distributions were happily applied over R+, or even over circular spaces. Consider, oh, “almost” all of the robotics navigation and mapping literature of the 90′s: positions x, y and orientations \theta of robots were put together in a single “pose” variable, a linear Gaussian models were applied directly on this 3D variable… :)

I’m afraid it doesn’t make sense to talk about the independence of random variables over different sample spaces. Keep in mind that it’s OK to have different outcomes in the same sample space — the sample space itself is abstract. For instance, if you have two discrete distros with outcomes {A,B} and {X,Y,Z}, then you can have a sample space with six points, {AX, AY, AZ, BX, BY, BZ}. The value of the first variable is A for samples AX, AY and AZ, and B for samples BX, BY and BZ. Similar, the value of the second variable is X for AX and BX, and so on.

For instance, writing independence out in full, variables and over a sample space are independent if and only if for all . The reason you need this to be over a single sample space is that you can’t define the joint distribution otherwise.

In practice, the sample space is rarely mentioned and random variables are defined by their distributions. It’s just assumed everything comes from the same sample space. Often you define by first defining and and then stating that they’re independent.

Thanks for the wordpress/latex tip :)

In practical modeling papers, where there are parameters, matrices, etc., the upper-case/lower-case thing gets difficult to maintain. And it’s so rare to see anything other than that it seems awfully pedantic to include all those subscripts.

The real kicker is that you almost never see random variables defined as maps . In fact, you never see the sample space even mentioned. Instead, you’ll see statements like “assume is a random variable distributed as .”

This seems less heinous than . At least, I think it’s more natural to imply bindings from underlying density measures than vice versa…

(sorry if this doesn’t come out nicely latex-ified—I’m no wordpress expert) [ed. I added the latex escape for you; all you needed was to put latex after the $ and before the first bit of LaTeX.]