In the canonical example of a biased coin flip, the outcome (heads or tails) of each trial is observable. The chance of landing heads or tails is not observable. In the example of human height, the height of a person is observable, but the mean height of the population is not.

In practice, this leads to the frequentist notion of hypothesis testing. A hypothesis test can either reject a hypothesis (such as that height and weight are independent) or fail to reject a hypothesis. They do not allow you to conclude that the alternative to the null hypothesis is correct! In probabilistic terms, the null hypothesis is usually a point hypothesis with zero probability of being correct in most continuous models (such as chance of a coin landing heads or a normally distributed population of women).

In practice, many people incorrectly interpret frequentist confidence intervals with Bayesian posterior intervals.

People misleadingly call Bayesian stats subjective, but this is a misunderstanding of the role of modeling and a misunderstanding of estimation. All of statistics, even frequentist statistics, is subjective in the sense that we’re using a mathematical model to approximate an unknown reality. Priors in the Bayesian sense may be estimated from data just like other parameters.

Back on topic, I liked this brief synopsis of Bayesian stats. I recall the week that we spent on this in Stats I as the reason I continued my studies in the field of statistics. Such an elegant, powerful tool inspired me to really dig in to probability theory as an undergrad.

Thanks again!

I’d rather use a built-in HTML-based solution, but WordPress is hosting and I haven’t figured out how to incorporate one of the LaTeX plugins. Any help appreciated!

P.S. If you right-click on images and look at their properties, you’ll see the link to sitmo.

Just curious, what are you using to format your mathematics? It looks like html-based LaTeX. Is that what it is? Looks very nice.

The criticisms were sound — it’s too technical (i.e. jargon filled) for someone to understand who doesn’t already get it. Ironically, I’ve been telling Andrew Gelman that about his Bayesian Data Analysis book for years.

Unix man pages are the usual exemplar of doc that only works if you mostly know the answer. They’re great once you already understand something, but terrible for learning.

I think Andrew’s BDA is that way — it’s clear, concise and it actually does explain everything from first principles. And there are lots of examples. So why is this so hard to understand?

I usually write with my earlier self in mind as an audience. Sorry for not targeting a far-enough back version of myself this time! The jargon should be familiar to anyone who’s taken math stats. I don’t think it’d have helped if I’d have defined the sum for the prior defined as a marginal.

His educational software on MCMC are also excellent teaching tools. An example lecture of his on Bayesian statistics and how they are used in phylogenetic analysis can be found here: http://www.molecularevolution.org/si/people/faculty/lewis_paul.php