The short answer is: there is no real differences between a MaxEnt model and a logistic regression. They are both log linear models. And now, the long answer: The logistic regression is a probabilistic model for binomial cases. The MaxEnt generalizes t…

]]>(Aside: f should be familiar, though what usually shows up is df/dx.)

]]>Sorry I’m so slow with all this arithmetic precision stuff; I should’ve definitely put exp(2) and exp(2) together in this case.

]]>For everyone else who’s not knee deep in these derivations, the binary case is:

log(exp(a) + exp(b)) = a + log(1 + exp(b-a))

and this is where the special calculation for log(1+x) comes into play.

Check out John Cook’s discussion of log(1+x), which uses a Taylor approximation for x < 1e-4 (that is, x < 0.0001), and simply makes a lib call otherwise.

]]>