I was browsing through the doc for the Eigen matrix package (C++ templated, which I need for automatic differentiation) and they had a pointer to the following extended (70+ page) matrix “cheat sheet”:
- K. B. Petersen and M. S. Pedersen. 2008. The Matrix Cookbook. Technical Report.
It contains all kinds of useful definitions and derivations, most relevantly for me, gradients of all the matrix operations (inverses, determinants, traces, eigenvalues). They even have the gradient for the multivariate normal density function (see section 8), which has a pleasingly simple form. Now if I could just get them to do the Wishart and multivariate-t, which they define, but don’t differentiate.
And how great is that pair of author names, “Petersen” and “Pedersen”? They’re reminiscent of Tintin’s Thomson and Thompson.
The authors even have a blog about The Matrix Cookbook, though it hasn’t been updated in almost a year.