estimating mixtures by polynomials

Sida Wang, Arun Tejasvi, and Chaganty Percy Liang have just arXived a paper about using the method of moments to estimate mixtures of distributions. Method that was introduced (?) by Pearson in 1894 for a Gaussian mixture and crab data. And studied in fair details by Bruce Lindsay and his co-authors, including his book, which makes it the more surprising that Bruce’s work is not mentioned at all in the paper. In particular the 1989 Annals of Statistics paper which connects the number of components with the rank of a moment matrix in exponential family and which made a strong impression on me at the time, just when I was starting to work on mixtures. The current paper addresses more specifically the combinatoric difficulty of solving the moment equation. The solution proceeds via a relaxed convex optimisation problem involving a moment matrix, the relaxation removing the rank condition that identifies the parameters of the mixture. While I am no expert in the resolution of the associated eigenvalue problem (Algorithm 1), I wonder at (i) the existence and convergence of a solution when using empirical moments. And (ii) the impact of the choice of the moment equations, on both existence and efficiency of the moment method. It is clearly not invariant by reparameterisation, hence parameterisation matters. It is even unclear to me how many terms should be used in the resolution: if a single dimension is acceptable, determining this dimension may prove a complex issue.

mixtures, Heremite polynomials, and ideals

A 3 page note that got arXived today is [University of Colorado?!] Andrew Clark’s “Expanding the Computation of Mixture Models by the use of Hermite Polynomials and Ideals“. With a typo on Hermite‘s name in the pdf title. The whole point of the note is to demonstrate that mixtures of different types of distributions (like t and Gaussian) are manageable.  A truly stupendous result… As if no one had ever mixed different distributions before.

“Using Hermite polynomials and computing ideals allows the investigator to mix distributions from distinct families.”

The second point of the paper is to derive the mixture weights from an algebraic equation based on the Hermite polynomials of the components, which implies that the components and the mixture distribution itself are already known. Which thus does not seem particularly relevant for mixture estimation…