mixtures, Heremite polynomials, and ideals

Posted in Books, Kids, Statistics, University life with tags , , , , on September 24, 2015 by xi'an A 3 page note that got arXived today is [University of Colorado?!] Andrew Clark’s “Expanding the Computation of Mixture Models by the use of Hermite Polynomials and Ideals“. With a typo on Hermite‘s name in the pdf title. The whole point of the note is to demonstrate that mixtures of different types of distributions (like t and Gaussian) are manageable.  A truly stupendous result… As if no one had ever mixed different distributions before.

“Using Hermite polynomials and computing ideals allows the investigator to mix distributions from distinct families.”

The second point of the paper is to derive the mixture weights from an algebraic equation based on the Hermite polynomials of the components, which implies that the components and the mixture distribution itself are already known. Which thus does not seem particularly relevant for mixture estimation…

reaching transcendence for Gaussian mixtures

Posted in Books, R, Statistics with tags , , , , on September 3, 2015 by xi'an “…likelihood inference is in a fundamental way more complicated than the classical method of moments.”

Carlos Amendola, Mathias Drton, and Bernd Sturmfels arXived a paper this Friday on “maximum likelihood estimates for Gaussian mixtures are transcendental”. By which they mean that trying to solve the five likelihood equations for a two-component Gaussian mixture does not lead to an algebraic function of the data. (When excluding the trivial global maxima spiking at any observation.) This is not highly surprising when considering two observations, 0 and x, from a mixture of N(0,1/2) and N(μ,1/2) because the likelihood equation $(x-\mu)\exp\{\mu^2\}-x+\mu\exp\{-\mu(2x-\mu)\}=0$

involves both exponential and algebraic terms. While this is not directly impacting (statistical) inference, this result has the computational consequence that the number of critical points ‘and also the maximum number of local maxima, depends on the sample size and increases beyond any bound’, which means that EM faces increasing difficulties in finding a global finite maximum as the sample size increases…