how many modes in a normal mixture?

cover of Mixture Estimation and ApplicationsAn interesting question I spotted on Cross Validated today: How to tell if a mixture of Gaussians will be multimodal? Indeed, there is no known analytical condition on the parameters of a fully specified k-component mixture for the modes to number k or less than k… Googling around, I immediately came upon this webpage by Miguel Carrera-Perpinan, who studied the issue with Chris Williams when writing his PhD in Edinburgh. And upon this paper, which not only shows that

  1. unidimensional Gaussian mixtures with k components have at most k modes;
  2. unidimensional non-Gaussian mixtures with k components may have more than k modes;
  3. multidimensional mixtures with k components may have more than k modes.

but also provides ways of finding all the modes. Ways which seem to reduce to using EM from a wide variety of starting points (an EM algorithm set in the sampling rather than in the parameter space since all parameters are set!). Maybe starting EM from each mean would be sufficient.  I still wonder if there are better ways, from letting the variances decrease down to zero until a local mode appear, to using some sort of simulated annealing…

Edit: Following comments, let me stress this is not a statistical issue in that the parameters of the mixture are set and known and there is no observation(s) from this mixture from which to estimate the number of modes. The mathematical problem is to determine how many local maxima there are for the function

f(x)\,:\,x \longrightarrow \sum_{i=1}^k p_i \varphi(x;\mu_i,\sigma_i)

6 Responses to “how many modes in a normal mixture?”

  1. Georges Henry Says:

    Où va se loger l’ exhibitionnisme de l’écrivain d’internet, au point de répondre à côté…

  2. My favourite thing that I’ve seen recently was a “non-local prior” solution, which can basically be interpreted as putting a repellent point process on the means (to make it tractable it’s actually a deterministic PP conditioned on having k points [actually, they didn’t derive it that way, but that’s what it turns out to be]), which should go a long way towards killing these spurious modes. (ignoring, of course, label switching)

    I like the idea of committing to an interpretation (clusters WILL BE SEPARATE) and using a prior to enforce that, rather than just let these modes form. Or, to put it differently, I don’t know how to interpret the individual components of a mixture model without some sort of “separating” condition. (The sum is always interpretable as a “non-parameteric” model for the density, but I would suspect that the sum has fewer modes as all roads lead to Rome)

    • Dan: my writing skills were never that great but they must have been dropping even further: the question is about the genuine number of modes in a given mixture density, so there is no data, no uncertainty, no prior, no statistician involved! This is pure calculus, in a way, except that there is no formula that tells how many modes result from a certain collection of parameters in a k-component mixture. And hence the need to resort to numerical analysis or simulation…

  3. Alan J. Izenman Says:

    The mixtures paper makes no mention of my Applications paper in JASA in which I examine Silverman’s nonparametric test for multimodality and compare its performance against a parametric test. The reference is:

    “Philatelic Mixtures and Multimodal Densities,” JASA, 83 (1988), 941-953.

    The data set used is fascinating and has a great story attached to it.

    The analysis can also be found in Section 4.5.3 in my book,

    “Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning,” Springer, 2013.

    I’m surprised that it was ignored in the paper.

    • Dear Alan: Thank you for pointing out your paper and this great Mexican stamp dataset! I am afraid I was not clear enough in this post: the question is not to estimate the number of modes based on a sample of observations, but to determine the number of modes of a given Gaussian mixture with all parameters known and set. This is thus a purely mathematical problem.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s