**T**oday I refereed a paper where the authors used ABC to bypass convergence (and implementation) difficulties with their MCMC algorithm. And I am still pondering whether or not this strategy makes sense. If only because ABC needs to handle the same complexity and the same amount of parameters as an MCMC algorithm. While shooting “in the dark” by using the prior or a coarse substitute to the posterior. And I wonder at the relevance of simulating new data when the [true] likelihood value [at the observed data] can be computed. This would sound to me like the relevant and unique “statistics” worth considering…

## Archive for the Statistics Category

## running ABC when the likelihood is available

Posted in Statistics with tags ABC, intractable likelihood, latent variable models, MCMC, MCMC convergence, refereeing on September 19, 2017 by xi'an## Le Monde puzzle [#1021]

Posted in Books, Kids, R with tags binary integer linear programming, competition, Le Monde, linear programming, mathematical puzzle, R on September 17, 2017 by xi'an**A** puzzling Le Monde mathematical puzzle for which I could find no answer in the allotted time!:

*A most democratic electoral system allows every voter to have at least one representative by having each of the N voters picking exactly m candidates among the M running candidates and setting the size n of the representative council towards this goal, prior to the votes. If there are M=25 candidates, m=10 choices made by the voters, and n=10 representatives, what is the maximal possible value of N? And if N=55,555 and M=33, what is the minimum value of n for which m=n is always possible?
*

**I** tried a brute force approach by simulating votes from N voters at random and attempting to find the minimal number of councillors for this vote, which only provides an upper bound of the minimum [for one vote], and a lower bound in the end [over all votes]. Something like

for (i in 1:N) votz[i,]=sample(1:M,n) #exploration by majority remz=1:N;conz=NULL while (length(remz)>0){ seatz=order(-hist(votz[remz,], breaks=(0:M)+0.5,plot=FALSE)$density)[1] conz=c(conz,seatz);nuremz=NULL for (v in remz) if (!(seatz%in%votz[v,])) nuremz=c(nuremz,v) remz=nuremz} solz=length(conz) #exploration at random kandz=matrix(0,N,M) for (i in 1:N) kandz[i,votz[i,]]=1 for (t in 1:1e3){ #random choice of councillors zz=sample(c(0,1),M,rep=TRUE) while (min(kandz%*%zz)!=1) zz=sample(c(0,1),M,rep=TRUE) solz=min(solz,sum(zz)) #random choice of remaining councillor per voter remz=1:N;conz=NULL while (length(remz)>0){ seatz=sample(votz[remz[1],],1) conz=c(conz,seatz);nuremz=NULL for (i in remz) if (!(seatz%in%votz[i,])) nuremz=c(nuremz,i) remz=nuremz} solz=min(solz,length(conz))} maxz=max(solz,maxz)}

which leads to a value near N=4050 for the first question, with 0% confidence… Obviously, the problem can be rephrased as a binary integer linear programming problem of the form

where A is the NxM matrix of votes and c is the vector of selected councillors. But I do not see a quick way to fix it!

## Le Chemin [featuring Randal Douc]

Posted in Books, pictures, Statistics, Travel, University life with tags Cambodia, film, French mathematicians, French movies, MCMC, Randal Douc on September 17, 2017 by xi'an**M**y friend and co-author Randal Douc is one of the main actors in the film Le Chemin that came out last week in French cinemas. Taking place in Cambodia and directed by Jeanne Labrune. I have not yet seen the film but will next week as it is scheduled in a nearby cinema (and only six in Paris!)… (Randal was also a main actor in Rithy Panh’s Un barrage contre le Pacifique, as well as the off-voice in the Oscar nominated Rithy Panh’s L’image manquante.) In connection with this new movie, Randal was interviewed in Allociné, the major French website on current movies. With questions about his future film and theatre projects, but none about his on-going maths research!!!

## zurück nach Wien

Posted in pictures, Running, Statistics, Travel, University life, Wines with tags Austria, Book, Die Zauberflöte, mixtures of distributions, opera, Vienna, Volksoper Wien, Wolfgang Amadeus Mozart, WU Wirtschaftsuniversität Wien on September 16, 2017 by xi'an**T**oday, I am travelling to Vienna for a few days, primarily for assessing a grant renewal for a research consortium federating most Austrian research groups on a topic for which Austria is a world-leader. (Sorry for being cryptic but I am unsure how much I can disclose about this assessment!) And taking advantage on being in Vienna, for a two-day editing session with Sylvia Früwirth-Schnatter and Gilles Celeux on our Handbook of mixtures analysis project. Which started a few years ago with another meeting in Vienna. And taking further advantage on being in Vienna, for an evening at the Volksoper, conveniently playing Die Zauberflöte!

## Le Monde puzzle [#1020]

Posted in Books, Kids, R with tags competition, Le Monde, liar puzzle, mathematical puzzle on September 15, 2017 by xi'an**A** collection of liars in this Le Monde mathematical puzzle:

*A circle of 16 liars and truth-tellers is such that everyone states that their immediate neighbours are both liars. How many liars can there be?**A circle of 12 liars and truth-tellers is such that everyone state that their immediate neighbours are one liar plus one truth-teller. How many liars can there be?**A circle of 8 liars and truth-tellers is such that four state that their immediate neighbours are one liar plus one truth-teller and four state that their immediate neighbours are both liars . How many liars can there be?*

**T**hese questions can easily be solved by brute force simulation. For the first setting, using 1 to code truth-tellers and -1 liars, I simulate acceptable configurations as

tabz=rep(0,16) tabz[1]=1 #at least one tabz[2]=tabz[16]=-1 for (i in 3:15){ if (tabz[i-1]==1){ tabz[i]=-1}else{ if (tabz[i+1]==-1){ tabz[i]=1}else{ if (tabz[i+1]==1){ tabz[i]=-1}else{ if (tabz[i-2]==-1){ tabz[i]=1}else{ tabz[i]=sample(c(-1,1),1) }}}}}

which produces 8, 9, and 10 as possible (and obvious) values.

The second puzzle is associated with the similar R code

tabz=sample(c(-1,1),12,rep=TRUE) rong=FALSE while (!rong){ for (i in sample(12)){ if (tabz[i-1+12*(i==1)]*tabz[i%%12+1]==-1){ tabz[i]=1}else{ tabz[i]=sample(c(-1,1),1)} } rong=TRUE for (i in (1:12)[tabz==1]) rong=rong&(tabz[i-1+12*(i==1)]*tabz[i%%12+1]==-1) if (rong){ for (i in (1:12)[tabz==-1]) rong=rong&(tabz[i-1+12*(i==1)]*tabz[i%%12+1]!=-1) }}

with numbers of liars (-1) either 12 (obvious) or 4.

The final puzzle is more puzzling in that figuring out the validating function (is an allocation correct?) took me a while, the ride back home plus some. I ended up with the following code that samples liars (-1) and thruth-seekers (1) at random, plus forces wrong and right answers (in 0,1,2) on these, and check for the number of answers of both types:

rong=FALSE while (!rong){ tabz=sample(c(-1,1),8,rep=TRUE) #truth tabz[1]=1;tabz[sample(2:8,1)]=-1 tt=(1:8)[tabz==1];lr=(1:8)[tabz==-1] statz=rep(0,8) #stmt statz[tt]=(tabz[tt-1+8*(tt==1)]*tabz[tt%%8+1]==-1)+ 2*(tabz[tt-1+8*(tt==1)]+tabz[tt%%8+1]==-2) #answering 0 never works statz[lr]=2*(tabz[lr-1+8*(lr==1)]*tabz[lr%%8+1]==-1)+ (tabz[lr-1+8*(lr==1)]+tabz[lr%%8+1]==-1)+ sample(c(1,2),8,rep=TRUE)[lr]* (tabz[lr-1+8*(lr==1)]+tabz[lr%%8+1]==1) rong=(sum(statz==1)==4)&(sum(statz==2)==4)}

with solutions 3, 4, 5 and 6.

## positions in North-East America

Posted in Kids, pictures, Statistics, Travel, University life with tags academic position, Cambridge, Canada, Harvard University, Massachusetts, professorship, Québec, Université de Montréal, USA on September 14, 2017 by xi'an**T**oday I received emails about openings in both Université de Montréal, Canada, and Harvard University, USA:

- Professor in Statistics, Biostatistics or Data Science at U de M, deadline October 30th, 2017, a requirement being proficiency in the French language;
- Tenure-Track Professorship in Statistics at Harvard University, Department of Statistics, details there.

## relabelling in Bayesian mixtures by pivotal units

Posted in Statistics with tags Bayesian Essentials with R, finite mixtures, label switching, relabelling, ResearchGate on September 14, 2017 by xi'an**Y**et another paper on relabelling for mixtures, when one would think everything and more has already be said and written on the topic… This one appeared in Statistics and Computing last August and I only became aware of it through ResearchGate which sent me an unsolicited email that this paper quoted one of my own papers. As well as Bayesian Essentials.

The current paper by

The next part of the paper compares this approach with seven other solutions found in the literature, from Matthew Stephens’ (2000) to our permutation reordering. Which does pretty well in terms of MSE in the simulation study (see the massive Table 3) while being much cheaper to implement than the proposed pivotal relabelling (Table 4). And which, contrary to the authors’ objection, *does not require* the precise computation of the MAP since, as indicated in our paper, the relative maximum based on the MCMC iterations can be used as a proxy. I am thus less than convinced at the improvement brought by this alternative…