## MCMSki IV, Jan. 6-9, 2014, Chamonix (news #13)

Posted in Mountains, Statistics, Travel, University life with tags , , , , , , , , , , , , , , on December 9, 2013 by xi'an

Now, most poster abstracts have been received (or at least 63 of them),  even though newcomers can still send them to my wordpress address (if they realise the message gets posted immediately!, so the format Subject: firstname secondname (affiliation): title and text: abstract must be respected! No personal message or query please!). We have now above 200 registered participants, with all sessions remaining miraculously full (after a few permutations in the program).

S0 it is time to mention a wee bit of the “ski” side of MCMski. Chamonix has two types of ski passes, Chamonix Le Pass, and Mont Blanc Unlimited, the later allowing a wide access to the Mont Blanc area, up to 3800 meters and in France, Italy, and Switzerland, but presumably harder to exploit to the fullest on a 4 hour afternoon break. (You have to arrange renting skis and buying passes on your own! The conference centre may answer moderate queries but not make any booking.)  The temperature in the town of Chamonix is currently between -7 and 0 (centigrades), with ten centimetres of snow in town. All ski areas will be open by Dec. 21. If you plan to ski the Vallée Blanche from Aiguille du Midi, at 3800m, I strongly advise renting a guide for this ultimate skiing experience!

Big big big news: not only the ski race will take place on Wed., Jan. 08, afternoon, organised by ESF Chamonix, but Antonietta Mira managed to secure one or two pairs of skis for the winner(s) of the race! I doubt there will be other opportunities of that magnitude for winning a magnificent pair of skis made in Italy by Blossom skis. Thanks a lot to Anto!!! And to Blossom skis (whose collection includes a series called FreeTibet.)

## convergence speeds

Posted in pictures, Running, Statistics, Travel, University life with tags , , , , , , on December 5, 2013 by xi'an

While waiting for Jean-Michel to leave a thesis defence committee he was part of, I read this recently arXived survey by Novak and Rudolf, Computation of expectations by Markov chain Monte Carlo methods. The first part hinted at a sort of Bernoulli factory problem: when computing the expectation of f against the uniform distribution on G,

For x ∈ G we can compute f (x) and G is given by a membership oracle, i.e. we are able to check whether any x is in G or not.

However, the remainder of the paper does not get (in) that direction but recalls instead convergence results for MCMC schemes under various norms. Like spectral gap and Cheeger’s inequalities. So useful for a quick reminder, e.g. to my Monte Carlo Statistical Methods class Master students, but altogether well-known. The paper contains some precise bounds on the mean square error of the Monte Carlo approximation to the integral. For instance, for the hit-and-run algorithm, the uniform bound (for functions f bounded by 1) is

$9.5\cdot 10^{7}\dfrac{dr}{\sqrt{n}}+6.4\cdot 10^{15}\dfrac{d^2r^2}{n}$

where d is the dimension of the space and r a scale of the volume of G. For the Metropolis-Hastings algorithm, with (independent) uniform proposal on G, the bound becomes

$\dfrac{2C\alpha_dr^d}{n}+\dfrac{4C^2\alpha_d^2r^{2d}}{n^2}\,,$

where C is an upper bound on the target density (no longer the uniform). [I rephrased Theorem 2 by replacing vol(G) with the containing hyper-ball to connect both results, αd being the proportionality constant.] The paper also covers the case of the random walk Metropolis-Hastings algorithm, with the deceptively simple bound

$1089\dfrac{(d+1)\max\{\alpha,\sqrt{d+1}\}}{\sqrt{n}}+8.38\cdot 10^5\dfrac{(d+1)\max\{\alpha^2,d+1\}}{n}$

but this is in the special case when G is the ball of radius d. The paper concludes with a list of open problems.

## Importance sampling schemes for evidence approximation in mixture models

Posted in R, Statistics, University life with tags , , , , , , , , , on November 27, 2013 by xi'an

Jeong Eun (Kate) Lee and I completed this paper, “Importance sampling schemes for evidence approximation in mixture models“, now posted on arXiv. (With the customary one-day lag for posting, making me bemoan the days of yore when arXiv would give a definitive arXiv number at the time of submission.) Kate came twice to Paris in the past years to work with me on this evaluation of Chib’s original marginal likelihood estimate (also called the candidate formula by Julian Besag). And on the improvement proposed by Berkhof, van Mechelen, and Gelman (2003), based on averaging over all permutations, idea that we rediscovered in an earlier paper with Jean-Michel Marin. (And that Andrew seemed to have completely forgotten. Despite being the very first one to publish [in English] a paper on a Gibbs sampler for mixtures.) Given that this averaging can get quite costly, we propose a preliminary step to reduce the number of relevant permutations to be considered in the averaging, removing far-away modes that do not contribute to the Rao-Blackwell estimate and called dual importance sampling. We also considered modelling the posterior as a product of k-component mixtures on the components, following a vague idea I had in the back of my mind for many years, but it did not help. In the above boxplot comparison of estimators, the marginal likelihood estimators are

1. Chib’s method using T = 5000 samples with a permutation correction by multiplying by k!.
2. Chib’s method (1), using T = 5000 samples which are randomly permuted.
3. Importance sampling estimate (7), using the maximum likelihood estimate (MLE) of the latents as centre.
4. Dual importance sampling using q in (8).
5. Dual importance sampling using an approximate in (14).
6. Bridge sampling (3). Here, label switching is imposed in hyperparameters.

## MCMSki IV, Jan. 6-8, 2014, Chamonix (news #12)

Posted in Mountains, R, Statistics, University life with tags , , , , , , , , , , , , on November 26, 2013 by xi'an

We are converging towards MCMSki IV getting closer and closer to the conference! I hope that by now all intended participants have registered (registration is still open!), found a place where to stay during and around the conference (still feasible!), and booked their flight to Geneva (or nearby).

First, please send me asap the  poster abstract to bayesianstatistics@gmail.com if you plan to present a poster. We are currently with 45 abstracts on my special wordpress blog and there is no deadline for sending your abstracts. Even though Jan. 07 may be a wee bit extreme….

Second, we are currently 195 registered participants. This is fantastic! I am looking forward this great company and do not expect to find free time to go skiing during the meeting! Note that there will be hardly any conference material, except for a single sheet with the program and rooms, so make sure to plan your session in advance. I also remind participants that the banquet is a paying option in the registration form. The cost is not included in the basic registration…

Third, make sure of your travel plans to and back from Chamonix. The airport in Geneva is 80 km away and you need to book a shuttle or a bus if your timing does not coincide with the two three shuttles (each way) available via the conference registration page. There are two doodles to monitor arrivals and departures, but hardly any entry so far. Not sure I can add any extra shuttle, but if there are enough of you… Check on the conference website for travel tips. And do not, I repeat do not!, consider booking a taxi at Geneva airport as an option, since they are extremely expensive. Horrendously so. (Both on the French and Swiss sides of the airport.)

## On the use of marginal posteriors in marginal likelihood estimation via importance-sampling

Posted in R, Statistics, University life with tags , , , , , , , , , , , , , on November 20, 2013 by xi'an

Perrakis, Ntzoufras, and Tsionas just arXived a paper on marginal likelihood (evidence) approximation (with the above title). The idea behind the paper is to base importance sampling for the evidence on simulations from the product of the (block) marginal posterior distributions. Those simulations can be directly derived from an MCMC output by randomly permuting the components. The only critical issue is to find good approximations to the marginal posterior densities. This is handled in the paper either by normal approximations or by Rao-Blackwell estimates. the latter being rather costly since one importance weight involves B.L computations, where B is the number of blocks and L the number of samples used in the Rao-Blackwell estimates. The time factor does not seem to be included in the comparison studies run by the authors, although it would seem necessary when comparing scenarii.

After a standard regression example (that did not include Chib’s solution in the comparison), the paper considers  2- and 3-component mixtures. The discussion centres around label switching (of course) and the deficiencies of Chib’s solution against the current method and Neal’s reference. The study does not include averaging Chib’s solution over permutations as in Berkoff et al. (2003) and Marin et al. (2005), an approach that does eliminate the bias. Especially for a small number of components. Instead, the authors stick to the log(k!) correction, despite it being known for being quite unreliable (depending on the amount of overlap between modes). The final example is Diggle et al. (1995) longitudinal Poisson regression with random effects on epileptic patients. The appeal of this model is the unavailability of the integrated likelihood which implies either estimating it by Rao-Blackwellisation or including the 58 latent variables in the analysis.  (There is no comparison with other methods.)

As a side note, among the many references provided by this paper, I did not find trace of Skilling’s nested sampling or of safe harmonic means (as exposed in our own survey on the topic).