Archive for the pictures Category

common derivation for Metropolis–Hastings and other MCMC algorithms

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , on July 25, 2016 by xi'an

Khoa Tran and Robert Kohn from UNSW just arXived a paper on a comprehensive derivation of a large range of MCMC algorithms, beyond Metropolis-Hastings. The idea is to decompose the MCMC move into

  1. a random completion of the current value θ into V;
  2. a deterministic move T from (θ,V) to (ξ,W), where only ξ matters.

If this sounds like a new version of Peter Green’s completion at the core of his 1995 RJMCMC algorithm, it is bedowntown Sydney from under Sydney Harbour bridge, July 15, 2012cause it is indeed essentially the same notion. The resort to this completion allows for a standard form of the Metropolis-Hastings algorithm, which leads to the correct stationary distribution if T is self-inverse. This representation covers Metropolis-Hastings algorithms, Gibbs sampling, Metropolis-within-Gibbs and auxiliary variables methods, slice sampling, recursive proposals, directional sampling, Langevin and Hamiltonian Monte Carlo, NUTS sampling, pseudo-marginal Metropolis-Hastings algorithms, and pseudo-marginal Hamiltonian  Monte Carlo, as discussed by the authors. Given this representation of the Markov chain through a random transform, I wonder if Peter Glynn’s trick mentioned in the previous post on retrospective Monte Carlo applies in this generic setting (as it could considerably improve convergence…)

on Monte Rosa [a failed attempt at speed climbing]

Posted in Kids, Mountains, pictures, Running, Travel with tags , , , , , , , , , , , on July 24, 2016 by xi'an

With my daughter Rachel and her friend Clément, we tried last week to bag a few summits in the Monte Rosa massif, which stands between Italy (Aosta) and Switzerland (Zermatt). I wanted to take advantage of the Bastille Day break and we drove from Paris to Aosta in the very early morning, stopping in Chamonix to rent shoes and crampons, and meeting with our guide Abele Blanc at noon, before going together to the hut Rifugio Città di Mantova. At 3500m. Our goal was to spent the night there and climb to Punta Gnifetti (Rifugio Margherita) and Zumstein the next morning. Before heading back to Paris in the evening. However, it did not work out that way as I got a slight bout of mountain sickness that left me migrainous, nauseous, and having a pretty bad night, despite great conditions at the hut… So (despite my intense training of the previous weeks!) I did not feel that great when we left the hut at 5am. The weather was fine if cold and windy, but after two hours of moderate climbing in a fairly pleasant crispy snow of a glacier, Rachel was too out of breath to continue and Abele realised my nose had [truly] frozen (I could not feel anything!) and took us down before continuing with Clément to both peaks. This was quite a disappointment as we had planned this trip over several months, but it was clearly for the best as my fingers were definitely close to frozen (with my worst case ever of screamin’ barfies on the way down!). And we thus spent the rest of the morning waiting for our friends, warming up with tea in the sunshine. Upon reflection, planning one extra day of acclimatisation to altitude and cold would have been more reasonable and keeping handwarmers in our backpacks as well… In any case, Clément made it to the top with Abele and we got a good altitude training for the incoming San Francisco half-marathon. Plus an epic hike the next day around Cogne.

art brut [at Rifugio Città di Mantova]

Posted in Kids, Mountains, pictures, Travel with tags , , , , , , on July 23, 2016 by xi'an

sunset over Rifugio Città di Mantova

Posted in Kids, Mountains, pictures, Travel with tags , , , , , , on July 22, 2016 by xi'an

Bayesian empirical likelihood

Posted in Books, pictures, Statistics with tags , , , , , , on July 21, 2016 by xi'an

non-tibetan flags in Pula, Sardinia, June 12, 2016Sid Chib, Minchul Shin, and Anna Simoni (CREST) recently arXived a paper entitled “Bayesian Empirical Likelihood Estimation and Comparison of Moment Condition Models“. That Sid mentioned to me in Sardinia. The core notion is related to earlier Bayesian forays into empirical likelihood pseudo-models, like Lazar (2005) or our PNAS paper with Kerrie Mengersen and Pierre Pudlo. Namely to build a pseudo-likelihood using empirical likelihood principles and to derive the posterior associated with this pseudo-likelihood. Some novel aspects are the introduction of tolerance (nuisance) extra-parameters when some constraints do not hold, a maximum entropy (or exponentially tilted) representation of the empirical  likelihood function, and a Chib-Jeliazkov representation of the marginal likelihood. The authors obtain a Bernstein-von Mises theorem under correct specification. Meaning convergence. And another one under misspecification.

While the above Bernstein-von Mises theory is somewhat expected (if worth deriving) in the light of frequentist consistency results, the paper also considers a novel and exciting aspect, namely to compare models (or rather moment restrictions) by Bayes factors derived from empirical likelihoods. A grand (encompassing) model is obtained by considering all moment restrictions at once, which first sounds like more restricted, except that the extra-parameters are there to monitor constraints that actually hold. It is unclear from my cursory read of the paper whether priors on those extra-parameters can be automatically derived from a single prior. And how much they impact the value of the Bayes factor. The consistency results found in the paper do not seem to depend on the form of priors adopted for each model (for all three cases of both correctly, one correctly and none correctly specified models). Except maybe for some local asymptotic normality (LAN). Interestingly (?), the authors consider the Poisson versus Negative Binomial test we used in our testing by mixture paper. This paper is thus bringing a better view of the theoretical properties of a pseudo-Bayesian approach based on moment conditions and empirical likelihood approximations. Without a clear vision of the implementation details, from the parameterisation of the constraints (which could be tested the same way) to the construction of the prior(s) to the handling of MCMC difficulties in realistic models.

Chateau Mussar

Posted in pictures, Statistics, Travel, Wines with tags , , , , , on July 20, 2016 by xi'an

and another position in Oxford

Posted in Kids, pictures, Statistics, Travel, University life with tags , , , , , on July 20, 2016 by xi'an

St. Edmund Hall, Oxford, Feb. 01, 2016After a position in Bristol advertised a few days ago, I want to point out there also is a position opening in Oxford, Department of Statistics, in conjunction with a fellowship from University College. The deadline is August 26, 2016, and applicants should contact Professor Arnaud Doucet for details.

Follow

Get every new post delivered to your Inbox.

Join 1,068 other followers