accelerating MCMC

I have recently [well, not so recently!] been asked to write a review paper on ways of accelerating MCMC algorithms for the [review] journal WIREs Computational Statistics and would welcome all suggestions towards the goal of accelerating MCMC algorithms. Besides [and including more on]

• coupling strategies using different kernels and switching between them;
• tempering strategies using flatter or lower dimensional targets as intermediary steps, e.g., à la Neal;
• sequential Monte Carlo with particle systems targeting again flatter or lower dimensional targets and adapting proposals to this effect;
• Hamiltonian MCMC, again with connections to Radford (and more generally ways of avoiding rejections);
• adaptive MCMC, obviously;
• Rao-Blackwellisation, just as obviously (in the sense that increasing the precision in the resulting estimates means less simulations).

John Kruschke on Bayesian assessment of null values

John Kruschke pointed out to me a blog entry he wrote last December as a follow-up to my own entry on an earlier paper of his. Induced by an X validated entry. Just in case this sounds a wee bit too convoluted for unraveling the threads (!), the central notion there is to replace a point null hypothesis testing [of bad reputation, for many good reasons] with a check whether or not the null value stands within the 95% HPD region [modulo a buffer zone], which offers the pluses of avoiding a Dirac mass at the null value and a long-term impact of the prior tails on the decision, as well as the possibility of a no-decision, with the minuses of replacing the null with a tolerance region around the null and calibrating both the rejection level and the buffer zone. The December blog entry exposes this principle with graphical illustrations familiar to readers of Doing Bayesian Data Analysis.

As I do not want to fall into an infinite regress of mirror discussions, I will not proceed further than referring to my earlier post, which covers my reservations about the proposal. But interested readers may want to check the latest paper by Kruschke and Liddel on that perspective. (With the conclusion that “Bayesian estimation does everything the New Statistics desires, better”.) Available on PsyArXiv, an avatar of arXiv for psychology papers.

optimal Bernoulli factory

One of the last arXivals of the year was this paper by Luis Mendo on an optimal algorithm for Bernoulli factory (or Lovàsz‘s or yet Basu‘s) problems, i.e., for producing an unbiased estimate of f(p), 0<p<1, from an unrestricted number of Bernoulli trials with probability p of heads. (See, e.g., Mark Huber’s recent book for background.) This paper drove me to read an older 1999 unpublished document by Wästlund, unpublished because of the overlap with Keane and O’Brien (1994). One interesting gem in this document is that Wästlund produces a Bernoulli factory for the function f(p)=√p, which is not of considerable interest per se, but which was proposed to me as a puzzle by Professor Sinha during my visit to the Department of Statistics at the University of Calcutta. Based on his 1979 paper with P.K. Banerjee. The algorithm is based on a stopping rule N: throw a fair coin until the number of heads n+1 is greater than the number of tails n. The event N=2n+1 occurs with probability

[Using a biased coin with probability p to simulate a fair coin is straightforward.] Then flip the original coin n+1 times and produce a result of 1 if at least one toss gives heads. This happens with probability √p.

Mendo generalises Wästlund‘s algorithm to functions expressed as a power series in (1-p)

with the sum of the weights being equal to one. This means proceeding through Bernoulli B(p) generations until one realisation is one or a probability

event occurs [which can be derived from a Bernoulli B(p) sequence]. Furthermore, this version achieves asymptotic optimality in the number of tosses, thanks to a form of Cramer-Rao lower bound. (Which makes yet another connection with Kolkata!)

incredible India

[The following is a long and fairly naïve rant about India and its contradiction, without pretence at anything else than writing down some impressions from my last trip. JATP: Just another tourist post!]

Incredible India (or Incredible !ndia) is the slogan chosen by the Indian Ministry of Tourism to promote India. And it is indeed an incredible country, from its incredibly diverse landscapes [and not only the Himalayas!] and eco-systems, to its incredibly huge range of languages [although I found out during this trip that the differences between Urdu and Hindi are more communitarian and religious than linguistic, as they both derive from Hindustani, although the alphabets completely differ] and religions [a mixed blessing], to its incredibly rich history and culture, to its incredibly wide offer of local cuisines [as shown by the Bengali sample below, where the mustard seed fish cooked in banana leaves and the fried banana flowers are not visible!] and even wines [like Sula Vineyards, which offers a pretty nice Viognier]. Not to mention incredibly savoury teas from Darjeeling and Assam.

tea rampage

While in India, I took the opportunity to buy Darjeeling tea in large quantities (15 boxes!) as teas from a wide variety of gardens and types were available, including my favourite, Muscatel. Some of which are for family and friends. We also drove through huge tea gardens on our way back to Bagdogra airport, with new buds already growing on the tea bushes. (The pictures below are taken from the Long View tea estate.)

the terminal

The Terminal is this (terrible) movie featuring Tom Hanks getting stuck in an airport international zone for an indefinite time (and based on a real story that saw Karimi Nasseri remain in Terminal 1 of Roissy, for 18 years, when being there for a few hours is already unbearable!). In a similar spirit, we got quarantined to the international zone in Delhi airport for 24 hours, thanks to a missed connection. And to an Air India representative who could not be bothered in finding another route, letting us out to visit the city, or even providing us access to our bags. So we ended up waiting in the airport short stay hotel, around the clock, with a bed, food and wifi. Not the end of the World, obviously! And with a rather unique view on the registration desks below.

Great North Road [book review]

As I was unsure of the Internet connections and of the more than likely delays I would face during my trip to India, I went fishing for a massive novel on Amazon and eventually ordered Peter Hamilton’s Great North Road, a 1088 pages behemoth! I fear the book qualifies as space opera, with the conventional load of planet invasions, incomprehensible and infinitely wise aliens, gateways for instantaneous space travels, and sentient biospheres. But the core of the story is very, very, Earth-bound, with a detective story taking place in a future Newcastle that is not so distant from now in many ways. (Or even from the past as the 2012 book did not forecast Brexit…) With an occurrence of the town moor where I went running a few years ago.

The book is mostly well-designed, with a plot gripping enough to keep me hooked for Indian evenings in Kolkata and most of the flight back. I actually finished it just before landing in Paris. There is no true depth in the story, though, and the science fiction part is rather lame: a very long part of the detective plot is spent on the hunt for a taxi by an army of detectives, a task one would think should be delegated to a machine-learning algorithm and solved in a nano-second or so. The themes heavily borrow from those of classics like Avatar, Speaker for the Dead, Hyperion [very much Hyperion!], Alien… And from The Girl with the Dragon Tattoo for an hardcore heroin who is perfect at anything she undertakes.  Furthermore, the Earth at the centre of this extended universe is very close to its present version, with English style taxis, pub culture, and a geopolitic structure of the World pretty much unchanged. Plus main brands identical to currents ones (Apple, BMW, &tc), to the point it sounds like sponsored links! And no clue of a major climate change despite the continued use of fuel engines. Nonetheless, an easy read when stuck in an airport or a plane seat for several hours.