MCMC convergence assessment

Richard Everitt tweetted yesterday about a recent publication in JCGS by Rajib Paul, Steve MacEachern and Mark Berliner on convergence assessment via stratification. (The paper is free-access.) Since this is another clear interest of mine’s, I had a look at the paper in the train to Besançon. (And wrote this post as a result.)

The idea therein is to compare the common empirical average with a weighted average relying on a partition of the parameter space: restricted means are computed for each element of the partition and then weighted by the probability of the element. Of course, those probabilities are generally unknown and need to be estimated simultaneously. If applied as is, this idea reproduces the original empirical average! So the authors use instead batches of simulations and corresponding estimates, weighted by the overall estimates of the probabilities, in which case the estimator differs from the original one. The convergence assessment is then to check both estimates are comparable. Using for instance Galin Jone’s batch method since they have the same limiting variance. (I thought we mentioned this damning feature in Monte Carlo Statistical Methods, but cannot find a trace of it except in my lecture slides…)

The difference between both estimates is the addition of weights p__in/q__ijn, made of the ratio of the estimates of the probability of the ith element of the partition. This addition thus introduces an extra element of randomness in the estimate and this is the crux of the convergence assessment. I was slightly worried though by the fact that the weight is in essence an harmonic mean, i.e. 1/q__ijn/Σ q__imn… Could it be that this estimate has no finite variance for a finite sample size? (The proofs in the paper all consider the asymptotic variance using the delta method.) However, having the weights adding up to K alleviates my concerns. Of course, as with other convergence assessments, the method is not fool-proof in that tiny, isolated, and unsuspected spikes not (yet) visited by the Markov chain cannot be detected via this comparison of averages.

WSC 20[1]1

This morning I attended the “Bruce Schmeiser session” at WSC 2011. I had once a meeting with Bruce (and Jim Berger) in Purdue to talk about MCMC methods but I never interacted directly with him. The first two talks were about batch methods, which I did not know previously, and I had trouble understanding what was the problem: for a truly iid normal sample, building an optimal confidence interval on the mean relies on the sufficient statistic rather than on the batch mean variance… It is only through the second talk that I understood that neither normality nor independence was guaranteed, hence the batches. I still wonder whether or a bootstrap strategy could be used instead, given the lack of confidence in the model assumptions. The third talk was about a stochastic approximation algorithm developed by Bruce Schmeiser, called retrospective approximation, where successive and improving approximations of the target to maximise are used in order not to waste time at the beginning. I thus found the algorithm had a simulated annealing flavour, even though the connection is rather tenuous…

The second session of WSC 2011 I attended was about importance sampling, The first talk was about mixtures of importance sampling distributions towards improved efficiency for cross-entropy, à la Rubinstein and Kroese. Its implementation seemed to depend very much on some inner knowledge of the target problem. The second talk was on zero-variance approximations for computing the probability that two notes are connected in a graph, using clever collapsing schemes. The third talk of the session was unrelated with the theme since it was about cross-validated non-parametric density estimation.

My own session was not terribly well attended and, judging from some questions I got at the end I am still unsure I had chosen the right level. Nonetheless, I got interesting discussions afterwards which showed that ABC was also appealing to some members of the audience. And I had a long chat with Enlu Zhou, a nice assistant professor from Urbana-Champaign who was teaching out of Monte Carlo Statistical Method, and had challenging questions about restricted support MCMC. Overall, an interesting day, completed with a light conference dinner in the pleasant company of Jingchen Liu from Columbia and some friends of his.