An incomplete history of Markov Chain Monte Carlo
Last August, George Casella and I posted on arXiv a paper entitled “A History of Markov Chain Monte Carlo — Subjective Recollections from Incomplete Data —” that contained some recollections (of ours and of others) on the emergence of MCMC methods. The paper still is under review but Brad Carlin just sent me the following comments:
I learned a ton by reading your paper, and I thought I knew a little MC history! I have a few comments which I obnoxiously organize by page number (at least in the version I got):
p.2: I actually was in a hotel room at Valencia 4 (1991) and saw Andrew Thomas sitting cross-legged on a hotel bed with a primitive version of BUGS running on a primitive “portable” computer. I think they were doing the rat data problem from G&S (1990). Even then it was easy to see they were on to something.
p.5: I love that Metropolis et al had a data analysis where they used 16 burn-in and 64 “production” makes me feel better about my past applied work.
p.15: Thanks for the nice ref to Carlin and Chib — and you are going to send me scrambling to Brooks et al to re-read that “completion scheme”. But then at the end of this para I was a little disappointed you didn’t mention Spiegelhalter et al (2002), since the ready availability of DIC within BUGS is another reason why nobody ever uses RJMCMC for model choice, as was originally envisioned. I actually had a PhD student go insane trying to successfully program an RJMCMC algorithm, and since then I’ve tried to avoid asking anyone else to do it.
p.17: you sure you want to say “plod on”? Sounds like you’re tired, and I know this is not true!
p.18, etc: Thanks so much for this reminder of the 1991 OSU conference. I remember giving that talk in the second session on the first day with Mockus (who like me was visiting CMU at that time) and Cliff Litton. The LCG paper wound up being my interview talk for the Minnesota job, and later a discussion paper in JASA. but then so many of these talks became important papers: Tierney (94), Gelman and Rubin (92) and its ‘evil twin’ Geyer (92), Gilks (92), Albert and Chib (93), and on and on. It really was a who’s who and a heady time for a kid like me, less than 2 years out of grad school. Fortunately I had a big ego.
If you can stand one more anecdote, the most memorable aspect of the conference was this apparently older, bald guy standing in the back of the room shouting that the guys on stage didn’t know what they were talking about. It didn’t matter who the guys were (Gelman, Smith, Gelfand, etc), he was not shy about expressing his opinions. Many involved the notion that many parallel chains would be worse than one long chain. Of course this guy turned out to be Charlie Geyer, and the whole episode (esp with Gelman) presaged the train wreck their papers hit at Stat Sci a few months later.
We’ll make sure to incorporate those helpful recollections into the revision when (if) the paper comes back! (Maybe leaving out the health warning that RJMCMC may cause insanity…)