an introduction to MCMC sampling
Following a rather clueless question on X validated, I had a quick read of A simple introduction to Markov Chain Monte–Carlo sampling, by Ravenzwaaij, Cassey, and Brown, published in 2018 in Psychonomic Bulletin & Review, which I had never opened to this day. The setting is very basic and the authors at pain to make their explanations as simple as possible, but I find the effort somehow backfires under the excess of details. And the characteristic avoidance of mathematical symbols and formulae. For instance, in the Normal mean example that is used as introductory illustration and that confused the question originator, there is no explanation for the posterior being a N(100,15) distribution, 100 being the sample average, the notation N(μ|x,σ) is used for the posterior density, and then the Metropolis comparison brings an added layer of confusion:
“Since the target distribution is normal with mean 100 (the value of the single observation) and standard deviation 15, this means comparing N(100|108, 15) against N(100|110, 15).”
as it most unfortunately exchanges the positions of μ and x (which is equal to 100). There is no fundamental error there, due to the symmetry of the Normal density, but this switch from posterior to likelihood certainly contributes to the confusion of the QO. Similarly for the Metropolis step description:
“If the new proposal has a lower posterior value than the most recent sample, then randomly choose to accept or
reject the new proposal, with a probability equal to the height of both posterior values. “
“The method will “work” (i.e., the sampling distribution will truly be the target distribution) as long as certain conditions are met.
Firstly, the likelihood values calculated (…) to accept or reject the new proposal must accurately reflect the density of the proposal in the target distribution. When MCMC is applied to Bayesian inference, this means that the values calculated must be posterior likelihoods, or at least be proportional to the posterior likelihood (i.e., the ratio of the likelihoods calculated relative to one another must be correct).”
which leaves me uncertain as to what the authors do mean by the alternative situation, i.e., by the proposed value not reflecting the proposal density. Again, the reluctance in using (more) formulae hurts the intended pedagogical explanations.
Xi'an's Og 
August 9, 2022 at 10:12 pm
I somehow mangled my sentence. I meant: there must be some way to avoid a rigorous explanation using math and instead use plain English/whatever to get the point across.
August 10, 2022 at 9:27 am
Oh definitely, to some extent. But this introduction did not make it there. My opinion is that the plainer the language the harder it gets and the more the chances that it brings confusion. I also posit that there exists a bottom line in abstraction/formality/numeracy under which it proves impossible to carry the point..!
August 9, 2022 at 6:46 am
Hi Christian, I haven’t read the paper, but I just wanted to point out that the paper is probably written for a particular audience. I have the impression that in France, math education is much more formally oriented at the high school level compared to other countries, so that even “lay” people will have a good handle on formal presentations (is this true? Someone must have researched the relative complexity of math education in France vs elsewhere). This means that one has to simplify the presentation quite a bit and avoid formalism if one has to communicate anything at all to the intended audience. That is why I never understood your complaint about the lack of mathematics-based exposition in such articles (intended for a non-stats audience).
One has to consider one’s audience when explaining something, no?
August 9, 2022 at 8:05 am
Hi Shravan, thanks for the comments (long time no see!) but I think I understood that the paper was directed to a low-tech audience. My point though is that the attempt failed by making things more confusing and complicated than by using a few maths formula, worth a thousand words!. Witness the confusion of the poor soul who posted the question on X validated….
And I do not think my complaint is a French idiosyncrasy! Shying away from a rigorous explanation and resorting to highly verbose prose does not help.
As an aside, while, indeed, high school math education in France is attempting to distill rigour and logic, it does not really permeate through. Witness the poor ranking of French students in international comparisons.