Think Bayes: Bayesian Statistics Made Simple

Almost Bayes can!By some piece of luck, I came upon the book Think Bayes: Bayesian Statistics Made Simple, written by Allen B. Downey and published by Green Tea Press [which I could relate to No Starch Press, focussing on coffee!, which published Statistics Done Wrong that I reviewed a while ago] which usually publishes programming books with fun covers. The book is available on-line for free in pdf and html formats, and I went through it during a particularly exciting administrative meeting…

“Most books on Bayesian statistics use mathematical notation and present ideas in terms of mathematical concepts like calculus. This book uses Python code instead of math, and discrete approximations instead of continuous mathematics. As a result, what would be an integral in a math book becomes a summation, and most operations on probability distributions are simple loops.”

The book is most appropriately published in this collection as most of it concentrates on Python programming, with hardly any maths formula. In some sense similar to Jim Albert’s R book. Obviously, coming from maths, and having never programmed in Python, I find the approach puzzling, But just as obviously, I am aware—both from the comments on my books and from my experience on X validated—that a large group (majority?) of newcomers to the Bayesian realm find the mathematical approach to the topic a major hindrance. Hence I am quite open to this editorial choice as it is bound to include more people to think Bayes, or to think they can think Bayes.

“…in fewer than 200 pages we have made it from the basics of probability to the research frontier. I’m very happy about that.”

The choice made of operating almost exclusively through motivating examples is rather traditional in US textbooks. See e.g. Albert’s book. While it goes against my French inclination to start from theory and concepts and end up with illustrations, I can see how it operates in a programming book. But as always I fear it makes generalisations uncertain and understanding more shaky… The examples are per force simple and far from realistic statistics issues. Hence illustrates more the use of Bayesian thinking for decision making than for data analysis. To wit, those examples are about the Monty Hall problem and other TV games, some urn, dice, and coin models, blood testing, sport predictions, subway waiting times, height variability between men and women, SAT scores, cancer causality, a Geiger counter hierarchical model inspired by Jaynes, …, the exception being the final Belly Button Biodiversity dataset in the final chapter, dealing with the (exciting) unseen species problem in an equally exciting way. This may explain why the book does not cover MCMC algorithms. And why ABC is covered through a rather artificial normal example. Which also hides some of the maths computations under the carpet.

“The underlying idea of ABC is that two datasets are alike if they yield the same summary statistics. But in some cases, like the example in this chapter, it is not obvious which summary statistics to choose.¨

In conclusion, this is a very original introduction to Bayesian analysis, which I welcome for the reasons above. Of course, it is only an introduction, which should be followed by a deeper entry into the topic, and with [more] maths. In order to handle more realistic models and datasets.

9 Responses to “Think Bayes: Bayesian Statistics Made Simple”

  1. I had a chance to hear Prof Downey speak this year and he has several of his PyCon lectures available on youtube. if you like his book, these are worth checking out as he is a great speaker too!

  2. […] Book Comment: Think Bayes […]

  3. Great that someone took to time to do this!

    One can go further to more realistic problems (e.g. using some simple importance sampling ideas as demonstrated in R code here https://phaneron0.wordpress.com/2012/11/23/two-stage-quincunx-2/ in the BinaryABC link ).

    As for how much understanding people can achieve with just this approach, only time will tell.

    One important question might be: could their be a group of _competent_ Bayesian analysts that never have to learn MCMC?

    Keith O’Rourke

    • Opps, had a look at the ABC entry which does not seem to be ABC as there is no rejection step based on the observed summaries but rather (an motivated) a substitution of a quadratic likelihood for the true likelihood…

      Weird given the ABC wiki page is so clear https://en.wikipedia.org/wiki/Approximate_Bayesian_computation#Summary_statistics

      Maybe some other idea of approximate?

      Keith O’Rourke

      • Thanks, Keith.

        Yes indeed, I had not looked close enough: the likelihood is clearly replaced with a Gaussian approximation for both the empirical mean and more controversially the empirical variance [top of p.119], Or the median and the inter-quartile range (p.122), with no detail on the derivation of the associated variance. This is most curious indeed, since it ends up as a crude version of Laplace’s approximation and not at all related with “our” ABC methods.

  4. Plan to buy this one. In fact these books lower the barrier for programmers to enter. Exactly like many MOOC’s. I took some ML courses which focussed on code but then now I am ready to read math texts.

  5. Thank you for this review. Although I currently don’t have the time to spend on a purely mathematical approach (although I wish I had), I would actually prefer a book that takes me from the basic ideas upwards, perhaps sacrificing a little rigour. Do you have suggestions for such books on Bayesian statistics?

    • Navneeth: I am afraid I am not very qualified for answering this query. When we wrote Bayesian Core, we thought we had a basic introduction to Bayesian Statistics, but the main criticisms were that the book was “too” mathematical!

  6. Sean O'Riordain Says:

    I think this comes down to “Learning Styles” (refer https://en.wikipedia.org/wiki/Learning_styles) where some of us learn more effectively in different ways, e.g. when I look at the help page for a function in R that I am unfamilar with, I will read the top level description, then I will look at the examples and only then will I go to the definition, whereas mathematicians will often go to the definition first. When teaching I do find it challenging to allow for the diversity of my students learning styles as no doubt do authors!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s