SAS on Bayes

Following a question on X Validated, I became aware of the following descriptions of the pros and cons of Bayesian analysis, as perceived by whoever (Tim Arnold?) wrote SAS/STAT(R) 9.2 User’s Guide, Second Edition. I replied more specifically on the point

It [Bayesian inference] provides inferences that are conditional on the data and are exact, without reliance on asymptotic approximation. Small sample inference proceeds in the same manner as if one had a large sample. Bayesian analysis also can estimate any functions of parameters directly, without using the “plug-in” method (a way to estimate functionals by plugging the estimated parameters in the functionals).

which I find utterly confusing and not particularly relevant. The other points in the list are more traditional, except for this one

It provides interpretable answers, such as “the true parameter θ has a probability of 0.95 of falling in a 95% credible interval.”

that I find somewhat unappealing in that the 95% probability has only relevance wrt to the resulting posterior, hence has no absolute (and definitely no frequentist) meaning. The criticisms of the prior selection

It does not tell you how to select a prior. There is no correct way to choose a prior. Bayesian inferences require skills to translate subjective prior beliefs into a mathematically formulated prior. If you do not proceed with caution, you can generate misleading results.

It can produce posterior distributions that are heavily influenced by the priors. From a practical point of view, it might sometimes be difficult to convince subject matter experts who do not agree with the validity of the chosen prior.

are traditional but nonetheless irksome. Once acknowledged there is no correct or true prior, it follows naturally that the resulting inference will depend on the choice of the prior and has to be understood conditional on the prior, which is why the credible interval has for instance an epistemic rather than frequentist interpretation. There is also little reason for trying to convince a fellow Bayesian statistician about one’s prior. Everything is conditional on the chosen prior and I see less and less why this should be an issue.

 

6 Responses to “SAS on Bayes”

  1. “Everything is conditional on the chosen prior and I see less and less why this should be an issue.”

    Everytime I read you I feel more comfortable with this idea and everytime I am front of the clinical staff to design experiments I feel my words just fail. I personally experience this as a critical point to apply Bayesian idea today.

  2. Radford Neal Says:

    The statements from the SAS user guide you quote seem pretty much correct to me. I’m puzzled by your hostile reaction.

    The first quote might be misinterpreted since the scope of “exact” is not made clear. I take it as a contrast to asymptotic frequentist methods, such as use of the Fisher information to construct a frequentist confidence interval around the MLE, based on the assumption that the sampling distribution of the MLE is Gaussian, which is usually true only asymptotically (or maybe not at all). A Bayesian credible interval can be computed for a small sample without any approximation based on asymptotic assumptions. Even a naive reader is unlikely to take “exact” as meaning the parameter estimates are exactly equal to their true values, and the later text also makes clear that the inference is “exact” only for the prior used.

    The point about the advantages over using plug-in estimates seems quite correct to me. I’m unaware of any general frequentist solution to this problem that doesn’t rely on asymptotics.

    The point about interpretable answers is also quite correct. I think it is beyond dispute that almost all users of statistical inference want to interpret a 95% interval of (1.2,1.7) as indicating that there is a 95% probability that the parameter is between 1.2 and 1.7. And most of them don’t realize that frequentist intervals do not have that interpretation. To criticize this statement on the grounds that the Bayesian intervals are not frequentist intervals is obtuse.

    I’m baffled that you find the statement quoted last “irksome”. It seems quite correct to me, and properly cautions the users about what seem to be exactly the matters you are concerned about. Why would you say in response that “There is also little reason for trying to convince a fellow Bayesian statistician about one’s prior.” when the statement talks about convincing “subject matter experts”?

    • Thanks, Radford. I guess I felt the presentation was (and is) over-selling… And I must confess a part of my hostility certainly stems from the fact that it comes from SAS, which reminds me of my graduate studies when I was forced to write data analysis projects in SAS. On punched cards. With an output made of endless pages of listing. I was starting to code in Pascal at the time and felt this was a total waste of my time..!

    • The other thing is that I find less and less patience in dealing with the absurd notion of the “right prior”. I consider my inference conditional on my prior and would not try to convince anyone [whether a Bayesian, a frequentist, or an expert] of the righteousness of that prior, given that I do not myself “believe” in it.

    • Radford Neal Says:

      Well, as a most-of-the-time subjective Bayesian, I’m quite happy talking about the “right prior”, which is the prior that captures your actual prior beliefs. Of course, there are difficulties. Your actual beliefs change constantly. And for high-dimensional problems, visualizing whether or not your mathematical formula for the prior density really captures your beliefs is difficult. So sometimes the “right prior” may be elusive. But the “right model” is also often rather elusive. And MCMC convergence is difficult to assess. And rigorous analysis of floating-point round-off errors is more than most people can manage. I don’t see why the idealization behind the phrase “right prior” should be seen as absurd, when other idealizations are being accepted all over the place.

  3. […] Please comment on the article here: R – Xi’an’s Og […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s