Bounded target support

Here is an interesting question from Tomàs that echoes a lot of related emails:

I’m turning to you for advice. I’m facing problem  where parameter space is bounded, e.g. all parameters have to be positive.
If in MCMC as proposal distribution I use normal distribution, then at some iterations I get negative proposals. So my question is: should I use recalculation of acceptance probability every time I reject the proposal (something like in delayed rejection method), or I have to use another proposal (like lognormal, trancated normal, etc.)?

It is indeed a popular belief that something needs to be done to counteract restricted supports. However, there is no mathematical reason for doing so! Consider the following illustration

target=function(x) (x>0)*(x<1)*dnorm(x,mean=4)
mcmc=rep(0.5,1e5)
for (t in 2:1e5){
prop=mcmc[t-1]+rnorm(1,.1)
if (runif(1)<target(prop)/target(mcmc[t-1]))
mcmc[t]=prop
else
mcmc[t]=mcmc[t-1]
}
hist(mcmc,prob=TRUE,col="wheat",border=FALSE,main="",xlab="")
curve(dnorm(x-4)/(pnorm(-3)-pnorm(-4)),add=TRUE,col="sienna",lwd=2)

and the following outcome, with a perfect fit!

One Response to “Bounded target support”

  1. You may want to highlight that you check for out-of-support in the evaluation of the target density. What may puzzle those readers is that they try to evaluate the target at the given point, without checking for the support first: think (in pseudocode)
    return N(x; mu, sigma)
    when sigma is negative, instead of
    if sigma < 0
    return 0
    else
    return N(x; mu, sigma)
    end

    Otherwise stated, in more classical Bayesian terms: short-circuit the likelihood evaluation if the prior density is null.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s