Bounded target support

July 4, 2011

(This article was first published on Xi'an's Og » R, and kindly contributed to R-bloggers)

Here is an interesting question from Tomàs that echoes a lot of related emails:

I’m turning to you for advice. I’m facing problem where parameter space is bounded, e.g. all parameters have to be positive.
If in MCMC as proposal distribution I use normal distribution, then at some iterations I get negative proposals. So my question is: should I use recalculation of acceptance probability every time I reject the proposal (something like in delayed rejection method), or I have to use another proposal (like lognormal, trancated normal, etc.)?

It is indeed a popular belief that something needs to be done to counteract restricted supports. However, there is no mathematical reason for doing so! Consider the following illustration

target=function(x) (x>0)*(x<1)*dnorm(x,mean=4)
mcmc=rep(0.5,10^5)
for (t in 2:10^5){
prop=mcmc[t-1]+rnorm(1,.1)
if (runif(1)<target(prop)/target(mcmc[t-1]))
mcmc[t]=prop
else
mcmc[t]=mcmc[t-1]
}
hist(mcmc,prob=TRUE,col="wheat",border=FALSE,main="",xlab="")
curve(dnorm(x-4)/(pnorm(-3)-pnorm(-4)),add=TRUE,col="sienna",lwd=2)

and the following outcome, with a perfect fit!