Arrogance sampling

January 7, 2011
By

(This article was first published on Xi'an's Og » R, and kindly contributed to R-bloggers)

A new posting on arXiv by Benedict Escoto on a simulation method for approximating normalising constants (i.e. evidence) with an eye-catching name! Here is the abstract

This paper describes a method for estimating the marginal likelihood or Bayes factors of Bayesian models using non-parametric importance sampling (“arrogance sampling”). This method can also be used to compute the normalizing constant of probability distributions. Because the required inputs are samples from the distribution to be normalized and the scaled density at those samples, this method may be a convenient replacement for the harmonic mean estimator. The method has been implemented in the open source R package margLikArrogance.

The crux of the arrogant sampling method is in using a non-parametric estimation of the target function, based on a preliminary simulation from the posterior distribution. The nonparametric estimate is entered in an harmonic mean representation we previously exploited in our HPD proposal for evidence approximation

1bigg/frac{1}{T}sum_t {hat{pi}(theta_t)}big/{p(theta_t,x)}

This estimate hatpi is an histogram estimate that is smoothed by the knowledge of the joint density at points in the bins. Given that the support of the histogram is not restricted to be smaller than the support of pi(theta|x), as in our proposal, there could be support problems since the bins of the histogram are data-dependent. The associated R package margLikArrogance seems to be able to construct the bins by itself, but I have not tested it. Overall, the method seems to be bound to suffer from the curse of dimensionality, unless the bin construction is aggressively reacting to empty bins. Since the paper contains no illustration whatsoever, it is difficult to tell… I would also like to see more details on the convergence results, including the CLT, on the arrogant approximation because hatpi depends on the whole sample, thus does not benefit from standard importance convergence results.


Filed under: R, Statistics Tagged: Bayes factor, Bayesian model choice, evidence, harmonic mean estimator, marginal likelihood, margLikArrogance, normalising constant, R, R package

To leave a comment for the author, please follow the link and comment on his blog: Xi'an's Og » R.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , , , , , , , , ,

Comments are closed.