## where did the normalising constants go?! [part 1]

**W**hen listening this week to several talks in Banff handling large datasets or complex likelihoods by parallelisation, splitting the posterior as

and handling each term of this product on a separate processor or thread as proportional to a probability density,

then producing simulations from the m_{i}‘s and attempting at deriving simulations from the original product, I started to wonder where all those normalising constants went. What vaguely bothered me for a while, even prior to the meeting, and then unclicked thanks to Sylvia’s talk yesterday was the handling of the normalising constants ω_{i} by those different approaches… Indeed, it seemed to me that the samples from the m_{i}‘s should be weighted by

rather than just

or than the product of the other posteriors

which makes or should make a significant difference. For instance, a sheer importance sampling argument for the aggregated sample exhibited those weights

Hence processing the samples on an equal footing or as if the proper weight was the product of the other posteriors m_{j} should have produced a bias in the resulting sample. This was however the approach in both Scott et al.‘s and Neiswanger et al.‘s perspectives. As well as Wang and Dunson‘s, who also started from the product of posteriors. (Normalizing constants are considered in, e.g., Theorem 1, but only for the product density and its Weierstrass convolution version.) And in Sylvia’s talk. Such a consensus of high calibre researchers cannot get it wrong! So I must have missed something: what happened is that the constants eventually did not matter, as expanded in the next post…

*Related*

This entry was posted on March 11, 2014 at 12:14 am and is filed under R, Statistics, Travel with tags big data, consensus, embarassingly parallel, normalising constant, parallel processing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

March 11, 2014 at 6:06 pm

[…] article was first published on Xi'an's Og » R, and kindly contributed to […]

March 11, 2014 at 5:48 am

Waiting for part 2.