On ABC target ambiguity

On Monday, Sisson, Peters, Briers and Fan posted a note on arXiv whose purpose somehow defeats me. The paper indeed discriminates between the ABC simulation from

\pi_M(\theta|t_y) = c_M \int K_h(t-t_y) f(t|\theta) \pi(\theta) \text{d}t

and

\pi_J(\theta,t^{1:S}|t_y)=c_M\frac{1}{S}\sum_{s=1}^SK_h(t^s-t_y)\prod_{s=1}^Sf(t^s|\theta)\pi(\theta)\,.

However, the former is the marginal of the latter, so simulations from the latter produce as a by-product simulations from the former. This equivalence is clearly acknowledged in the paper but the authors then state that the ABC-MCMC sampler is theoretically only valid under the joint target posterior \pi_J (\theta; t^{1:S}jt_y) and that S = 1 is apparently too small to result in an unbiased sampler targeting \pi_M(\theta|t_y). Their reasoning is that, if we look at ABC as providing an unbiased estimate of \pi_M(\theta|t_y), using this unbiased estimate in a Metropolis-Hastings ratio results in a biased estimate. This is formaly correct (and the recent JRSS B paper of Andrieu, Doucet and Holenstein was addressing this very issue) but ABC-MCMC is never justified this way in the literature, rather as an MCMC algorithm targeting the auxiliary variable joint \pi_J(\theta,t^{1:S}|t_y), see e.g. Marjoram, Molitor, Plagnol and Tavaré (2003). Thus the ABC-MCMC algorithm of Marjoram et al. (2003) is perfectly valid per se and it delivers a sequence of \theta_n‘s that is converging to the target \pi_M(\theta|t_y) even though this sequence is not a Markov chain itself… The fact that different values of S produce different qualities of approximation of \pi_M(\theta|t_y) is altogether another issue that has not been thoroughly examined so far. But unbiasedness has nothing to do with its performances.

The last part of the paper covers the population Monte Carlo situation already discussed in earlier posts. I fail to understand the argument that the weight used in Sisson, Fan and Tanaka (2007) would become in practice unbiased through association with the equivalent sampler targeting \pi_J(\theta,t^{1:S}|t_y). if we look at the weight (8) in the current paper, using S=1,

K_h(t)=\mathbb{I}(|t|<h)/2h

and

L_{k-1}(\theta_k,\theta_{k-1})=M_{k}(\theta_{k-1},\theta_{k}),

this weight ends up as Sisson et al.’s (2007)

\dfrac{\pi(\theta_k)}{\pi(\theta_{k-1})}

which we proved is not a correct importance weight. Using the joint target does not evacuate this difficulty.

One Response to “On ABC target ambiguity”

  1. Dear Christian,

    Thanks for your comments on our paper. I’m not sure that I fully agree with them, so I would like to take the opportunity to address these. I have chatted to you informally about this at the conference we’re both currently attending(!), but I thought it would be useful to express them here for your readers.

    Firstly, you comment that “ABC-MCMC is never justified [on marginal space, using a Monte Carlo estimate of the likelihood] in the literature, rather as an MCMC algorthm targeting the auxiliary variable joint [distribution], see e.g. Marjoram et al (2003).”

    I’m not fully convinced by this comment. Nowhere in the Marjoram et al (2003) paper is even the suggestion of an auxiliary variable joint distribution mentioned (unless I misread, of course). The only discussion is on marginal space samplers. Marjoram et al (2003) also explicitly implement a sampler using Monte Carlo estimates of the likelihood (though this is to verify that their ABC-MCMC algorithm with a single dataset generation performs correctly). However, I stress again that nowhere (that I can find) is any mention of a joint target distribution mentioned. At best, therefore, the article is very ambiguous as to the nature/target of the algorithm. And of course, this is the point of our paper.

    Also, even if it were the case that Marjoram et al (2003) explicitly stated a joint distribution target (they do not), this would still not mitigate our point that some ABC algorithms are valid on both marginal and joint space targets, whereas others are only valid with joint space targets (though all are practically unbiased).

    Your second comment is “I fail to understand the argument that the weight used in Sisson, Fan and Tanaka (2007) would become in practice unbiased through association with the equivalent sampler targeting [the joint sampler].” Of course you *should* fail to understand this, as a) its not the point we’re making, and b) if it were the point, it would be incorrect!

    The point that we’re making is that SMC samplers on marginal space (using Monte Carlo estimates) are in practice unbiased through association with the equivalent sampler targeting joint space. You have commented that this argument is “formally correct” for MCMC samplers, so clearly this must also extend to SMC samplers, as the argument is identical!

    Your confusion arises as the Sisson et al (2007) algorithm is clearly biased. However (and we don’t say this explicitly in the paper, as its not the focus of the article) the clear implication from our article, is that this bias is *not* due to a ratio of unbiased likelihood estimates in the weight computation. The bias comes from a different source, and I belive that you have a recent paper discussing this.

    Hopefully this clarifies the above points, and allows some appreciation of the contribution of our article.

Leave a Reply to Scott Sisson Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.