## Gibbs sampling with incompatible conditionals

An interesting question (with no clear motivation) on X validated wondering why a Gibbs sampler produces NAs… Interesting because multi-layered:

1. The attached R code indeed produces NAs because it calls the Negative Binomial Neg(x¹,p) random generator with a zero success parameter, x¹=0, which automatically returns NAs. This can be escaped by returning a one (1) instead.
2. The Gibbs sampler is based on a Bin(x²,p) conditional for X¹ and a Neg(x¹,p) conditional for X². When using the most standard version of the Negative Binomial random variate as the number of failures, hence supported on 0,1,2…. these two conditionals are incompatible, i.e., there cannot be a joint distribution behind that returns these as conditionals, which makes the limiting behaviour of the Markov chain harder to study. It however seems to converge to a distribution close to zero, which is not contradictory with the incompatibility property: the stationary joint distribution simply does not enjoy the conditionals used by the Gibbs sampler as its conditionals.
3. When using the less standard version of the Negative Binomial random variate understood as a number of attempts for the conditional on X², the two conditionals are compatible and correspond to a joint measure proportional to $x_1^{-1} {x_1 \choose x_2} p^{x_2} (1-p)^{x_1-x_2}$, however this pmf does not sum up to a finite quantity (as in the original Gibbs for Kids example!), hence the resulting Markov chain is at best null recurrent, which seems to be the case for p different from ½. This is unclear to me for p=½.

### 5 Responses to “Gibbs sampling with incompatible conditionals”

2. The use of the Gibbs sampler when the full conditionals are incompatible has been touched on various times recently in this blog. It is an intriguing topic (one that I am particularly interested in, but not for wholly Bayesian reasons) and I admit that I had somewhat overlooked the subtleties of the arguments concerned until very recently.

There is still much more to be learnt, and while this is the case I feel that we need to make sure that we are presenting precise lines of reasoning. This is why I feel that I need to take you to task for what you wrote in point 3(!) In particular:

(a) You say that the two full conditionals are compatible, but then say that the pmf does not sum up to a finite quantity, meaning that the pmf does not exist, which contradicts the statement that the two full conditionals are compatible.

(b) You make a distinction between the cases (i) p different from 0.5 and (ii) p = 0.5, but it is unclear whether the distinction between these cases is being made in terms of the compatibility of the conditionals, or the classification of the Markov chain as being transient, null recurrent or positive recurrent. However, we can establish that:
Transient or null recurrent –> incompatible conditionals
Positive recurrent –> possibly compatible conditionals.

I know you often need to write this blog quickly, but on this occasion I am struggling to read between the lines to see what you really wanted to write.

• Thank you for calling for clarifications! On the first point (a), the full conditionals are conditionals of a sigma-finite measure rather than from a probability measure. Conditional measures that can be normalised into probability distributions, hence simulated from. And used to establish the stationarity of the measure, which does not imply anything about transience versus null recurrence. On the second point (b), I am unclear as to whether or not p=1/2 leads to a transient or a null recurrent chain, for incompatibility occurs for all values of p. Regarding your correspondence table, I disagree with both implications in that (i) null recurrence may occur with compatible conditionals if the joint is a measure and (ii) positive recurrence may still occur with incompatible conditionals, but need be rigorously established case by case.