Thanks

]]>First of all we would like to thank you for the attention you paid to our paper and for all your fruitful remarks. This will be taken into consideration to write the next version of the article. However, we have several comments to clarify some details.

First, you say that because of the thresholding operator, the Shrinkage-Thresholding MALA algorithm does not sample exactly from the target distribution. This is true for the hard thresholding operator (Section 3.2) which avoids the shrinkage of all the active rows (but which cannot draw rows with a norm lower than a given threshold). The two other operators, namely the L2,1 proximal and the soft thresholding function, do not suffer from this flaw and propose new rows with norms as close to zero as needed. This is illustrated in Figure 1. Therefore, in the numerical section, both RJMCMC and STMALA target the right distribution.

In the numerical section, Figure 5 displays the error obtained when the algorithms are used to estimate the activation probabilities (defined in equation (18)). In addition, the ordinate axis of this figure (as well as the one of Figure 10 for example) is in logarithmic scale which explains why there is no plateau in the curves while the algorithms do converge.

Once again, many thanks for your remarks which will help us to write an improved version of the article.

Best regards,

Amandine Schreck and her co-authors.

]]>* The methodology developed in the paper is not limited to large data sets but is applicable to any scenario where the Metropolis-Hastings ratio is intractable as long as one has exact confidence intervals for a Monte Carlo estimate of this ratio; e.g. this could be applied to doubly intractable targets where using standard pseudo-marginal techniques is often impossible as obtaining non-negative unbiased estimates requires perfect samples. It is also worth mentioning that the idea of using confidence intervals for deciding whether to accept/reject a proposal within Metropolis in such intractable scenarios is not original and has been proposed several times since the late 80′s in operation research (see the references in our paper). Unfortunately the approximate confidence intervals used in all previous work we are aware of can provide misleading results as demonstrated empirically in our paper. Our adaptive sampling strategy combined to exact confidence intervals is more conservative and more robust and it allows us to provide a quantitative bound between the target of interest and the perturbed target we are sampling from.

* The large data sets scenario is not a particularly compelling application of the methodology as it is clearly acknowledged in the paper. The problem is that, in a large data set context, the estimate of the MH ratio is obtained using a “blind” Monte Carlo strategy and typically admits a large variance (as we might miss very informative observations). We could obviously use an importance sampling type strategy to reduce this variance (and consequently improve the performance of the algorithm) but this does not appear very realistic from a practical point of view in the large data set context.

]]>