## parallel MCMC via Weierstrass sampler (a reply by Xiangyu Wang)

Posted in Books, Statistics, University life with tags , , , , , , , , , , , , on January 3, 2014 by xi'an

Almost immediately after I published my comments on his paper with David Dunson, Xiangyu Wang sent a long comment that I think worth a post on its own (especially, given that I am now busy skiing and enjoying Chamonix!). So here it is:

Thanks for the thoughtful comments. I did not realize that Neiswanger et al. also proposed the similar trick to avoid combinatoric problem as we did for the rejection sampler. Thank you for pointing that out.

For the criticism 3 on the tail degeneration, we did not mean to fire on the non-parametric estimation issues, but rather the problem caused by using the product equation. When two densities are multiplied together, the accuracy of the product mainly depends on the tail of the two densities (the overlapping area), if there are more than two densities, the impact will be more significant. As a result, it may be unwise to directly use the product equation, as the most distant sub-posteriors could be potentially very far away from each other, and most of the sub posterior draws are outside the overlapping area. (The full Gibbs sampler formulated in our paper does not have this issue, as shown in equation 5, there is a common part multiplied on each sub-posterior, which brought them close.)

Point 4 stated the problem caused by averaging. The approximated density follows Neiswanger et al. (2013) will be a mixture of Gaussian, whose component means are the average of the sub-posterior draws. Therefore, if sub-posteriors stick to different modes (assuming the true posterior is multi-modal), then the approximated density is likely to mess up the modes, and produce some faked modes (eg. average of the modes. We provide an example in the simulation 3.)

Sorry for the vague description of the refining method (4.2). The idea is kinda dull. We start from an initial approximation to θ and then do one step Gibbs update to obtain a new θ, and we call this procedure ‘refining’, as we believe such process would bring the original approximation closer to the true posterior distribution.

The first (4.1) and the second (4.2) algorithms do seem weird to be called as ‘parallel’, since they are both modified from the Gibbs sampler described in (4) and (5). The reason we want to propose these two algorithms is to overcome two problems. The first is the dimensionality curse, and the second is the issue when the subset inferences are not extremely accurate (subset effective sample size small) which might be a common scenario for logistic regression (with large parameters) even with huge data set. First, algorithm (4.1) and (4.2) both start from some initial approximations, and attempt to improve to obtain a better approximation, thus avoid the dimensional issue. Second, in our simulation 1, we attempt to pull down the performance of the simple averaging by worsening the sub-posterior performance (we allocate smaller amount of data to each subset), and the non-parametric method fails to approximate the combined density as well. However, the algorithm 4.1 and 4.2 still work in this case.

I have some problem with the logistic regression example provided in Neiswanger et al. (2013). As shown in the paper, under the authors’ setting (not fully specified in the paper), though the non-parametric method is better than simple averaging, the approximation error of simple averaging is small enough for practical use (I also have some problem with their error evaluation method), then why should we still bother to use a much more complicated method?

Actually I’m adding a new algorithm into the Weierstrass rejection sampling, which will render it thoroughly free from the dimensionality curse of p. The new scheme is applicable to the nonparametric method in Neiswanger et al. (2013) as well. It should appear soon in the second version of the draft.

## MCMSki IV, Jan. 6-8, 2014, Chamonix (news #16)

Posted in Mountains, R, Statistics, University life with tags , , , , , , , , , , , on December 31, 2013 by xi'an

I am now in Chamonix, a week before MCMSki IV. The town is packed with tourists from all over Europe, English being the dominant language. There is not much snow so far, even though some runs reach town. (I did ski in the nearby Les Houches today and the red runs were either icy or very thin, with stones and grass showing here and there…) Since the town is quite pricey in comparison with other French ski resorts, esp. in terms of rental, let me point out the special store of Technique Extrême, with very low rates, located a few blocks away from the conference centre. I am currently trying to see whether or not we could get special rates for the daily passes. I checked whether ski passes could be sold at the registration desk, but the only solution was for me to buy them in advance, so I gave up on that. If you arrive on Sunday my advice is to buy the pass from one of the cablecar stations (which close about 5pm). You can also buy it on line. And pick it later from a cablecar station booth…

## MCMSki IV, Jan. 6-8, 2014, Chamonix (news #15)

Posted in Mountains, R, Statistics, University life with tags , , , , , , , , , , , on December 26, 2013 by xi'an

The programs of the talks, posters and workshop are now printed and available on Speaker Deck (talks, posters, workshop). Please let me know if you spot anything wrong (even though it will not be reprinted!). This is presumably the last news item till Jan. 5 as I am almost off to Chamonix for a week of hard work preparing the conference happily skiing with my family, looking forward seeing some participants in the coming week in the streets of Chamonix and all of them/you on Monday, Jan. 6, 8:30! (Snow is falling right now, so there should be no issue with finding open runs…)

## MCMSki IV, Jan. 6-8, 2014, Chamonix (news #14)

Posted in Mountains, R, Statistics, University life with tags , , , , , , , , , , , , on December 22, 2013 by xi'an

We are quite glad to announce that, thanks to the support of several publishers (CRC Press/Taylor & Francis, Elsevier, and Springer-Verlag), there will be book prizes for both poster nights at MCMSki IV ! I was reminded of this possibility during O-Bayes 2013 and those publishers reacted both quickly and positively to my request (and so did the members of the juries). The list and abstracts of the 75 posters are available on the special blog. (It will not necessarily be provided on site so make sure to store it and bring it with you!)

Another thing of importance: if you have not booked your shuttle through the registration webpage, you have to find a private shuttle on your own. The doodles I set to monitor arrivals and departures did not fill up well enough to be of use. Once again, do not rely on taxis! Looking forward seeing the 200+ participants on January 06 in Chamonix, best Season Wishes!

And even though this is only of interest to locals, Nicolas and Robin have planned an “MCMSki decompression session” at the BiP seminar on 23rd January: MCMSki attendees will give a short presentation of a talk that they liked (or disliked?). Something to keep in mind during the conference. (Guest posts for the ‘Og are also invited from all participants.)

## MCMSki IV, Jan. 6-9, 2014, Chamonix (news #13)

Posted in Mountains, Statistics, Travel, University life with tags , , , , , , , , , , , , , , on December 9, 2013 by xi'an

Now, most poster abstracts have been received (or at least 63 of them),  even though newcomers can still send them to my wordpress address (if they realise the message gets posted immediately!, so the format Subject: firstname secondname (affiliation): title and text: abstract must be respected! No personal message or query please!). We have now above 200 registered participants, with all sessions remaining miraculously full (after a few permutations in the program).

S0 it is time to mention a wee bit of the “ski” side of MCMski. Chamonix has two types of ski passes, Chamonix Le Pass, and Mont Blanc Unlimited, the later allowing a wide access to the Mont Blanc area, up to 3800 meters and in France, Italy, and Switzerland, but presumably harder to exploit to the fullest on a 4 hour afternoon break. (You have to arrange renting skis and buying passes on your own! The conference centre may answer moderate queries but not make any booking.)  The temperature in the town of Chamonix is currently between -7 and 0 (centigrades), with ten centimetres of snow in town. All ski areas will be open by Dec. 21. If you plan to ski the Vallée Blanche from Aiguille du Midi, at 3800m, I strongly advise renting a guide for this ultimate skiing experience!

Big big big news: not only the ski race will take place on Wed., Jan. 08, afternoon, organised by ESF Chamonix, but Antonietta Mira managed to secure one or two pairs of skis for the winner(s) of the race! I doubt there will be other opportunities of that magnitude for winning a magnificent pair of skis made in Italy by Blossom skis. Thanks a lot to Anto!!! And to Blossom skis (whose collection includes a series called FreeTibet.)