model misspecification in ABC

With David Frazier and Judith Rousseau, we just arXived a paper studying the impact of a misspecified model on the outcome of an ABC run. This is a question that naturally arises when using ABC, but that has been not directly covered in the literature apart from a recently arXived paper by James Ridgway [that was earlier this month commented on the ‘Og]. On the one hand, ABC can be seen as a robust method in that it focus on the aspects of the assumed model that are translated by the [insufficient] summary statistics and their expectation. And nothing else. It is thus tolerant of departures from the hypothetical model that [almost] preserve those moments. On the other hand, ABC involves a degree of non-parametric estimation of the intractable likelihood, which may sound even more robust, except that the likelihood is estimated from pseudo-data simulated from the “wrong” model in case of misspecification.

In the paper, we examine how the pseudo-true value of the parameter [that is, the value of the parameter of the misspecified model that comes closest to the generating model in terms of Kullback-Leibler divergence] is asymptotically reached by some ABC algorithms like the ABC accept/reject approach and not by others like the popular linear regression [post-simulation] adjustment. Which suprisingly concentrates posterior mass on a completely different pseudo-true value. Exploiting our recent assessment of ABC convergence for well-specified models, we show the above convergence result for a tolerance sequence that decreases to the minimum possible distance [between the true expectation and the misspecified expectation] at a slow enough rate. Or that the sequence of acceptance probabilities goes to zero at the proper speed. In the case of the regression correction, the pseudo-true value is shifted by a quantity that does not converge to zero, because of the misspecification in the expectation of the summary statistics. This is not immensely surprising but we hence get a very different picture when compared with the well-specified case, when regression corrections bring improvement to the asymptotic behaviour of the ABC estimators. This discrepancy between two versions of ABC can be exploited to seek misspecification diagnoses, e.g. through the acceptance rate versus the tolerance level, or via a comparison of the ABC approximations to the posterior expectations of quantities of interest which should diverge at rate Vn. In both cases, ABC reference tables/learning bases can be exploited to draw and calibrate a comparison with the well-specified case.

Hell’s ride down Ardèche

Being in vacation in Ardèche, south east of France, we went for a “white water” canoe trip along the Ardèche canyon. The river sounded peaceful enough for a 25km family trip down the river and the company who rented us the three (sea) kayaks dumped us at Pont d’Arc with the boats and waterproof barrels without any explanation about handling the boats or managing the 30 or so rapids along the way. We had used sea-kayaks in Australia for turtle watching near Mission Beach with no other damages than sunburned feet, but this trip almost ended up dramatically! After passing the first half of the way most pleasantly (or in blissful ignorance), we ended up hitting a huge rock in a naughty passage that made our boat capsize. I hit the rock as well while my wife had a difficult time getting from under the boat and it is only thanks to the other kayaks in the group that we managed to recover our boat! I lost my glasses in the event and spent the rest of the descent in a colourful fog. And experienced a fit of hypothermia from spending five minutes swimming in the 10C water. (We capsized a second time in a milder passage with no further consequence and another boat in our group also capsized, hitting a rock that cut deeply my brother-in-law’s hand…)

We came back to the kayak bases with those few cuts and no further accident, but I am fairly unhappy with the whole trip! In restrospect, the potential dangers of the trip were too important to be ignored and the lack of warning from the companies renting the kayak is appaling in that they present the trip as fun and thrilling, while omitting to mention the high accident rate (that we discovered a posteriori). This irresponsible attitude is equivalent to providing beginners with ropes and harnesses and letting them rappel a 10m cliff on their own! Especially when kids above 7 can take part in the trip with only a life jacket and an helmet. (When the boat capsizes in a strong and swirling current, it is impossible to get hold of anyone else: I got separated from my wife by a distance of twenty meters within a few seconds!) On the positive side, the views were terrific, the weather very nice, and I even avoided the usual blisters! But I am not considering repeating the experiment without first pairing with an experienced kayaker.