With Gael Martin, Brendan McCabe, David T. Frazier, and Worapree Maneesoonthorn, we arXived (and submitted) a strongly revised version of our earlier paper. We begin by demonstrating that reduction to a set of sufficient statistics of reduced dimension relative to the sample size is infeasible for most state-space models, hence calling for the use of partial posteriors in such settings. Then we give conditions [like parameter identification] under which ABC methods are Bayesian consistent, when using an auxiliary model to produce summaries, either as MLEs or [more efficiently] scores. Indeed, for the order of accuracy required by the ABC perspective, scores are equivalent to MLEs but are computed much faster than MLEs. Those conditions happen to to be weaker than those found in the recent papers of Li and Fearnhead (2016) and Creel et al. (2015). In particular as we make no assumption about the limiting distributions of the summary statistics. We also tackle the dimensionality curse that plagues ABC techniques by numerically exhibiting the improved accuracy brought by looking at marginal rather than joint modes. That is, by matching individual parameters via the corresponding scalar score of the integrated auxiliary likelihood rather than matching on the multi-dimensional score statistics. The approach is illustrated on realistically complex models, namely a (latent) Ornstein-Ulenbeck process with a discrete time linear Gaussian approximation is adopted and a Kalman filter auxiliary likelihood. And a square root volatility process with an auxiliary likelihood associated with a Euler discretisation and the augmented unscented Kalman filter. In our experiments, we compared our auxiliary based technique to the two-step approach of Fearnhead and Prangle (in the Read Paper of 2012), exhibiting improvement for the examples analysed therein. Somewhat predictably, an important challenge in this approach that is common with the related techniques of indirect inference and efficient methods of moments, is the choice of a computationally efficient and accurate auxiliary model. But most of the current ABC literature discusses the role and choice of the summary statistics, which amounts to the same challenge, while missing the regularity provided by score functions of our auxiliary models.
Archive for Monash University
Along with David Frazier and Gael Martin from Monash University, Melbourne, we have just completed (and arXived) a paper on the (Bayesian) consistency of ABC methods, producing sufficient conditions on the summary statistics to ensure consistency of the ABC posterior. Consistency in the sense of the prior concentrating at the true value of the parameter when the sample size and the inverse tolerance (intolerance?!) go to infinity. The conditions are essentially that the summary statistics concentrates around its mean and that this mean identifies the parameter. They are thus weaker conditions than those found earlier consistency results where the authors considered convergence to the genuine posterior distribution (given the summary), as for instance in Biau et al. (2014) or Li and Fearnhead (2015). We do not require here a specific rate of decrease to zero for the tolerance ε. But still they do not hold all the time, as shown for the MA(2) example and its first two autocorrelation summaries, example we started using in the Marin et al. (2011) survey. We further propose a consistency assessment based on the main consistency theorem, namely that the ABC-based estimates of the marginal posterior densities for the parameters should vary little when adding extra components to the summary statistic, densities estimated from simulated data. And that the mean of the resulting summary statistic is indeed one-to-one. This may sound somewhat similar to the stepwise search algorithm of Joyce and Marjoram (2008), but those authors aim at obtaining a vector of summary statistics that is as informative as possible. We also examine the consistency conditions when using an auxiliary model as in indirect inference. For instance, when using an AR(2) auxiliary model for estimating an MA(2) model. And ODEs.
How can one validate the outcome of a validation model? Or can we even imagine validation of this outcome? This was the starting question for the conference I attended in Hannover. Which obviously engaged me to the utmost. Relating to some past experiences like advising a student working on accelerated tests for fighter electronics. And failing to agree with him on validating a model to turn those accelerated tests within a realistic setting. Or reviewing this book on climate simulation three years ago while visiting Monash University. Since I discuss in details below most talks of the day, here is an opportunity to opt away! Continue reading
While it took quite a while (!), with several visits by three of us to our respective antipodes, incl. my exciting trip to Melbourne and Monash University two years ago, our paper on ABC for state space models was arXived yesterday! Thanks to my coauthors, Gael Martin, Brendan McCabe, and Worapree Maneesoonthorn, I am very glad of this outcome and of the new perspective on ABC it produces. For one thing, it concentrates on the selection of summary statistics from a more econometrics than usual point of view, defining asymptotic sufficiency in this context and demonstrated that both asymptotic sufficiency and Bayes consistency can be achieved when using maximum likelihood estimators of the parameters of an auxiliary model as summary statistics. In addition, the proximity to (asymptotic) sufficiency yielded by the MLE is replicated by the score vector. Using the score instead of the MLE as a summary statistics allows for huge gains in terms of speed. The method is then applied to a continuous time state space model, using as auxiliary model an augmented unscented Kalman filter. We also found in the various state space models tested therein that the ABC approach based on the marginal [likelihood] score was performing quite well, including wrt Fearnhead’s and Prangle’s (2012) approach… I like the idea of using such a generic object as the unscented Kalman filter for state space models, even when it is not a particularly accurate representation of the true model. Another appealing feature of the paper is in the connections made with indirect inference.
After about ten days in Melbourne, I am (getting) ready to move again. This longer stay in Melbourne and at Monash was quite profitable, both from a professional perspective as I had many discussions with faculty and students, gave several lectures with interesting feedback from the audience, planned MCMski IV on the side, and worked on ABC calibration, and from a personal perspective, as I recharged my batteries, shook off travel fatigue, had long and diverse runs every morning, including one to St Kilda’s beach, ate at diverse and mostly great restaurants (from Ethiopian to Thai, to French, &tc.) and truly terrific Australian wines (incl. a 20 year old Baileys of Glenrowan from Murray Smith‘s collection!).
Because of its compact downtown, I also found Melbourne easier to apprehend than Sydney, with the biases due to staying there longer and being walking distance from the centre. The Victoria market is as thriving as the last time I visited it, offering an impressive range of foods to pick from or sample on the spot. I also visited the National Gallery (Ian Potter permanent collection) enjoying very much the large collection of aboriginal paintings (as well as some of the other paintings).
I am now off for a family vacation along the Great Ocean Road and beyond so will not post (news) for a few days! Enjoy summer in the northern hemisphere/winter in the southern one, and JSM if you are in San Diego!
Yesterday night I gave my AMSI-SSAI public lecture on simulation at the University of Melbourne. Following a seminar in the early afternoon on ABC (essentially the same as in Adelaide and UWS, although I should shorten it). The seminar was well-attended, despite being during the first week of the semester and between classes. I am afraid the lecture did not draw many members of the public, though, which is not a great surprise given my esoteric (?) title, and I am afraid the academics who attended the talk did not really need this basic intro to simulations… I also visited the offices of AMSI on the campus, where I was very warmly welcomed, thank you! This even included an interview with a media officer who happened to be a Physics Honour student at the University of Melbourne, working on a cool radar data analysis. (This Honour program is an interesting entry into research that is missing in the French curriculum, providing students interested in research to spend a year mostly working on a research project right after undergraduate graduation…) In addition, it was an opportunity to look at the great posters made by AMSI to promote math in high schools with the motto “maths make your career count“. Today, I give a seminar at Monash on ABC model choice.
Here are the slides for the second day of my course at Monash University, Melbourne, in the Special Lectures in Econometrics, with a strong strong similarity with the slides of my course in Roma this Spring. (Ah, sunny Roma…) The first day lecture was very well attended and I hope this remains true for the second! (I also think I should spend more time on particle filters in general, the next time I give a similar course…)