Archive for the pictures Category

efficient approximate Bayesian inference for models with intractable likleihood

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , on July 6, 2015 by xi'an

Awalé board on my garden table, March 15, 2013Dalhin, Villani [Mattias, not Cédric] and Schön arXived a paper this week with the above title. The type of intractable likelihood they consider is a non-linear state-space (HMM) model and the SMC-ABC they propose is based on an optimised Laplace approximation. That is, replacing the posterior distribution on the parameter θ with a normal distribution obtained by a Taylor expansion of the log-likelihood. There is no obvious solution for deriving this approximation in the case of intractable likelihood functions and the authors make use of a Bayesian optimisation technique called Gaussian process optimisation (GPO). Meaning that the Laplace approximation is the Laplace approximation of a surrogate log-posterior. GPO is a Bayesian numerical method in the spirit of the probabilistic numerics discussed on the ‘Og a few weeks ago. In the current setting, this means iterating three steps

  1. derive an approximation of the log-posterior ξ at the current θ using SMC-ABC
  2. construct a surrogate log-posterior by a Gaussian process using the past (ξ,θ)’s
  3. determine the next value of θ

In the first step, a standard particle filter cannot be used to approximate the observed log-posterior at θ because the conditional density of observed given latent is intractable. The solution is to use ABC for the HMM model, in the spirit of many papers by Ajay Jasra and co-authors. However, I find the construction of the substitute model allowing for a particle filter very obscure… (A side effect of the heat wave?!) I can spot a noisy ABC feature in equation (7), but am at a loss as to how the reparameterisation by the transform τ is compatible with the observed-given-latent conditional being unavailable: if the pair (x,v) at time t has a closed form expression, so does (x,y), at least on principle, since y is a deterministic transform of (x,v). Another thing I do not catch is why having a particle filter available prevent the use of a pMCMC approximation.

The second step constructs a Gaussian process posterior on the log-likelihood, with Gaussian errors on the ξ’s. The Gaussian process mean is chosen as zero, while the covariance function is a Matérn function. With hyperparameters that are estimated by maximum likelihood estimators (based on the argument that the marginal likelihood is available in closed form). Turning the approach into an empirical Bayes version.

The next design point in the sequence of θ’s is the argument of the maximum of a certain acquisition function, which is chosen here as a sort of maximum regret associated with the posterior predictive associated with the Gaussian process. With possible jittering. At this stage, it reminded me of the Gaussian process approach proposed by Michael Gutmann in his NIPS poster last year.

Overall, the method is just too convoluted for me to assess its worth and efficiency without a practical implementation to… practice upon, for which I do not have time! Hence I would welcome any comment from readers having attempted such implementations. I also wonder at the lack of link with Simon Wood‘s Gaussian approximation that appeared in Nature (2010) and was well-discussed in the Read Paper of Fearnhead and Prangle (2012).

snapshot from Montpellier

Posted in Books, pictures, Travel, University life, Wines with tags , , , , on July 5, 2015 by xi'an


Bayesian statistics from methods to models and applications

Posted in Books, Kids, pictures, Statistics, Travel, University life, Wines with tags , , , , , , , , on July 5, 2015 by xi'an

A Springer book published in conjunction with the great BAYSM 2014 conference in Wien last year has now appeared. Here is the table of contents:

  • Bayesian Survival Model Based on Moment Characterization by Arbel, Julyan et al.
  • A New Finite Approximation for the NGG Mixture Model: An Application to Density Estimation by Bianchini, Ilaria
  • Distributed Estimation of Mixture Model by Dedecius, Kamil et al.
  • Jeffreys’ Priors for Mixture Estimation by Grazian, Clara and X
  • A Subordinated Stochastic Process Model by Palacios, Ana Paula et al.
  • Bayesian Variable Selection for Generalized Linear Models Using the Power-Conditional-Expected-Posterior Prior by Perrakis, Konstantinos et al.
  • Application of Interweaving in DLMs to an Exchange and Specialization Experiment by Simpson, Matthew
  • On Bayesian Based Adaptive Confidence Sets for Linear Functionals by Szabó, Botond
  • Identifying the Infectious Period Distribution for Stochastic Epidemic Models Using the Posterior Predictive Check by Alharthi, Muteb et al.
  • A New Strategy for Testing Cosmology with Simulations by Killedar, Madhura et al.
  • Formal and Heuristic Model Averaging Methods for Predicting the US Unemployment Rate by Kolly, Jeremy
  • Bayesian Estimation of the Aortic Stiffness based on Non-invasive Computed Tomography Images by Lanzarone, Ettore et al.
  • Bayesian Filtering for Thermal Conductivity Estimation Given Temperature Observations by Martín-Fernández, Laura et al.
  • A Mixture Model for Filtering Firms’ Profit Rates by Scharfenaker, Ellis et al.


snapshot from Boston

Posted in pictures, Travel with tags , , , , on July 4, 2015 by xi'an


“UK outmoded universities must modernise”

Posted in Books, Kids, pictures, University life with tags , , , , on July 3, 2015 by xi'an

[A rather stinky piece in The Guardian today, written by a consultant self-styled Higher Education expert… No further comments needed!]

“The reasons cited for this laggardly response [to innovations] will be familiar to any observer of the university system: an inherently conservative and risk-averse culture in most institutions; sclerotic systems and processes designed for a different world, and a lack of capacity, skills and willingness to change among an ageing academic community. All these are reinforced by perceptions that most proposed innovations are over-hyped and that current ways of operating have plenty of life left in them yet.”

R brut

Posted in Kids, pictures, R, Statistics, University life with tags , , , on July 2, 2015 by xi'an


the (expected) demise of the Bayes factor [#2]

Posted in Books, Kids, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , on July 1, 2015 by xi'an

AmsterXXFollowing my earlier comments on Alexander Ly, Josine Verhagen, and Eric-Jan Wagenmakers, from Amsterdam, Joris Mulder, a special issue editor of the Journal of Mathematical Psychology, kindly asked me for a written discussion of that paper, discussion that I wrote last week and arXived this weekend. Besides the above comments on ToP, this discussion contains some of my usual arguments against the use of the Bayes factor as well as a short introduction to our recent proposal via mixtures. Short introduction as I had to restrain myself from reproducing the arguments in the original paper, for fear it would jeopardize its chances of getting published and, who knows?, discussed.


Get every new post delivered to your Inbox.

Join 879 other followers