art brut [no!]

Posted in pictures, Statistics, Travel with tags , , , , , , , , , , , , on June 27, 2019 by xi'an


Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , on June 26, 2019 by xi'an

I found my (short) trip to Abdijan for the CISEA 2019 conference quite fantastic as it allowed me to meet with old friends, from the earliest days at CREST and even before, and to meet new ones. Including local students of ENSEA who had taken a Bayesian course out of my Bayesian Choice book. And who had questions about the nature of priors and the difficulty they had in accepting that several replies were possible with the same data! I wish I had had more time to discuss the relativity of Bayesian statements with them but this was a great and rare opportunity to find avid readers of my books! I also had a long chat with another student worried about the use or mis-use of reversible jump algorithms to draw inference  on time-series models in Bayesian Essentials, chat that actually demonstrated his perfect understanding of the matter. And it was fabulous to meet so many statisticians and econometricians from West Africa, most of them French-speaking. My only regret is not having any free time to visit Abidjan or the neighbourhood as the schedule of the conference did not allow for it [or even for a timely posting of a post!], especially as it regularly ran overtime. (But it did provide for a wide range of new local dishes that I definitely enjoyed tasting!) We are now discussing further opportunities to visit there, e.g. by teaching a short course at the Master or PhD levels.

Bayesian conjugate gradients [open for discussion]

Posted in Books, pictures, Statistics, University life with tags , , , , , on June 25, 2019 by xi'an

When fishing for an illustration for this post on Google, I came upon this Bayesian methods for hackers cover, a book about which I have no clue whatsoever (!) but that mentions probabilistic programming. Which serves as a perfect (?!) introduction to the call for discussion in Bayesian Analysis of the incoming Bayesian conjugate gradient method by Jon Cockayne, Chris Oates (formerly Warwick), Ilse Ipsen and Mark Girolami (still partially Warwick!). Since indeed the paper is about probabilistic numerics à la Mark and co-authors. Surprisingly dealing with solving the deterministic equation Ax=b by Bayesian methods. The method produces a posterior distribution on the solution x⁰, given a fixed computing effort, which makes it pertain to the anytime algorithms. It also relates to an earlier 2015 paper by Christian Hennig where the posterior is on A⁻¹ rather than x⁰ (which is quite a surprising if valid approach to the problem!) The computing effort is translated here in computations of projections of random projections of Ax, which can be made compatible with conjugate gradient steps. Interestingly, the choice of the prior on x is quite important, including setting a low or high convergence rate…  Deadline is August 04!

efficient MCMC sampling

Posted in Statistics with tags , , , on June 24, 2019 by xi'an

Maxime Vono, Daniel Paulin and Arnaud Doucet recently arXived a paper about a regularisation technique that allows for efficient sampling from a complex posterior which potential function factorises as a large sum of transforms of linear projections of the parameter θ

U(\theta)=\sum_i U_i(A_i\theta)

The central idea in the paper [which was new to me] is to introduce auxiliary variates for the different terms in the sum, replacing the projections in the transforms, with an additional regularisation forcing these auxiliary variates to be as close as possible from the corresponding projection

U(\theta,\mathbf z)=\sum_i U_i(z_i)+\varrho^{-1}||z_i-A_i\theta||^2

This is only an approximation to the true target but it enjoys the possibility to run a massive Gibbs sampler in quite a reduced dimension. As the variance ρ of the regularisation term goes to zero the marginal posterior on the parameter θ converges to the true posterior. The authors manage to achieve precise convergence rates both in total variation and in Wasserstein distance.

From a practical point of view, only judging from the logistic example, it is hard to fathom how much this approach improves upon other approaches (provided they still apply) as the impact of the value of ρ should be assessed on top of the convergence of the high-dimensional Gibbs sampler. Or is there an annealing version in the pipe-line? While parallelisation is a major argument, it also seems that the Gibbs sampler need a central monitoring for each new simulation of θ. Unless some asynchronous version can be implemented.

A la Bienale di Venezia

Posted in Books, pictures, Travel with tags , , , , , , , , , , on June 23, 2019 by xi'an

Taking advantage of staying in Venezia over the weekend, we went to the huge international contemporary art exhibit located all over the city but mostly in the Arsenale and in the gardens. This was quite impressive in terms of diversity and style, of course, although the general feeling was rather bleak, centering on pollution and apocalyptic themes. The particularly ugly French exhibit was for instance a highly polluted sea surface, made of glass and only accessible by going around piles of gravel in the basement of the pavilion. Most exhibits also involved videos, often not making much sense, and comparatively few paintings or photographs. Within this depressing catalogue, a few beautiful highlights from my own perspective. One was a construct of several thousands shell-like objects, sculpted from sheep leather by Zahrah Al Ghamdi, a female Saudi Arabia artist Another one, representing Ghana, by the artists El Anatsui and Ibrahim Mahama, recycled aluminum stickers into huge maps, reminding me of the recycled maps in Munbai airport.Yet another one, difficult to catch, was a huge construct from the Philippines by Mark Justiniani, made of glass that gave an impression of infinite depth and again recycled different objects into wells, reminding me of the automated art pieces appearing in Gibson’s Count Zero. Called “Island Weather” to reflect upon the elusive nature of truth and the notion that everyone is an island, with bottomless layers of accumulated memories.

A series [called Angst] of remarkable night photographs by Soham Gupta of some inhabitants of the slums in Kolkata where the persons chose to act in relation with the hardship or trauma that led them to survive in the street. And still exhibiting joy and engaging into farciful behaviours. A video was however striking [from my perspective], describing the fight of a Nunavuk father to prevent his children being sent far away for schooling by the Canadian government, as it reminded me of a so different time when, as a child then, a catholic missionary from the Far North had come to our primary school and told us fascinating stories of the cruelly beautiful (or beautifully cruel?) like in the Arctic, in what did not appear yet as a strongly biased manner… The title of the Bienale this year was May you live in interesting times, which prompted many attendees to scrawl Theresa May you leave in interesting times over the exhibit panels! Interesting if bleak times indeed.

cuida tu vestido y no hables tanto, Jorge!

Posted in Books, pictures with tags , , , , , , , , , , on June 22, 2019 by xi'an

“”In our societies, it even seems homosexuality is fashionable. And this mentality, in some way, also influences the life of the Church (..) Therefore, the Church recommends that people with that kind of ingrained tendency should not be accepted into the ministry or consecrated life. The ministry or the consecrated life is not his place.” [En nuestras sociedades parece incluso que la homosexualidad está de moda y esa mentalidad, de alguna manera, también influye en la vida de la Iglesia (…) En la vida consagrada y en la vida sacerdotal, ese tipo de afectos no tienen cabida] 2 December 2018

“One cannot live a whole life of accusing, accusing, accusing, the Church. Those who spend their lives accusing, accusing, accusing are not the devil’s children because the devil has none. [They are] friends, cousins and relatives of the devil, and this is wrong.” 22 February 2019

“It is becoming increasingly clear that we are now facing with what might accurately be called an educational crisis, especially in the field of affectivity and sexuality. [Gender theory] denies the difference and reciprocity in nature of a man and a woman and envisages a society without sexual differences, thereby eliminating the anthropological basis of the family. This ideology leads to educational programs and legislative enactments that promote a personal identity and emotional intimacy radically separated from the biological difference between male and female. Consequently, human identity becomes the choice of the individual, one which can also change over time.” Male and Female He Created Them: Towards a path of dialogue on the question of gender in education, June 2019

sampling and imbalanced

Posted in Statistics with tags , , , , , on June 21, 2019 by xi'an

Deborshee Sen, Matthias Sachs, Jianfeng Lu and David Dunson have recently arXived a sub-sampling paper for  classification (logistic) models where some covariates or some responses are imbalanced. With a PDMP, namely zig-zag, used towards preserving the correct invariant distribution (as already mentioned in an earlier post on the zig-zag zampler and in a recent Annals paper by Joris Bierkens, Paul Fearnhead, and Gareth Roberts (Warwick)). The current paper is thus an improvement on the above. Using (non-uniform) importance sub-sampling across observations and simpler upper bounds for the Poisson process. A rather practical form of Poisson thinning. And proposing unbiased estimates of the sub-sample log-posterior as well as stratified sub-sampling.

I idly wondered if the zig-zag sampler could itself be improved by not switching the bouncing directions at random since directions associated with almost certainly null coefficients should be neglected as much as possible, but the intensity functions associated with the directions do incorporate this feature. Except for requiring computation of the intensities for all directions. This is especially true when facing many covariates.

Thinking of the logistic regression model itself, it is sort of frustrating that something so close to an exponential family causes so many headaches! Formally, it is an exponential family but the normalising constant is rather unwieldy, especially when there are many observations and many covariates. The Polya-Gamma completion is a way around, but it proves highly costly when the dimension is large…