Archive for Berlin Tegel flughafen

a case for Bayesian deep learnin

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , on September 30, 2020 by xi'an

Andrew Wilson wrote a piece about Bayesian deep learning last winter. Which I just read. It starts with the (posterior) predictive distribution being the core of Bayesian model evaluation or of model (epistemic) uncertainty.

“On the other hand, a flat prior may have a major effect on marginalization.”

Interesting sentence, as, from my viewpoint, using a flat prior is a no-no when running model evaluation since the marginal likelihood (or evidence) is no longer a probability density. (Check Lindley-Jeffreys’ paradox in this tribune.) The author then goes for an argument in favour of a Bayesian approach to deep neural networks for the reason that data cannot be informative on every parameter in the network, which should then be integrated out wrt a prior. He also draws a parallel between deep ensemble learning, where random initialisations produce different fits, with posterior distributions, although the equivalent to the prior distribution in an optimisation exercise is somewhat vague.

“…we do not need samples from a posterior, or even a faithful approximation to the posterior. We need to evaluate the posterior in places that will make the greatest contributions to the [posterior predictive].”

The paper also contains an interesting point distinguishing between priors over parameters and priors over functions, ony the later mattering for prediction. Which must be structured enough to compensate for the lack of data information about most aspects of the functions. The paper further discusses uninformative priors (over the parameters) in the O’Bayes sense as a default way to select priors. It is however unclear to me how this discussion accounts for the problems met in high dimensions by standard uninformative solutions. More aggressively penalising priors may be needed, as those found in high dimension variable selection. As in e.g. the 10⁷ dimensional space mentioned in the paper. Interesting read all in all!

flight to Berlin

Posted in Kids, pictures, Travel with tags , , , , , on December 11, 2012 by xi'an

Berlin Tegel Flughafen, Dec. 9, 2012Tough ride to Berlin, although nothing terrible in the end! First, the plane departure was quite delayed in Paris by snow falling over Berlin, meaning the runways had to be cleaned. Second, loud kids in the seat behind me with a fairly ineffective if equally loud father repeating he was going to get angry if they did not stop…and eventually switching seats with his wife who surprise surprise managed much much better! Third, more delays in Berlin as the runways were only partly cleared and all gates were busy and the bags took more than one hour to come…

Here is a sample of the conversation in the plane (French fluency clearly superfluous!, but I could not transcribe Henri’s part, as it was half in French, halbst im Deutsch…):

Henri, ce n’est pas gentil, tu arrêtes de donner des coups de pied à Joséphine!

Attaches-toi, Henri, sinon Papa va se fâcher…

Donne-lui des serviettes pour essuyer, chacun une. Fais-le bien…

Encore un gâteau, Henri ? Je vais demander au stewart.. Maman dit non, plus de gâteau.

Tu lui as fait mal, Henri, ramasse ton biberon. Et le chapeau du biberon, il est où?

Henri, tu te rattaches et tu te détaches plus, sinon Papa se met en colère.