## Archive for the Kids Category

## Orange the World

Posted in Kids, Uncategorized with tags International Day for the Elimination of Violence against Women, Orange Day, UNiTE to End Violence against Women on November 25, 2015 by xi'an## a programming bug with weird consequences

Posted in Kids, pictures, R, Statistics, University life with tags acceptance probability, convergence assessment, heavy-tail distribution, independent Metropolis-Hastings algorithm, Metropolis-Hastings algorithm, normal distribution, Student's t distribution on November 25, 2015 by xi'an**O**ne student of mine coded by mistake an independent Metropolis-Hastings algorithm with too small a variance in the proposal when compared with the target variance. Here is the R code of this implementation:

#target is N(0,1) #proposal is N(0,.01) T=1e5 prop=x=rnorm(T,sd=.01) ratop=dnorm(prop,log=TRUE)-dnorm(prop,sd=.01,log=TRUE) ratav=ratop[1] logu=ratop-log(runif(T)) for (t in 2:T){ if (logu[t]>ratav){ x[t]=prop[t];ratav=ratop[t]}else{x[t]=x[t-1]} }

It produces outputs of the following shape

which is quite amazing because of the small variance. The reason for the lengthy freezes of the chain is the occurrence with positive probability of realisations from the proposal with very small proposal density values, as they induce very small Metropolis-Hastings acceptance probabilities and are almost “impossible” to leave. This is due to the lack of control of the target, which is flat over the domain of the proposal for all practical purposes. Obviously, in such a setting, the outcome is unrelated with the N(0,1) target!

It is also unrelated with the normal proposal in that switching to a t distribution with 3 degrees of freedom produces a similar outcome:

It is only when using a Cauchy proposal that the pattern vanishes:

## borderline infinite variance in importance sampling

Posted in Books, Kids, Statistics with tags continuity, importance sampling, infinite variance estimators, moments, Monte Carlo experiment, Monte Carlo Statistical Methods on November 23, 2015 by xi'an**A**s I was still musing about the posts of last week around infinite variance importance sampling and its potential corrections, I wondered at whether or not there was a fundamental difference between “just” having a [finite] variance and “just” having none. In conjunction with Aki’s post. To get a better feeling, I ran a quick experiment with Exp(1) as the target and Exp(a) as the importance distribution. When estimating **E[X]**=1, the above graph opposes a=1.95 to a=2.05 (variance versus no variance, bright yellow versus wheat), a=2.95 to a=3.05 (third moment versus none, bright yellow versus wheat), and a=3.95 to a=4.05 (fourth moment versus none, bright yellow versus wheat). The graph below is the same for the estimation of **E[**exp(**X**/2)**]**=2, which has an integrand that is not square integrable under the target. Hence seems to require higher moments for the importance weight. Hard to derive universal theories from those two graphs, however they show that protection against sudden drifts in the estimation sequence. As an aside [not really!], apart from our rather confidential Confidence bands for Brownian motion and applications to Monte Carlo simulation with Wilfrid Kendall and Jean-Michel Marin, I do not know of many studies that consider the sequence of averages time-wise rather than across realisations at a given time and still think this is a more relevant perspective for simulation purposes.

## Sunday morning puzzle

Posted in Books, Kids, R with tags conditional probability, cross validated, mathematical puzzle, R on November 22, 2015 by xi'an**A** question from X validated that took me quite a while to fathom and then the solution suddenly became quite obvious:

If a sample taken from an arbitrary distribution on {0,1}⁶ is censored from its (0,0,0,0,0,0) elements, and if the marginal probabilities are know for all six components of the random vector, what is an estimate of the proportion of (missing) (0,0,0,0,0,0) elements?

Since the censoring modifies all probabilities by the same renormalisation, i.e. divides them by the probability to be different from (0,0,0,0,0,0), *ρ*, this probability can be estimated by looking at the marginal probabilities to be equal to 1, which equal the original and known marginal probabilities divided by *ρ*. Here is a short R code illustrating the approach that I wrote in the taxi home yesterday night:

#generate vectors N=1e5 zprobs=c(.1,.9) #iid example smpl=matrix(sample(0:1,6*N,rep=TRUE,prob=zprobs),ncol=6) pty=apply(smpl,1,sum) smpl=smpl[pty>0,] ps=apply(smpl,2,mean) cor=mean(ps/rep(zprobs[2],6)) #estimated original size length(smpl[,1])*cor

A broader question is how many values (and which values) of the sample can be removed before this recovery gets impossible (with the same amount of information).

## fluctuat nec cogitat

Posted in Kids, pictures with tags fundamentals on November 21, 2015 by xi'an**A**s was alas predictable, the mass assassinations in Paris last Friday led to senseless gesticulations and warmongering declarations from French political leaders. Hence my title, borrowed from Paris motto: intended to mean “Fluctuating without thinking”, or, as Google translate would inadvertently put it in a strangely appropriate way, to “turn and toss”. Martial emergency order has been declared and is contemplated for months (months!) ahead. After the “Charlie Hebdo” law on intelligence voted a few months ago (which actually goes against everything Charlie stood for!), more attacks on civil liberties are now to come, with a revision of the Constitution with the next days (days!)… Hence this drawing from Steve Bell in The Guardian: While French military forces are engaged in Irak and Syria in a dubious campaign, I completely dispute the notion that France is “at war” and see present declarations and new legislations as catering to the rightmost fringes of the public opinion. With no other clear consequences than to pave the way for the French extreme right to win the coming election(s). Just as withdrawing French citizenship from double nationals who anyway do not consider themselves as French and who are determined to blow themselves on demand is ridiculously inappropriate. The similitude between the reactions of Bush after 09/11/01 and Hollande after 11/13/15 is striking and frightening in the lack of long-term vision and of mere rational thinking… To end up with another translation, “to rock with no purpose”, indeed.

## OxWaSP seminar

Posted in Kids, Statistics, University life on November 19, 2015 by xi'an**T**his Friday, I give a seminar talk on Rao-Blackwellisation for MCMC methods to the new students of the Oxford-Warwick Statistical Programme (OxWaSP). Here are my slides, working on three papers of mine on the topic, with George Casella (1996), Randal Douc (2011), and Marco Banterle, Clara Grazian and Anthony Lee (2015).

## intractable likelihoods (even) for Alan

Posted in Kids, pictures, Statistics with tags ABC, Alan Turing Institute, consensus, decision theory, intractable likelihood, likelihood function, misspecified model, network, privacy, RKHS, Sex Pistols, summary statistics, University of Warwick on November 19, 2015 by xi'an**I**n connection with the official launch of the Alan Turing Institute (or ATI, of which Warwick is a partner), it funded an ATI Scoping workshop ~~yesterday~~ a week ago in Warwick around the notion(s) of intractable likelihood(s) and how this could/should fit within the themes of the Institute [hence the scoping]. This is one among many such scoping workshops taking place at all partners, as reported on the ATI website. Workshop that was quite relaxed and great fun, if only for getting together with most people (and friends) in the UK interested in the topic. But also pointing out some new themes I had not previously though of as related to ilike. For instance, questioning the relevance of likelihood for inference and putting forward decision theory under model misspecification, connecting with privacy and ethics [hence making intractable “good”!], introducing uncertain likelihood, getting more into network models, RKHS as a natural summary statistic, swarm of solutions for consensus inference… (And thanks to Mark Girolami for this homage to the iconic LP of the Sex Pistols!, that I played maniacally all over 1978…) My own two-cents into the discussion were mostly variations of other discussions, borrowing from ABC (and ABC slides) to call for a novel approach to approximate inference: