nested sampling: any prior anytime?!

A recent arXival by Justin Alsing and Will Handley on “nested sampling with any prior you like” caught my attention. If only because I was under the impression that some priors would not agree with nested sampling. Especially those putting positive weight on some fixed levels of the likelihood function, as well as improper priors.

“…nested sampling has largely only been practical for a somewhat restrictive class of priors, which have a readily available representation as a transform from the unit hyper-cube.”

Reading from the paper, it seems that the whole point is to demonstrate that “any proper prior may be transformed onto the unit hypercube via a bijective transformation.” Which seems rather straightforward if the transform is not otherwise constrained: use a logit transform in every direction. The paper gets instead into the rather fashionable direction of normalising flows as density representations. (Which suddenly reminded me of the PhD dissertation of Rob Cornish at Oxford, which I examined last year. Even though nested was not used there in the same understanding.) The purpose appearing later (in the paper) or in fine to express a random variable simulated from the prior as the (generative) transform of a Uniform variate, f(U). Resuscitating the simulation from an arbitrary distribution from first principles.

“One particularly common scenario where this arises is when one wants to use the (sampled) posterior from one experiment as the prior for another”

But I remained uncertain at the requirement for this representation in implementing nested sampling as I do not see how it helps in bypassing the hurdles of simulating from the prior constrained by increasing levels of the likelihood function. It would be helpful to construct normalising flows adapted to the truncated priors but I did not see anything related to this version in the paper.

The cosmological application therein deals with the incorporation of recent measurements in the study of the ΛCDM cosmological model, that is, more recent that the CMB Planck dataset we played with 15 years ago. (Time flies, even if an expanding Universe!) Namely, the Baryon Oscillation Spectroscopic Survey and the SH₀ES collaboration.

a bad graph about Hubble discrepancies

Here is a picture seen in a Nature Reviews Physics paper I came across, on  the Hubble constant being consistently estimated as large now than previously. I have no informed comment to make on the paper, which thinks that these discrepancies support altering the composition of the Universe shortly before the emergence of the Cosmological Background Noise (CMB), but the way it presented the confidence assessments of the same constant H⁰ based on 13 different experiments is rather ghastly, from using inclined confidence intervals, to adding a USA Today touch to the graph via a broken bridge and a river below, to resorting to different scales for both parts of the bridge…  

big bang/data/computers

I missed this astrostatistics conference announcement (and the conference itself, obviously!), occurring next door… Actually, I would have had (wee) trouble getting there as I was (and am) mostly stuck at home with a bruised knee and a doctor ban on any exercise in the coming day, thanks to a bike fall last Monday! (One of my 1991 bike pedals broke as I was climbing a steep slope and I did not react fast enough… Just at the right time to ruin my training preparation of the Argentan half-marathon. Again.) Too bad because there was a lot of talks that were of interest to me!

550 billion particles

“Space,” it says, “is big. Really big. You just won’t believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space, listen…” The Hitchhiker’s Guide to the Galaxy, Douglas Adams

“There is a theory which states that if ever anyone discovers exactly what the Universe is for and why it is here, it will instantly disappear and be replaced by something even more bizarre and inexplicable. There is another theory which states that this has already happened.” The Hitchhiker’s Guide to the Galaxy, Douglas Adams

Following a link on Science Daily when looking at this 64 kcal mystery, I found an interesting annoucement about the most complete simulation of the evolution of the Universe from the Big Bang till now. The cosmology research unit in charge of the project is furthermore called DEUS (for Dark Energy Universe Simulation!), mostly located at Université Paris-Diderot, and its “goal is to investigate the imprints of dark energy on cosmic structure formation through high-performance numerical simulations”. It just announced the “simulation of the full observable universe for the concordance ΛCDM model”, which allows for the comparison of several cosmological models. (Data is freely available.) Besides the sheer scientific appeal of the project, the simulation side is also fascinating, although quite remote from Monte Carlo principles, in that the approach relies on very few repetitions of the simulation. The statistics are based on a single simulation, for a completely observed (simulated) Universe.

“If life is going to exist in a Universe of this size, then the one thing it cannot afford to have is a sense of proportion…” The Hitchhiker’s Guide to the Galaxy, Douglas Adams

The amounts involved in this simulation are simply mindboggling: 92 000 CPUs,  150 PBytes of data, 2 (U.S.) quadrillion flops (2 PFlop/s), the equivalent of 30 million computing hours, each particle has the size of the Milky Way, and so on… Here is a videoed description of the project (make sure to turn the sounds off if, like me, you simply and definitely hate Strauss’ music, and even if you like it, since the pictures do not move at the same pace as the music!):

Core in CiRM [1]

Jean-Michel Marin and myself have thus started our “research in pair” in CIRM, Luminy, for a fortnight. We are working on the second edition of Bayesian Core and, despite working round the clock on the project (except for a one hour run around Mont Puget this morning), we are not going as fast as planned… Today, we worked in parallel on the normal and the regression chapters, looking for a sexy normal dataset to replace the larceny (normaldata) and the large and delicate CMB datasets. We eventually settled for a modern version of the Michelson-Morley dataset (available in R as morley), produced by K.K. Illingworth in 1927. I hope the spectral data and the relevance of the experiment will not be lost on the readers.