Archive for the Books Category

Great North Road [book review]

Posted in Books, Running, Travel with tags , , , , , , , , , , , , , , , on January 6, 2017 by xi'an

As I was unsure of the Internet connections and of the more than likely delays I would face during my trip to India, I went fishing for a massive novel on Amazon and eventually ordered Peter Hamilton’s Great North Road, a 1088 pages behemoth! I fear the book qualifies as space opera, with the conventional load of planet invasions, incomprehensible and infinitely wise aliens, gateways for instantaneous space travels, and sentient biospheres. But the core of the story is very, very, Earth-bound, with a detective story taking place in a future Newcastle that is not so distant from now in many ways. (Or even from the past as the 2012 book did not forecast Brexit…) With an occurrence of the town moor where I went running a few years ago.

The book is mostly well-designed, with a plot gripping enough to keep me hooked for Indian evenings in Kolkata and most of the flight back. I actually finished it just before landing in Paris. There is no true depth in the story, though, and the science fiction part is rather lame: a very long part of the detective plot is spent on the hunt for a taxi by an army of detectives, a task one would think should be delegated to a machine-learning algorithm and solved in a nano-second or so. The themes heavily borrow from those of classics like Avatar, Speaker for the Dead, Hyperion [very much Hyperion!], Alien… And from The Girl with the Dragon Tattoo for an hardcore heroin who is perfect at anything she undertakes.  Furthermore, the Earth at the centre of this extended universe is very close to its present version, with English style taxis, pub culture, and a geopolitic structure of the World pretty much unchanged. Plus main brands identical to currents ones (Apple, BMW, &tc), to the point it sounds like sponsored links! And no clue of a major climate change despite the continued use of fuel engines. Nonetheless, an easy read when stuck in an airport or a plane seat for several hours.

quantic random generators

Posted in Books, Statistics with tags , , , , , on January 5, 2017 by xi'an

“…the random numbers should be unpredictable by any physical observer, that is, any observer whose actions are constrained by the laws of physics.”

A review paper in Nature by Acin and Masanes is the first paper I ever read there about random number generation! The central debate in the paper is about the notion of randomness, which the authors qualify as above. This seems to exclude the use of “our” traditional random number generators, although I do not see why they could not be used with an unpredictable initialisation, which does not have to be done according to a specific probability distribution. The only thing that matters is unpredictability.

“…the standard method for certifying randomness consists of running statistical tests1 on sequences generated by the device. However, it is unclear what passing these tests means and, in fact, it is impossible to certify with finite computational power that a given sequence is random.”

The paper supports instead physical and quantum devices. Justified or certified by [violations of] the Bell inequality, which separates classic from quantum. Not that I know anything about this. Or that I can make sense of the notations in the paper, like

nature20119-m1which is supposed to translate that the bits are iid Uniform and independent of the environment. Actually, I understood very little of the entire review paper, which is quite frustrating since this may well be the only paper ever published in Nature about random number generation!

“…a generation rate of 42 random bits after approximately one month of measurements, was performed using two entangled ions in two traps at 1-m distance.”

It is also hard to tell whether or not this approach to quantum random number generation has foreseeable practical consequences. There already exist QRNGs, as shown by this example from ANU. And this much more readable review.

truncated normal algorithms

Posted in Books, pictures, R, Statistics, University life with tags , , , , on January 4, 2017 by xi'an

Nicolas Chopin (CREST) just posted an entry on Statisfaction about the comparison of truncated Normal algorithms run by Alan Rogers, from the University of Utah. Nicolas wrote a paper in Statistics and Computing about a simulation method, which proposes a Ziggurat type of algorithm for this purpose, and which I do not remember reading, thanks to my diminishing memory buffer!  As shown in the picture below, when truncating to the half-line (a,∞), this method improves upon my accept-reject approach except in the far tails.

truncanormOn the top graph, made by Alan Rogers, my uniform proposal (r) seems to be doing better for a Normal truncated to (a,b) when b<0, or when a gets large and close to b. Nicolas’ ziggurat (c) works better than the Gaussian accept-reject method (c) on the positive part. (I wonder what the exponential proposal (e) stands for, in terms of scale parameter.)

recycling Gibbs auxiliaries [a reply]

Posted in Books, pictures, Statistics, University life with tags , , , , , , , on January 3, 2017 by xi'an

[Here is a reply sent to me by Luca Martino, Victor Elvira, and Gustau Camp-Vallis, after my earlier comments on their paper.]

We provide our contribution to the discussion, reporting our experience with the application of Metropolis-within-Gibbs schemes. Since in literature there are miscellaneous opinions, we want to point out the following considerations:

– according to our experience, the use of M>1 steps of the Metropolis-Hastings (MH) method for drawing from each full-conditional (with or without recycling), decreases the MSE of the estimation (see code Ex1-Ex2 and related Figure 7(b) and Figures 8). If the corresponding full conditional is very concentrated, one possible solution is to applied an adaptive or automatic MH for drawing from this full-conditional (it can require the use of M internal steps; see references in Section 3.2).

– Fixing the number of evaluations of the posterior, the comparison between a longer Gibbs chain with a single step of MH and a shorter Gibbs chain with M>1 steps of MH per each full-conditional, is required. Generally, there is no clear winner. The better performance depends on different aspects: the specific scenario, if and adaptive MH is employed or not, if the recycling is applied or not (see Figure 10(a) and the corresponding code Ex2).

The previous considerations are supported/endorsed by several authors (see the references in Section 3.2). In order to highlight the number of controversial opinions about the MH-within-Gibbs implementation, we report a last observation:

– If it is possible to draw directly from the full-conditionals, of course this is the best scenario (this is our belief). Remarkably, as also reported in Chapter 1, page 393 of the book “Monte Carlo Statistical Methods”, C. Robert and Casella, 2004, some authors have found that a “bad” choice of the proposal function in the MH step (i.e., different from the full conditional, or a poor approximation of it) can improve the performance of the MH-within-Gibbs sampler. Namely, they assert that a more “precise” approximation of the full-conditional does not necessarily improve the overall performance. In our opinion, this is possibly due to the fact that the acceptance rate in the MH step (lower than 1) induces an “accidental” random scan of the components of the target pdf in the Gibbs sampler, which can improve the performance in some cases. In our work, for the simplicity, we only focus on the deterministic scan. However, a random scan could be also considered.

Jubilee at the University of Calcutta

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , on January 2, 2017 by xi'an

The main reason for my trip to India was taking part in the celebrations of the 75th anniversary of the Department of Statistics at the University of Calcutta and of the 100th anniversary of the birth of P.K. Bose (whom I did not know before visiting Kolkata). The Department of Statistics was created in 1941 by Mahalanobis, the very first statistics department in Asia. (Mahalanobis was also instrumental in creating the ISI in 1932. And Sankhyā in 1933.)  Fisher visited Calcutta very often and was very supportive of Mahalanobis’ efforts: in the corridor, the above picture of Fisher is displayed, with him surrounded by faculties and graduates from the Department when he came in 1941.

Although I missed the first two days of the conference (!), I enjoyed very much the exchanges I had with graduate students there, about my talk on folded MCMC and other MCMC and Bayesian issues. (With The Bayesian Choice being an easy conversational bridge-way between us as it is their Bayesian textbook.) The setting reminded me of the ISBA conference in Varanasi four years ago, with the graduate students being strongly involved and providing heavy support in the organisation, as well as eager to discuss academic and non-academic issue. (Plus offering us one evening an amazing cultural show of songs and dances.) Continue reading


Posted in Books, Kids, pictures with tags , , on January 1, 2017 by xi'an

I find this xkcd entry very much in tune with my own feelings and misgivings about 2017. I like the notion that 2016 is sending us in the future without things (and people) it would have been better to keep. Like reaching out instead of building barriers, whether about staying in the EU or uniting all Americans under one’s presidency, rather than tweeting scorn, exclusion, and righteousness. Like keeping hospitals standing and operating, instead of flattening them out, in Syria, Irak, Yemen and Afghanistan. Like preserving women’s access to contraception and abortion, instead of [old men] ruling over their body and rights. No, 2017 does not look too promising.

zig, zag, and subsampling

Posted in Books, Statistics, University life with tags , , , , , , , , , on December 29, 2016 by xi'an

ENSAE, Nov. 17, 2010Today, I alas missed a seminar at BiPS on the Zig-Zag (sub-)sampler of Joris Bierkens, Paul Fearnhead and Gareth Roberts, presented here in Paris by James Ridgway. Fortunately for me, I had some discussions with Murray Pollock in Warwick and then again with Changye Wu in Dauphine that shed some light on this complex but highly innovative approach to simulating in Big Data settings thanks to a correct subsampling mechanism.

The zig-zag process runs a continuous process made of segments that turn from one diagonal to the next at random times driven by a generator connected with the components of the gradient of the target log-density. Plus a symmetric term. Provided those random times can be generated, this process is truly available and associated with the right target distribution. When the components of the parameter are independent (an unlikely setting), those random times can be associated with an inhomogeneous Poisson process. In the general case, one needs to bound the gradients by more manageable functions that create a Poisson process that can later be thinned. Next, one needs to simulate the process for the upper bound, a task that seems hard to achieve apart from linear and piecewise constant upper bounds. The process has a bit of a slice sampling taste, except that it cannot be used as a slice sampler but requires continuous time integration, given that the length of each segment matters. (Or maybe random time subsampling?)

A highly innovative part of the paper concentrates on Big Data likelihoods and on the possibility to subsample properly and exactly the original dataset. The authors propose Zig-Zag with subsampling by turning the gradients into random parts of the gradients. While remaining unbiased. There may be a cost associated with this gain of one to n, namely that the upper bounds may turn larger as they handle all elements in the likelihood at once, hence become (even) less efficient. (I am more uncertain about the case of the control variates, as it relies on a Lipschitz assumption.) While I still miss an easy way to implement the approach in a specific model, I remain hopeful for this new approach to make a major dent in the current methodologies!