Archive for forking paths

the ocean at the end of the lane [book review]

Posted in Books, Kids, pictures, Travel with tags , , , , , , , , , , , , , on August 24, 2018 by xi'an

While in Vancouver, waiting for a friend at the Waterfront ferry station, we entered the Simon Fraser University bookshop across the street. This was a most disconcerting experience in that the bookstore contained essentially no book! Just a tiny bookshelf with local authors and another one with a medley of genres. Including Gaiman’s The Ocean at the End of the Lane. Which I bought against my better judgement as I had tried to read American Dogs years ago and failed. (But liked very much Neverwhere, again a chance occurrence on a bookstore shelf!) As I started reading the book on the ferry to Vancouver Island, hence on the Pacific Ocean!, I first thought this was about the author’s childhood in rural Sussex, with no other friends than his books, finding some ways to relate to the story of a modest household in the early 60’s, only to be interrupted by three whales swimming along the ferry route. The cheek of them! When I picked up the short novel later in Tofino (with Tonkin Beach above), reality started to unravel (in the book!) and horror to creep in (!). Without getting into spoilers, the  other world or old country starts appearing to the narrator, a seven year old, with about everything taking another and sinister meaning. And no-one else in his household paying any attention to his warnings. What I really enjoy in the book is the sheer ambiguity of the tale, where one cannot be sure this is pure fantasy made up by a lonely seven year old who strongly dislikes a new nanny and is impacted by his parents’ relationship, or an opening into that alternate reality and its dangers that he and only he is able to enter. The book never concludes and this is a strength of the story. Which works for both adult and children readers. It also reminded me of Miyazaki’s Chihiro Spirited Away (千と千尋の神隠し), in that the supernatural beings here and there are neither evil nor good but simply utterly alien. (This fantastic² movie is considered by my daughter as the most traumatic one she ever saw as a child!) Concluding about the book, this was a very good read, somewhat on the light side although full of forking paths.

asynchronous distributed Gibbs sampling

Posted in Books, Statistics, Travel, University life with tags , , , , , , , on October 13, 2015 by xi'an

Alexander Terenin, Dan Simpson, and David Draper just arXived a paper on an alternative brand of Gibbs sampler, which they think can revolutionise the sampler and overcome its well-known bottlenecks. David had sent me the paper in advance and thus I had time to read it in the plane to Calgary. (This is also the very first paper I see acknowledging a pair of trousers..! With no connection whatsoever with bottlenecks!)

“Note that not all updates that are sent will be received by all other workers: because of network traffic congestion and other types of failures, a significant portion of the updates will be lost along the way.”

The approach is inherently parallel in that several “workers” (processors or graphical units) run Gibbs samplers in parallel, using their current knowledge of the system. This means they update a component of the model parameter, based on the information they have last received, and then send back this new value to the system. For physical reasons, there is not instantaneity in this transmission and thus all workers do not condition on the same “current” value, necessarily. Perceiving this algorithm as a “garden of forking paths” where each full conditional uses values picked at random from a collection of subchains (one for each worker), I can see why the algorithm should remain valid.

“Thus, the quality of this [ABC] method rises and falls with the ingenuity of the user in identifying nearly-sufficient statistics.”

It is also clear that this approach allows for any degree of parallelisation. However, it is less clear to me why this should constitute an improvement. With respect to the bottlenecks mentioned at the beginning of the paper, I do not truly see how the large data problem is bypassed. Except in cases where conditionals only depend on small parts of the data. Or why large dimensions can be more easily managed when compared with a plain Gibbs sampler or, better, parallel plain Gibbs samplers that would run on the same number of processors. (I do not think the paper runs the comparison in that manner, using instead a one-processor Gibbs sampler as its benchmark. Or less processors in the third example.) Since the forking paths repeatedly merge back at aperiodic stages, there is no multiplication or clear increase of the exploratory abilities of the sampler. Except for having competing proposed values [or even proposals] selected randomly. So maybe reaching a wee bit farther from time to time.