Archive for Seattle

scalable Langevin exact algorithm

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , on October 18, 2016 by xi'an

“By employing a modification to existing naïve subsampling techniques we can obtain an algorithm which is still exact but has sub-linear iterative cost as a function of data size.”

A few weeks ago Murray Pollock, Paul Fearnhead, Adam Johansen and Gareth Roberts (all from Warwick except for Paul) arXived a paper The Scalable Langevin Exact Algorithm: Bayesian Inference for Big Data. (This was also the topic of Murray’s talk last year at JSM in Seattle.) One major advance found in the paper is the derivation of an “exact” algorithm that is sub-linear in the data size. As discussed in the introduction, the current approaches to large data problems either suffer from being approximate (like divide-and-conquer methods) or do not achieve significant reduction in the computing time, being of order O(n). The authors mention Teh and Welling (2011) sand their tochastic gradient approximation to the Langevin diffusion, when the gradient is based on a subsample. Without the Metropolis correction that would ensure an exact target but at a cost of order O(n). (Which makes the technique rather difficult to recommend.)

A novel [for me] notion at the core of this paper is the concept of quasi-stationary distribution, which is the limiting distribution of a Markov chain X[t] conditional on a Markov stopping time [being larger than t]. The approach is based on diffusions with appropriate stationary distributions like the Langevin diffusion. (Actually, as in most papers I have read and remember, the current paper only considers the Langevin diffusion.) In order to avoid the issues with unadjusted and Metropolis-adjusted Langevin schemes, a killed Brownian motion is created, which means a Brownian motion conditional of being alive till time T when the instantaneous killing rate is a given function of the chain, Φ(X[t]), related with the stationary measure of the Langevin diffusion ν. Under appropriate conditions, the density of this killed Brownian motion converges [in T] to √ν. Which immediately hints at creating a new Langevin diffusion targeting ν² instead of ν. And killing it with the proper rate, which can be done by thinning a Poisson process. Simulating the original path can be done by path-space rejection sampling, following the technique set by Gareth Roberts and co-authors more than ten years ago. Based on finite dimensional realisations of the path on [0,T]. And including the killing part can be done by importance sampling and checking that the simulated killing time is larger than the current (exponentially simulated) time.

One practical difficulty in the implementation of this neat principle is the derivation of the normalising constant, which evaluation degrades with the time horizon T. The solution adopted in the paper is through a sequential Monte Carlo method, using another discretisation of the time interval [0,T] (relying on the original one would get too costly?). As for subsampling, since the survival probability for the Brownian motion is based on an unbiased estimator, subsampling does not hurt if conducted in a random manner. Although this increases the variance on principle, the use of a control variate computed just once helps in reducing the complexity to O(1).

This is a tough paper and I have not gone through the effort of trying to implement it, but this is an original and innovative construct I would like to monitor in further details on a toy example, maybe next week while in Warwick. Or at least to discuss it with the authors.

photography exhibit in Seattle

Posted in Mountains, pictures, Running, Travel with tags , , , , , , , , on October 4, 2016 by xi'an

For readers in the Seattle vicinity, just mentioning that a friend and climbing guide, Brittany Aäe, is holding a photograph exhibit at Caffe Vita, 4301 Fremont Avenue North, in Seattle, this month. Enjoy.

Cadence

Posted in pictures, Travel, Wines with tags , , , , , , , on December 20, 2015 by xi'an

"another local red wine with a strong personality, Seattle, Aug. 10, 2015""more details...

superintelligence [book review]

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , , , , on November 28, 2015 by xi'an

“The first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” I.J. Good

I saw the nice cover of Superintelligence: paths, dangers, strategies by Nick Bostrom [owling at me!] at the OUP booth at JSM this summer—nice owl cover that comes will a little philosophical fable at the beginning about sparrows—and, after reading an in-depth review [in English] by Olle Häggström, on Häggström hävdar, asked OUP for a review copy. Which they sent immediately. The reason why I got (so) interested in the book is that I am quite surprised at the level of alertness about the dangers of artificial intelligence (or computer intelligence) taking over. As reported in an earlier blog, and with no expertise whatsoever in the field, I was not and am not convinced that the uncontrolled and exponential rise of non-human or non-completely human intelligences is the number One entry in Doom Day scenarios. (As made clear by Radford Neal and Corey Yanovsky in their comments, I know nothing worth reporting about those issues, but remain presumably irrationally more concerned about climate change and/or a return to barbarity than by the incoming reign of the machines.) Thus, having no competence in the least in either intelligence (!), artificial or human, or in philosophy and ethics, the following comments on the book only reflect my neophyte’s reactions. Which means the following rant should be mostly ignored! Except maybe on a rainy day like today…

“The ideal is that of the perfect Bayesian agent, one that makes probabilistically optimal use of available information.  This idea is unattainable (…) Accordingly, one can view artificial intelligence as a quest to find shortcuts…” (p.9)

Overall, the book stands much more at a philosophical and exploratory level than at attempting any engineering or technical assessment. The graphs found within are sketches rather than outputs of carefully estimated physical processes. There is thus hardly any indication how those super AIs could be coded towards super abilities to produce paper clips (but why on Earth would we need paper clips in a world dominated by AIs?!) or to involve all resources from an entire galaxy to explore even farther. The author envisions (mostly catastrophic) scenarios that require some suspension of belief and after a while I decided to read the book mostly as a higher form of science fiction, from which a series of lower form science fiction books could easily be constructed! Some passages reminded me quite forcibly of Philip K. Dick, less of electric sheep &tc. than of Ubik, where a superpowerful AI(s) turn humans into jar brains satisfied (or ensnared) with simulated virtual realities. Much less of Asimov’s novels as robots are hardly mentioned. And the third laws of robotics dismissed as ridiculously too simplistic (and too human). Continue reading

Browne

Posted in pictures, Travel, Wines with tags , , , , , on November 20, 2015 by xi'an

an enjoyable full-bodied wine from the Pacific North-West (Walla Walla, WA), Seatlle, Aug. 10, 2015"more details..."

melting hot!

Posted in Kids, pictures, Travel with tags , , , , on September 29, 2015 by xi'an

Despite majoring in her Physics class last year, my daughter forgot that microwaves are not very friendly to metal objects. Like this coffee tumbler I had brought back from Seattle, albeit not from Starbucks. The plastic part melted really well, even though the microwave oven resisted the experiment… And the coffee inside as well.

zero history [& zero memory]

Posted in Books, Travel with tags , , , , , , , on September 19, 2015 by xi'an

“There are too many improbable coincidences and the characters are definitely caricatural, while reminding me of the previous books: the female artist drawn into investigation for lack of money, the geek computer genius, the woodoo inspired ninja-like thug, the tough CIA spook, the media executive with unlimited wealth…”

During my short dash in a Seattle bookstore, when attending JSM 2015, I spotted William Gibson’s Zero history and bought it (along with the last volume in the Traitor Spy trilogy). This is not a most recent book of Gibson’s, since it first appeared in 2010, but I had somewhat missed it, after reading Spook Country in Finland a few years ago. Fact is, the book ends up a loose trilogy, along with Pattern Recognition and Spook Country of which I have very fragmented memories, fragmented enough to miss the connection with this current book. Despite it having several common characters. And the memes being just the same. (Terrible bilingual pun…) To the point of my earlier comments [reproduced as a self-quote above!] applying to this book as well. Which is not to say the book is uninteresting or dull, as I read it over a few days when I came back from Washington State. That is, across the bay from Gibson’s Vancouver.

The topic of this novel is characteristically branding and the action that takes place in the second part of the novel still relates to brand and industrial spying and commercial turf wars. There is very little of the novelistic feeling of the early books (like Neuromancer I read and reread ad nauseam in the mid 80’s). The only innovations [a wee spoiler!] are a drone with a Taser gun [about to be tested in the US!] and a tee-shirt deleting the owner from CTV footage. Oh, and an electric bike on steroids. However, the central plot of the story is quite nice, looking for the creator of an anti-brand… brand, whose clothes are sold at happenings, in very much a flash mob spirit, rather than in shops or through the Internet (even though they are mostly resold at high prices on eBay and co.). I find this notion appealing when considering the whole production of Gibson, his permanent fascination for brand and branding and for turning people outside the system inside, for either commercial or military purposes.