missing bit?

Nature of 7 December 2020 has a Nature Index (a supplement made of a series of articles, more journalistic than scientific, with corporate backup, which “have no influence over the content”) on Artificial Intelligence, including the above graph representing “the top 200 collaborations among 146 institutions based between 2015 and 2019, sized according to each institution’s share in artificial intelligence”, with only the UK, Germany, Switzerland and Italy identified for Europe… Missing e.g. the output from France and from its major computer science institute, INRIA. Maybe because “the articles picked up by [their] database search concern specific applications of AI in the life sciences, physical sciences, chemistry, and Earth and environmental sciences”.  Or maybe because of the identification of INRIA as such.

“Access to massive data sets on which to train machine-learning systems is one advantage that both the US and China have. Europe, on the other hand, has stringent data laws, which protect people’s privacy, but limit its resources for training AI algorithms. So, it seems unlikely that Europe will produce very sophisticated AI as a consequence”

This comment is sort of contradictory for the attached articles calling for a more ethical AI. Like making AI more transparent and robust. While having unrestricted access to personal is helping with social engineering and control favoured by dictatures and corporate behemoths, a culture of data privacy may (and should) lead to develop new methodology to work with protected data (as in an Alan Turing Institute project) and to infuse more trust from the public. Working with less data does not mean less sophistication in handling it but on the opposite! Another clash of events appears in one of the six trailblazers portrayed in the special supplement being Timnit Gebru, “former co-lead of the Ethical AI Team at Google”, who parted way with Google at the time the issue was published. (See Andrew’s blog for Jessica Hullman discussion of her firing. And the MIT Technology Review for an analysis of the paper potentially at the source of it.)

frontier of simulation-based inference

“This paper results from the Arthur M. Sackler Colloquium of the National Academy of Sciences, `The Science of Deep Learning,’ held March 13–14, 2019, at the National Academy of Sciences in Washington, DC.”

A paper by Kyle Cranmer, Johann Brehmer, and Gilles Louppe just appeared in PNAS on the frontier of simulation-based inference. Sounding more like a tribune than a research paper producing new input. Or at least like a review. Providing a quick introduction to simulators, inference, ABC. Stating the shortcomings of simulation-based inference as three-folded:

1. costly, since required a large number of simulated samples
2. loosing information through the use of insufficient summary statistics or poor non-parametric approximations of the sampling density.
3. wasteful as requiring new computational efforts for new datasets, primarily for ABC as learning the likelihood function (as a function of both the parameter θ and the data x) is only done once.

And the difficulties increase with the dimension of the data. While the points made above are correct, I want to note that ideally ABC (and Bayesian inference as a whole) only depends on a single dimension observation, which is the likelihood value. Or more practically that it only depends on the distance from the observed data to the simulated data. (Possibly the Wasserstein distance between the cdfs.) And that, somewhat unrealistically, that ABC could store the reference table once for all. Point 3 can also be debated in that the effort of learning an approximation can only be amortized when exactly the same model is re-employed with new data, which is likely in industrial applications but less in scientific investigations, I would think. About point 2, the paper misses part of the ABC literature on selecting summary statistics, e.g., the culling afforded by random forests ABC, or the earlier use of the score function in Martin et al. (2019).

The paper then makes a case for using machine-, active-, and deep-learning advances to overcome those blocks. Recouping other recent publications and talks (like Dennis on One World ABC’minar!). Once again presenting machine-learning techniques such as normalizing flows as more efficient than traditional non-parametric estimators. Of which I remain unconvinced without deeper arguments [than the repeated mention of powerful machine-learning techniques] on the convergence rates of these estimators (rather than extolling the super-powers of neural nets).

“A classifier is trained using supervised learning to discriminate two sets of data, although in this case both sets come from the simulator and are generated for different parameter points θ⁰ and θ¹. The classifier output function can be converted into an approximation of the likelihood ratio between θ⁰ and θ¹ (…) learning the likelihood or posterior is an unsupervised learning problem, whereas estimating the likelihood ratio through a classifier is an example of supervised learning and often a simpler task.”

The above comment is highly connected to the approach set by Geyer in 1994 and expanded in Gutmann and Hyvärinen in 2012. Interestingly, at least from my narrow statistician viewpoint!, the discussion about using these different types of approximation to the likelihood and hence to the resulting Bayesian inference never engages into a quantification of the approximation or even broaches upon the potential for inconsistent inference unlocked by using fake likelihoods. While insisting on the information loss brought by using summary statistics.

“Can the outcome be trusted in the presence of imperfections such as limited sample size, insufficient network capacity, or inefficient optimization?”

Interestingly [the more because the paper is classified as statistics] the above shows that the statistical question is set instead in terms of numerical error(s). With proposals to address it ranging from (unrealistic) parametric bootstrap to some forms of GANs.

FALL [book review]

The “last” book I took with me to Japan is Neal Stephenson’s FALL. With subtitle “Dodge in Hell”. It shares some characters with REAMDE but nothing prevents reading it independently as a single volume. Or not reading it at all! I am rather disappointed by the book and hence  sorry I had to carry it throughout Japan and back. And slightly X’ed at Nature writing such a positive review. And at The Guardian. (There is a theme there, as I took REAMDE for a trip to India with a similar feeling at the end. Maybe the sheer weight of the book is pulling my morale down…) The most important common feature to both books is the game industry, since the main (?) character is a game company manager, who is wealthy enough to ensure the rest of the story holds some financial likelihood. And whose training as a game designer impacts the construction of the afterlife that takes a good (or rather terrible) half of the heavy volume. The long minutes leading to his untimely death are also excruciatingly rendered (with none of the experimental nature of Leopold Bloom’s morning). With the side information that Dodge suffers from ocular migraine, a nuisance that visits me pretty regularly since my teenage years! The scientific aspects of the story are not particularly exciting either, since the core concept is that by registering the entire neuronal network of the brain of individuals after their death, a computer could revive them by simulating this network. With dead people keeping their personality if very little of their memories. And even more fanciful, interacting between them and producing a signal that can be understood by (living) humans. Despite having no sensory organs. The reconstruction of a world by the simulated NNs is unbearably slow and frankly uninteresting as it reproduces both living behaviours and borrows very heavily from the great myths, mostly Greek, with no discernible depth. The living side of the story is not much better, although with a little touch of the post-apocalyptic flavour I appreciated in Stephenson. But not enough to recover from the fall.

Among other things that set me off with the book, the complete lack of connection with the massive challenges currently facing humanity. Energy crisis? climate change? Nope. Keep taking an hydroplane to get from Seattle to islands on Puget Sound? Sure. Spending abyssal amounts of energy to animate this electronic Hades? By all means. More and more brittle democracies? Who cares, the Afterworld is a pantheon where gods clash and rule lower beings. Worse, the plot never reaches beyond America, from the heavily focused philosophical or religious background to the character life trajectories. Characters are surprisingly unidimensional, with no default until they become evil. Or die. Academics are not even unidimensional. For instance Sophie’s thesis defence is at best a chat in a café… And talks at a specialist workshop switch from impressive mathematical terms to a 3D representation of the activity of the simulated neuronal networks. Whille these few individuals keep impacting the whole World for their whole life. And beyond… By comparison, the Riverworld series of Phillip José Farmer (that I read forty years ago) is much more enjoyable as a tale of the Afterworld, even if one can object at “famous” people been central to the action. At least there are more of them and, judging from their (first) life, they may have interesting and innovative to say.

the $1,547.02 book on neural networks