flow contrastive estimation

On the flight back from Montpellier, last week, I read a 2019 paper by Gao et al. revisiting the MLE estimation of a parametric family parameter when the normalising constant Z=Z(θ) is unknown. Via noise-contrastive estimation à la Guttman & Hyvärinnen (or à la Charlie Geyer). Treating the normalising constant Z as an extra parameter (as in Kong et al.) and the classification probability as an objective function and calling it a likelihood, which it is not in my opinion as (i) the allocation to the groups is not random and (ii) the original density of the actual observations does not appear in the so-called likelihood.

“When q appears on the right of KL-divergence [against p],  it is forced to cover most of the modes of p, When q appears on the left of KL-divergence, it tends to chase the major modes of p while ignoring the minor modes.”

The flow in the title indicates that the contrastive distribution q is estimated by a flow-based estimator, namely the transform of a basic noise distribution via easily invertible and differentiable transforms, for instance with lower triangular Jacobians. This flow is also estimated directly from the data but the authors complain this estimation is not good enough for noise contrastive estimation and suggest instead resorting to a GAN version where the classification log-probability is maximised in the model parameters and minimsed in the flow parameters. Except that I feel it misses the true likelihood part. In other words, why on Hyperion would estimating all θ, Z=Z(θ), and α at once improve the estimation of Z?

The other aspect that puzzles me is that (12) uses integrated classification probabilities (with the unknown Z as extra parameter), rather than conditioning on the data, Bayes-like. (The difference between (12) and GAN is that here the discriminator function is constrained.) Esp. when the first expectation is replaced with its empirical version.

the 101 favourite novels of Le Monde readers

Le Monde called its readers to vote for their five favourite novels, with no major surprise in the results, except maybe Harry Potter coming up top. Before Voyage au bout de la nuit and (the predictable) A la recherche du temps perdu. And a complete unknown, Damasio’s La Horde du Contrevent, as 12th and first science fiction book. Above both the Foundation novels (16th). And Dune (32nd). And Hyperion Cantos (52). But no Jules Verne! In a sense, it reflects upon the French high school curriculum on literature that almost uniquely focus on French 19th and 20th books. (Missing also Abe, Conrad, Chandler, Dickens, Ishiguro, Joyce, Kawabata, Madame de Lafayette, Levi, Morante, Naipaul, Rabelais, Rushdie, Singer, and so many others…) Interestingly (or not), Sartre did not make it to the list, despite his literature 1953 Nobel Prize, maybe because so few read the (appalling) books of his chemins de la liberté trilogy.

I did send my vote in due time but cannot remember for certain all the five titles I chose except for Céline’s Voyage au bout de la nuit (2nd), Cormac McCarthy’s The Road (74th) and maybe Fedor Dostoievski’s Brothers Karamazov (24th). Maybe not as I may have included Barbey d’Aurevilly’s L’ensorcelée, Iain Pears’ An instance at the fingerpost, and Graham Greene’s The End of the affair, neither of which made it in the list. Here are some books from the list that would have made it to my own 101 list, although not necessarily as my first choice of titles for authors like Hugo (1793!) or Malraux (l’Espoir). (Warning: Amazon Associate links).

• Harry Potter
• Voyage au bout de la nuit
• One hundred years of solitude
• The Lord of the rings
• L’étranger
• Belle du seigneur
• Les misérables
• Mémoires d’Hadrien
• Pride and prejudices
• The unbearable lightness of being
• Foundation
• Crime and punishment
• Madame Bovary
• Le rouge et le noir
• The Brothers Karamazov
• La peste
• War and peace
• The grapes of wrath
• Les liaisons dangereuses
• Bel-Ami
• Anna Karenina
• The name of the rose
• Germinal
• To kill a mocking bird
• The three musketeers
• Kafka on the shore
• La vie mode d’emploi
• Lolita
• Wuthering heights
• Mort à crédit
• The picture of Dorian Gray
• Life and fate
• Chess story
• La chartreuse de Parme
• L’œuvre au noir
• The idiot
• L’assommoir
• The road
• Le père Goriot
• The trial
• Don Quixote
• Notre-Dame de Paris
• Illusions perdues
• The sound and the fury
• La condition humaine

space opera by John Scalzi [book review]

John Scalzi, author of the memorable Old Man’s War, has started a trilogy of which I only became aware recently (or more precisely became re-aware!), which has the perk of making two of the three books already published and hence available without a one or two year break. And having the book win the 2018 Locus Award in the meanwhile. This new series is yet again a space opera with space travel made possible by a fairly unclear Flow that even the mathematicians in the story have trouble understanding. And The Flow is used by guilds to carry goods and people to planets that are too hostile an environment for the “local” inhabitants to survive on their own. The whole setup is both homely and old-fashioned: the different guilds are associated with families, despite being centuries old, and the empire of 48 planets is still governed by the same dominant family, who also controls a fairly bland religion. Although the later managed to become the de facto religion.

“I’m a Flow physicist.  It’s high-order math. You don’t have to go out into the field for that.”

This does not sound much exciting, even for space operas, but things are starting to deteriorate when the novels start. Or more exactly, as hinted by the title, the Empire is about to collapse! (No spoiler, since this is the title!!!) However, the story-telling gets a wee bit lazy from that (early) point. In that it fixates on a very few characters [among millions of billions of inhabitants of this universe] who set the cogs spinning one way then the other then the earlier way… Dialogues are witty and often funny, those few characters are mostly well-drawn, albeit too one-dimensional, and cataclysmic events seem to be held at bay by the cleverness of one single person, double-crossing the bad guys. Mostly. While the second volume (unusually) sounds better and sees more action, more surprises, and an improvement in the plot itself, and while this makes for a pleasant travel read (I forgot The Collapsing Empire in a plane from B’ham!), I am surprised at the book winning the 2018 Locus Award indeed. It definitely lacks the scope and ambiguity of the two Ancillary novels. The convoluted philosophical construct and math background of Anathem. The historical background of Cryptonomicon and of the Baroque Cycle. Or the singularity of the Hyperion universe. (But I was also unimpressed by the Three-Body Problem! And by Scalzi’s Hugo Award Redshirts!) The third volume is not yet out.

As a French aside, a former king turned AI is called Tomas Chenevert, on a space-ship called Auvergne, with an attempt at coming from a French speaking planet, Ponthieu, except that is should have been spelled Thomas Chênevert (green oak!). Incidentally, Ponthieu is a county in the Norman marches, north of Rouen, that is now part of Picardy, although I do not think this has anything to do with the current novel!

Great North Road [book review]

As I was unsure of the Internet connections and of the more than likely delays I would face during my trip to India, I went fishing for a massive novel on Amazon and eventually ordered Peter Hamilton’s Great North Road, a 1088 pages behemoth! I fear the book qualifies as space opera, with the conventional load of planet invasions, incomprehensible and infinitely wise aliens, gateways for instantaneous space travels, and sentient biospheres. But the core of the story is very, very, Earth-bound, with a detective story taking place in a future Newcastle that is not so distant from now in many ways. (Or even from the past as the 2012 book did not forecast Brexit…) With an occurrence of the town moor where I went running a few years ago.

The book is mostly well-designed, with a plot gripping enough to keep me hooked for Indian evenings in Kolkata and most of the flight back. I actually finished it just before landing in Paris. There is no true depth in the story, though, and the science fiction part is rather lame: a very long part of the detective plot is spent on the hunt for a taxi by an army of detectives, a task one would think should be delegated to a machine-learning algorithm and solved in a nano-second or so. The themes heavily borrow from those of classics like Avatar, Speaker for the Dead, Hyperion [very much Hyperion!], Alien… And from The Girl with the Dragon Tattoo for an hardcore heroin who is perfect at anything she undertakes.  Furthermore, the Earth at the centre of this extended universe is very close to its present version, with English style taxis, pub culture, and a geopolitic structure of the World pretty much unchanged. Plus main brands identical to currents ones (Apple, BMW, &tc), to the point it sounds like sponsored links! And no clue of a major climate change despite the continued use of fuel engines. Nonetheless, an easy read when stuck in an airport or a plane seat for several hours.

Fall of Hyperion

“I have weighted the consequences of this remote… one would have to say statistically impossible… event many times. We find the risks acceptable. Should the impossible happen (…)” The Fall of Hyperion, p. 119

As I had immensely enjoyed Dan Simmons’ Hyperion a few weeks ago, I immediately went for the sequel/conclusion, The Fall of Hyperion. I alas found the second volume much less enjoyable as it was much less inspired and stunningly surprising than the first one… As illustrated by this very well-articulated criticism, I am certainly not the only one to feel as such. The innovative reproduction of Chauncer’s Canterbury Tales that makes stories within the story told in different styles is lost, and so is the complexity of the seven pilgrims, perfect in their imperfection: it seems like they caught a strange form of disease between both volumes, now solely working for the common good and ready to sacrifice themselves over and over again (since they can more or less resurrect!).

“First, that Abraham’s path of obedience can no longer be followed, even if ther eis a God demanding such obedience. Second, that we have offered too many sacrifices to that God for too many generations… that the payments of pain must stop.” The Fall of Hyperion, p.226

The fact that Simmons borrowed from many sources and myths was exciting in the first volume, but it showed its limits here, as he borrowed just too much! E.g., the connection between Sol and Abraham, the former being a scholar and philosopher spending his life analysing (and fighting) the motives for God requesting Abraham to sacrifice his son Isaac, but eventually giving his (Sol’s) sole daughter Rachel to the (monster) Shrike as depicted on the (ugly) cover of the book.

“I am merely a poet dying far from home.” The Fall of Hyperion, p.427

The other (original) idea of transplanting the poet Keats in this future dries out fast and the long agony of Keats in Roma does not add anything to the story line. Overall, I think that the second volume, The Fall of Hyperion, explains a lot (and too much) about the first book, Hyperion, but I am of the opinion that the story would have read better without those explanations, thoe too many deus ex machina, and suspensions of belief, and other points of view from implausible characters. A bit more work on the first book would have kept the magic and the mystery there, without going the easy path of space-opera war councils, malevolent AIs, and ambiguous cyborgs… So I really advise against reading The Fall of Hyperion, if you have not done so yet. (I know, I know: this is a 1990 novel so most people who could have read it must have read it!)