Archive for quantile regression

Gaussian hare and Laplacian tortoise

Posted in Books, Kids, pictures, Statistics, University life with tags , , , , , , , , , , , on October 19, 2018 by xi'an

A question on X validated on the comparative merits of L¹ versus L² estimation led me to the paper of Stephen Portnoy and Roger Koenker entitled “The Gaussian Hare and the Laplacian Tortoise: Computability of Squared-Error versus Absolute-Error Estimators”, which I had missed at the time, despite enjoying a subscription to Statistical Science till the late 90’s.. The authors went as far as producing a parody of Granville’s Fables de La Fontaine by sticking Laplace’s and Gauss’ heads on the tortoise and the hare!

I remember rather vividly going through Steve Stigler’s account of the opposition between Laplace’s and Legendre’s approaches, when reading his History of Statistics in 1990 or 1991… Laplace defending the absolute error on the basis of the default double-exponential (or Laplace) distribution, when Legendre and then Gauss argued in favour of the squared error loss on the basis of a defaul Normal (or Gaussian) distribution. (Edgeworth later returned to the support of the L¹ criterion.) Portnoy and Koenker focus mostly on ways of accelerating the derivation of the L¹ regression estimators. (I also learned from the paper that Koenker was one of the originators of quantile regression.)

brief stop in Edinburgh

Posted in Mountains, pictures, Statistics, Travel, University life, Wines with tags , , , , , , , , on January 24, 2015 by xi'an

Edinburgh1Yesterday, I was all too briefly in Edinburgh for a few hours, to give a seminar in the School of Mathematics, on the random forests approach to ABC model choice (that was earlier rejected). (The slides are almost surely identical to those used at the NIPS workshop.) One interesting question at the end of the talk was on the potential bias in the posterior predictive expected loss, bias against some model from the collection of models being evaluated for selection. In the sense that the array of summaries used by the random forest could fail to capture features of a particular model and hence discriminate against it. While this is correct, there is no fundamental difference with implementing a posterior probability based on the same summaries. And the posterior predictive expected loss offers the advantage of testing, that is, for representative simulations from each model, of returning the corresponding model prediction error to highlight poor performances on some models. A further discussion over tea led me to ponder whether or not we could expand the use of random forests to Bayesian quantile regression. However, this would imply a monotonicity structure on a collection of random forests, which sounds daunting…

My stay in Edinburgh was quite brief as I drove to the Highlands after the seminar, heading to Fort William, Although the weather was rather ghastly, the traffic was fairly light and I managed to get there unscathed, without hitting any of the deer of Rannoch Mor (saw one dead by the side of the road though…) or the snow banks of the narrow roads along Loch Lubnaig. And, as usual, it still was a pleasant feeling to drive through those places associated with climbs and hikes, Crianlarich, Tyndrum, Bridge of Orchy, and Glencoe. And to get in town early enough to enjoy a quick dinner at The Grog & Gruel, reflecting I must have had half a dozen dinners there with friends (or not) over the years. And drinking a great heather ale to them!

vector quantile regression

Posted in pictures, Statistics, University life with tags , , , , , , , on July 4, 2014 by xi'an

My Paris-Dauphine colleague Guillaume Carlier recently arXived a statistics paper entitled Vector quantile regression, co-written with Chernozhukov and Galichon. I was most curious to read the paper as Guillaume is primarily a mathematical analyst working on optimisation problems like optimal transport. And also because I find quantile regression difficult to fathom as a statistical problem. (As it happens, both his co-authors are from econometrics.) The results in the paper are (i) to show that a d-dimensional (Lebesgue) absolutely continuous random variable Y can always be represented as the deterministic transform Y=Q(U), where U is a d-dimensional [0,1] uniform (the paper expresses this transform as conditional on a set of regressors Z, but those essentially play no role) and Q is monotonous in the sense of being the gradient of a convex function,

Q(u) = \nabla q(u) and \{Q(u)-Q(v)\}^\text{T}(u-v)\ge 0;

(ii) to deduce from this representation a unique notion of multivariate quantile function; and (iii) to consider the special case when the quantile function Q can be written as the linear

\beta(U)^\text{T}Z

where β(U) is a matrix. Hence leading to an estimation problem.

While unsurprising from a measure theoretic viewpoint, the representation theorem (i) is most interesting both for statistical and simulation reasons. Provided the function Q can be easily estimated and derived, respectively. The paper however does not provide a constructive tool for this derivation, besides indicating several characterisations as solutions of optimisation problems. From a statistical perspective, a non-parametric estimation of  β(.) would have useful implications in multivariate regression, although the paper only considers the specific linear case above. Which solution is obtained by a discretisation of all variables and  linear programming.

thesis in Marseille

Posted in Books, Statistics, Travel, University life with tags , , , , , on July 4, 2013 by xi'an

   IMG_0212IMG_0218

Today, I went to Marseille for a PhD thesis defence: I biked to the RER train station (yay!) and the early (7am) flight was smooth, with clear views of nuclear plants along the way… I had previously and critically refereed the thesis, called “Essays on on the econometrics of inequality and poverty measurements” ; despite its strongly applied economics title it indeed was primarily an econometric work about mixtures and quantile regression. The thesis author and PhD incumbent Abdoul Aziz Ndoye being from Senegal, he had prepared a buffet after the defence with Senegalese (yummy) delicacies that I definitely enjoyed after such an early (Oat Squares, thanks to A&C.!) breakfast. (Actually, Aziz had presented a poster in Kyoto so some of you may have met him already!) The afternoon train ride to Montpelier was smooth as well, with nice views of Provençal villages along the way. (Too bad the train line does not stick more to the coastline, though.)

While the part on mixtures was rather traditional (still using Chib’s approach to evaluate marginal likelihoods and decide about the number of components in the mixture, while “resolving” the label switching problem by using assymmetric priors based on the sample quantiles [ok, “priors”!]), I got more interested in the quantile regression part. Maybe because quantile regression is mostly new to me, I have some difficulties in getting the motivation for (regular) quantile regression: I would see an estimation of the whole conditional cdf as linear in the regressor as a more natural goal than picking one or several probability levels to estimate the corresponding quantile. Also, the thesis follows an alternative approach called RIF where the density of the observables y is first estimated by a mixture of (log-)normals and then a quantile regression is operated on

q_\tau+\mathbb{I}_{y>q_\tau}/f_Y(q_\tau),

reintroducing the explanatory variables after estimating a joint density on the y’s, which puzzles me as well. (Note that this part of the thesis, written jointly with Michel Lubrano, got a Best Presentation Prize at the Scottish Economics meeting in 2012 and got published in the associated journal.) Overall, this is an innovative and interesting piece of work, even though it cannot be completely envisionned as a Bayesian resolution.