Archive for copula

arbitrary distributions with set correlation

Posted in Books, Kids, pictures, R, Statistics, University life with tags , , , , , , , , , , on May 11, 2015 by xi'an

A question recently posted on X Validated by Antoni Parrelada: given two arbitrary cdfs F and G, how can we simulate a pair (X,Y) with marginals  F and G, and with set correlation ρ? The answer posted by Antoni Parrelada was to reproduce the Gaussian copula solution: produce (X’,Y’) as a Gaussian bivariate vector with correlation ρ and then turn it into (X,Y)=(F⁻¹(Φ(X’)),G⁻¹(Φ(Y’))). Unfortunately, this does not work, because the correlation does not keep under the double transform. The graph above is part of my answer for a χ² and a log-Normal cdf for F amd G: while corr(X’,Y’)=ρ, corr(X,Y) drifts quite a  lot from the diagonal! Actually, by playing long enough with my function

tacor=function(rho=0,nsim=1e4,fx=qnorm,fy=qnorm)
{
  x1=rnorm(nsim);x2=rnorm(nsim)
  coeur=rho
  rho2=sqrt(1-rho^2)
  for (t in 1:length(rho)){
     y=pnorm(cbind(x1,rho[t]*x1+rho2[t]*x2))
     coeur[t]=cor(fx(y[,1]),fy(y[,2]))}
  return(coeur)
}

Playing further, I managed to get an almost flat correlation graph for the admittedly convoluted call

tacor(seq(-1,1,.01),
      fx=function(x) qchisq(x^59,df=.01),
      fy=function(x) qlogis(x^59))

zerocorNow, the most interesting question is how to produce correlated simulations. A pedestrian way is to start with a copula, e.g. the above Gaussian copula, and to twist the correlation coefficient ρ of the copula until the desired correlation is attained for the transformed pair. That is, to draw the above curve and invert it. (Note that, as clearly exhibited by the graph just above, all desired correlations cannot be achieved for arbitrary cdfs F and G.) This is however very pedestrian and I wonder whether or not there is a generic and somewhat automated solution…

ABC for copula estimation

Posted in Books, Kids, pictures, Statistics, Travel, University life with tags , , , , , , , on March 23, 2015 by xi'an

Roma from Piazzale Napoleone I, Villa Borghese, Feb. 29, 2012Clara Grazian and Brunero Liseo (di Roma) have just arXived a note on a method merging copulas, ABC, and empirical likelihood. The approach is rather hybrid and thus not completely Bayesian, but this must be seen as a consequence of an ill-posed problem. Indeed, as in many econometric models, the model there is not fully defined: the marginals of iid observations are represented as being from well-known parametric families (and are thus well-estimated by Bayesian tools), while the joint distribution remains uncertain and hence so does the associated copula. The approach in the paper is to proceed stepwise, i.e., to estimate correctly each marginal, well correctly enough to transform the data by an estimated cdf, and then only to estimate the copula or some aspect of it based on this transformed data. Like Spearman’s ρ. For which an empirical likelihood is computed and aggregated to a prior to make a BCel weight. (If this sounds unclear, each BEel evaluation is based on a random draw from the posterior samples, which transfers some uncertainty in the parameter evaluation into the copula domain. Thanks to Brunero and Clara for clarifying this point for me!)

At this stage of the note, there are two illustrations revolving around Spearman’s ρ. One on simulated data, with better performances than a nonparametric frequentist solution. And another one on a Garch (1,1) model for two financial time-series.

I am quite glad to see an application of our BCel approach in another domain although I feel a tiny bit uncertain about the degree of arbitrariness in the approach, from the estimated cdf transforms of the marginals to the choice of the moment equations identifying the parameter of interest like Spearman’s ρ. Especially if one uses a parametric copula which moments are equally well-known. While I see the practical gain in analysing each component separately, the object created by the estimated cdf transforms may have a very different correlation structure from the true cdf transforms. Maybe there exist consistency conditions on the estimated cdfs… Maybe other notions of orthogonality or independence could be brought into the picture to validate further the two-step solution…

ABC via regression density estimation

Posted in Statistics with tags , , , , , on December 19, 2012 by xi'an

Fan, Nott, and Sisson recently posted on arXiv a paper entitled Approximate Bayesian computation via regression density estimation. The theme of the paper is that one could take advantage of the joint simulation of the pair parameter/sample to derive a non-parametric estimate of the conditional distribution of the summary statistic given the parameter, i.e. the sampling distribution. While most or even all regular ABC algorithms do implicitly or explicitly rely on some level of non-parametric estimation, from Beaumont et al.’s (2002) non-parametric regression to Blum and François‘s (2010),  Fearnhead and Prangle (2012), and Biau et al. (2012) direct derivations on non-parametric convergence speeds on the kernel bandwidths, this paper centres on the idea to use those same simulations ABC relies upon to build an estimate of the sampling distribution, to be used afterwards as the likelihood in either Bayesian or frequentist inference. Rather than relying on traditional kernel estimates, the adopted approach merges mixtures of experts, namely normal regression mixtures with logit weights (Jordan and Jacobs, 1994) for the marginals, along with a copula representation of the joint distribution (of the summary statistics).

So this is a new kid on the large block of ABC methods! In terms of computing time, it sounds roughly equivalent to regular ABC algorithms in that it relies on the joint simulation of the pair parameter/sample. Plus a certain number of mixtures/mixtures of experts estimations. I have no intuition on how greedy those estimations are. In their unique illustration, the authors report density estimation in dimension 115, which is clearly impressive. I did not see any indication of respective computing times. In terms of inference and connection with the Bayesian exact posterior, I see a few potential caveats: first, the method provides an approximation of the conditional density of the summary statistics given the parameters, while the Bayesian approach considers the opposite. This could induce inefficiencies when the prior is vague and leads to a very wide spread for the values of the summary statistics. Using a neighbourhood of the observed statistics to restrict the range of the simulated statistics thus seems appropriate. (But boils down to our more standard ABC, isn’t it?!) Second, the use of mixtures of experts assume some linear connection between the parameters and the summary statistics: while this reflects Fearnhead and Prangle’s (2012) strategy, this is not necessarily appropriate in settings where those parameters cannot be expressed directly as expectations of the summary statistics (see, e.g., the case of population genetics). Third, the approximation proposed by the paper is a pluggin estimate, whose variability and imprecision are not accounted for in the inference process. Maybe not a major issue, as other solutions also rely on pluggin estimates. And I note the estimation is done once for all, when compared with, e.g., our empirical likelihood solution that requires a (fast) optimisation for each new value of the parameters. Fourth, once the approximation is constructed, a new MCMC run is required and since the (approximated) target is a black box the calibration of the MCMC sampler may turn to be rather delicate, as in the 115 dimensional example.

ISBA 2012 [#3]

Posted in Statistics, Travel, University life with tags , , , , , , on June 29, 2012 by xi'an

A third and again very intense day at ISBA 2012: as Steve Scott said, “we are  getting Bayes-ed out”… It started for me with Robert Kohn’s particle filter session, where Julien Cornebise gave us programming recommendations to improve our code, performances, and overall impact of our research, passionately pleading for an object oriented approach that would make everything we program much more portable. Scott Sisson presented a new approach to density estimation for ABC purposes, using first a marginal estimation for each component of the statistic vector, then a normal mixture copula on the normal transforms of the inverse cdfs, and Robert concluded with a extension of  PMCMC to eliminate nuisance parameters by importance sampling, a topic we will discuss again when I visit Sydney in two weeks. The second session of the morning was ABC II, where David Nott spoke about the combination of ABC with Bayes linear tools, a paper Scott had presented in Banff last Spring, Michael Blum summarised the survey on the selection of summary statistics discussed earlier on the ‘Og, Jean-Michel spoke about our (recently accepted) LDA paper, acknowledging our initial (2005) misgivings about ABC (!), and Olie Ratmann concluded the session with a fairly exciting new notion of using a testing perspective to define acceptable draws. While I clearly enjoyed the amount of “ABC talks” during this meeting, several attendees mentioned to me it was a bit overwhelming… Well, my impression is that this conveyed high and loud the message that ABC is now truly part of the Bayesian toolbox, and that further theoretical exploration would be most welcomed.

The afternoon session saw another session I was involved in organising, along with Marc Suchard, on parallel computing for Bayesian calculations. Marc motivated the use of GPUs for a huge medical dataset, showing impressive gains in time for a MAP calculation, with promises of a more complete Bayesian processing. Steve Scott gave the distributed computing version of the session, with Google requirements for a huge and superfast logistic regression, Jarad Niemi went into the (highly relevant!) details of random processors on GPUs and Kenichiro McAlinn described an application to portfolio selection using GPUs. (The topic attracted a huge crowd and the room was packed!) I am sorry the parallel session on Bayesian success stories was taking place at the same time. As it related very much to our on-going project with Kerrie Mengersen (we are currently waiting for the return from  selected authors). Then it was time for a bit of joint work, along with a succulent macha ice-cream in Kyoto station, and another fairly exhausting if quality poster session.

I am sorry to miss the sessions of Friday (and got “flak” from Arnaud for missing his lecture!) as these were promising as well. (Again, anyone for a guest post?!) Overall, I come home exhausted but richer for the exchanges and all I learn from a very good and efficient meeting. Not even mentioning this first experience of Japan. (Written from Kansai Osaka airport on a local machine.)

Confronting intractability in Bristol

Posted in pictures, Running, Statistics, Travel, University life, Wines with tags , , , , , , , , , , , , on April 18, 2012 by xi'an

Here are the (revised) slides of my talk this afternoon at the Confronting Intractability in Statistical Inference workshop in Bristol, supported by SuSTain. The novelty is in the final part, where we managed to apply our result to a three population genetic escenario using one versus two δμ summary statistics. This should be the central new example in the incoming revision of our paper to Series B.

More generally, the meeting is very interesting, with great talks and highly relevant topics: e.g., yesterday, I finally understood what transportation models meant (at the general level) and how they related to copula modelling, saw a possible connection from computer models to ABC, got inspiration to mix Gaussian processes with simulation output, and listened to the whole exposition of Simon Wood’s alternative to ABC (much more informative than the four pages of his paper in Nature!). Despite (or due to?) sampling Bath ales yesterday night, I even woke up early enough this morning to run over and under the Clifton suspension bridge, with a slight drizzle that could not really be characterized as rain…