ABConic mean evidence approximation

Following a question on X validated about evidence approximation in ABC settings, i.e., on returning an approximation of the evidence based on the outputs of parallel ABC runs for the models under comparison, I wondered at the relevance of an harmonic mean estimator in that context.

Rather than using the original ABC algorithm that proposes a model, a parameter from that model, and a simulated dataset from that model with that parameter,  an alternate, cost-free, solution would be to run an ABC version of [harmonic mean evidence approximation à la Newton & Raftery (1994). Since

the evidence can formally be approximated by

and its ABC version is

where K_ε(.) is the kernel used for the ABC acceptance/rejection step and d(.,.) is the distance used to measure the discrepancy between samples. Since the kernel values are already computed for evidence, the cost is null. Obviously, an indicator kernel does not return a useful estimate but something like a Cauchy kernel could do.

However, when toying with a normal-normal model and calibrating the Cauchy scale to fit the actual posterior as in the above graph, the estimated evidence 5 10⁻⁵ proved much smaller than the actual one, 8 10⁻².

ratio of Gaussians

Following (as usual) an X validated question, I came across two papers of George Marsaglia on the ratio of two arbitrary (i.e. unnormalised and possibly correlated) Normal variates. One was a 1965 JASA paper,

where the density of the ratio X/Y is exhibited, based on the fact that this random variable can always be represented as (a+ε)/(b+ξ) where ε,ξ are iid N(0,1) and a,b are constant. Surprisingly (?), this representation was challenged in a 1969 paper by David Hinkley (corrected in 1970).

And less surprisingly the ratio distribution behaves almost like a Cauchy, since its density is

meaning it is a two-component mixture of a Cauchy distribution, with weight exp(-a²/2-b²/2), and of an altogether more complex distribution ƒ². This is remarked by Marsaglia in the second 2006 paper, although the description of the second component remains vague, besides a possible bimodality. (It could have a mean, actually.) The density ƒ² however resembles (at least graphically) the generalised Normal inverse density I played with, eons ago.

order, order!

A very standard (one-line) question on X validated, namely whether min(X,Y) could enjoy a finite mean when both X and Y had infinite means [the answer is yes, possibly!] brought a lot of traffic, including an incorrect answer and bringing it to be one of the “Hot Network Questions“, for no clear reason. Beside my half-Cauchy example, some answers pointed out the connection between mean and cdf, as integrated cdf on the negative half-line and integrated complement cdf on the positive half-line, and between mean and quantile function, as

since it nicely expands to

but I remain bemused by the excitement..! (Including the many answers and the lack of involvement of the OP.)

more concentration, everywhere

Although it may sound like an excessive notion of optimality, one can hope at obtaining an estimator δ of a unidimensional parameter θ that is always closer to θ that any other parameter. In distribution if not almost surely, meaning the cdf of (δ-θ) is steeper than for other estimators enjoying the same cdf at zero (for instance ½ to make them all median-unbiased). When I saw this question on X validated, I thought of the Cauchy location example, where there is no uniformly optimal estimator, albeit a large collection of unbiased ones. But a simulation experiment shows that the MLE does better than the competition. At least than three (above) four of them (since I tried the Pitman estimator via Christian Henning’s smoothmest R package). The differences to the MLE empirical cd make it clearer below (with tomato for a score correction, gold for the Pitman estimator, sienna for the 38% trimmed mean, and blue for the median):I wonder at a general theory along these lines. There is a vague similarity with Pitman nearness or closeness but without the paradoxes induced by this criterion. More in the spirit of stochastic dominance, which may be achievable for location invariant and mean unbiased estimators…

complex Cauchys

During a visit of Don Fraser and Nancy Reid to Paris-Dauphine where Nancy gave a nice introduction to confidence distributions, Don pointed out to me a 1992 paper by Peter McCullagh on the Cauchy distribution. Following my recent foray into the estimation of the Cauchy location parameter. Among several most interesting aspects of the Cauchy, Peter re-expressed the density of a Cauchy C(θ¹,θ²) as

f(x;θ¹,θ²) = |θ²| / |x-θ|²

when θ=θ¹+ιθ² [a complex number on the half-plane]. Denoting the Cauchy C(θ¹,θ²) as Cauchy C(θ), the property that the ratio aX+b/cX+d follows a Cauchy for all real numbers a,b,c,d,

C(aθ+b/cθ+d)

[when X is C(θ)] follows rather readily. But then comes the remark that

“those properties follow immediately from the definition of the Cauchy as the ratio of two correlated normals with zero mean.”

which seems to relate to the conjecture solved by Natesh Pillai and Xiao-Li Meng a few years ago. But the fact that  a ratio of two correlated centred Normals is Cauchy is actually known at least from the1930’s, as shown by Feller (1930, Biometrika) and Geary (1930, JRSS B).