**A** revealing question on X validated the past week was asking for a random generator that is “more random” than the outcome of a specific random generator, à la Oliver Twist:The question is revealing of a quite common misunderstanding of the nature of random variables (as deterministic measurable transforms of a fundamental alea) and of their maybe paradoxical ability to enjoy stability or predictable properties. And much less that it relates to the long-lasting debate about the very [elusive] nature of randomness. The title of the question is equally striking: “Random numbers without pre-specified distribution” which could be given some modicum of meaning in a non-parametric setting, still depending on the choices made at the different levels of the model…

## Archive for cross validated

## more random than random!

Posted in Books, Kids, pictures, Statistics with tags Charles Dickens, cross validated, Oliver Twist, random number generator, random variates, randomness, xkcd on December 8, 2017 by xi'an## importance demarginalising

Posted in Books, Kids, pictures, Running, Statistics, Travel, University life with tags cross validated, importance sampling, Jacobian, Monte Carlo Statistical Methods, University of Warwick, Warwickshire on November 27, 2017 by xi'an**A** question on X validated gave me minor thought fodder for my crisp pre-dawn run in Warwick the other week: if one wants to use importance sampling for a variable Y that has no closed form density, but can be expressed as the transform (marginal) of a vector of variables with closed form densities, then, for Monte Carlo approximations, the problem can be reformulated as the computation of an integral of a transform of the vector itself and the importance ratio is given by the ratio of the true density of the vector over the density of the simulated vector. No Jacobian involved.

## Darmois, Koopman, and Pitman

Posted in Books, Statistics with tags cross validated, Don Fraser, exponential families, George Darmois, mathematical statistics, Pitman-Koopman theorem, proof, Stanford University, sufficient statistics on November 15, 2017 by xi'an**W**hen [X’ed] seeking a simple proof of the Pitman-Koopman-Darmois lemma [that exponential families are the only types of distributions with constant support allowing for a fixed dimension sufficient statistic], I came across a 1962 Stanford technical report by Don Fraser containing a short proof of the result. Proof that I do not fully understand as it relies on the notion that the likelihood function itself is a minimal sufficient statistic.

## normal variates in Metropolis step

Posted in Books, Kids, R, Statistics, University life with tags cross validated, Gaussian random walk, Markov chain Monte Carlo algorithm, MCMC, Metropolis-Hastings algorithm, Monte Carlo Statistical Methods, normal distribution, normal generator, random variates on November 14, 2017 by xi'an**A** definitely puzzled participant on X validated, confusing the Normal variate or variable used in the random walk Metropolis-Hastings step with its Normal density… It took some cumulated efforts to point out the distinction. Especially as the originator of the question had a rather strong a priori about his or her background:

“I take issue with your assumption that advice on the Metropolis Algorithm is useless to me because of my ignorance of variates. I am currently taking an experimental course on Bayesian data inference and I’m enjoying it very much, i believe i have a relatively good understanding of the algorithm, but i was unclear about this specific.”

despite pondering the meaning of the call to rnorm(1)… I will keep this question in store to use in class when I teach Metropolis-Hastings in a couple of weeks.

## golden Bayesian!

Posted in Statistics with tags badge, Bayesian, cross validated, introductory textbooks, MCMC, Monte Carlo Statistical Methods, simulation, Stack Exchange, Statistics Forum, wikipedia on November 11, 2017 by xi'an## probabilities larger than one…

Posted in Statistics with tags conditional density, cross validated, Gibbs sampling, Ising model, normalisation, normalising constant, pmf on November 9, 2017 by xi'an## fiducial inference

Posted in Books, Mountains, pictures, Running, Statistics, Travel with tags cross validated, fiducial distribution, fiducial inference, fjord, σ-algebra, measure theory, Norway, probability theory, Ronald Fisher, Trondheim, Trondheimsfjord on October 30, 2017 by xi'an**I**n connection with my recent tale of the many ε’s, I received from Gunnar Taraldsen [from Tronheim, Norge] a paper [jointly written with Bo Lindqvist and just appeared on-line in JSPI] on conditional fiducial models.

“The role of the prior and the statistical model in Bayesian analysis is replaced by the use of the fiducial model x=R(θ,ε) in fiducial inference. The fiducial is obtained in this case without a prior distribution for the parameter.”

Reading this paper after addressing the X validated question made me understood better the fundamental wrongness of fiducial analysis! If I may herein object to Fisher himself… Indeed, when writing x=R(θ,ε), as the representation of the [observed] random variable x as a deterministic transform of a parameter θ and of an [unobserved] random factor ε, the two random variables x and ε are based on the same random preimage ω, i.e., x=x(ω) and ε=ε(ω). Observing x hence sets a massive constraint on the preimage ω and on the conditional distribution of ε=ε(ω). When the fiducial inference incorporates another level of randomness via an independent random variable ε’ and inverts x=R(θ,ε’) into θ=θ(x,ε’), assuming there is only one solution to the inversion, it modifies the nature of the underlying σ-algebra into something that is incompatible with the original model. Because of this sudden duplication of the random variates. While the inversion of this equation x=R(θ,ε’) gives an idea of the possible values of θ when ε varies according to its [prior] distribution, it does not account for the connection between x and ε. And does not turn the original parameter into a random variable with an implicit prior distribution.

As to conditional fiducial distributions, they are defined by inversion of x=R(θ,ε), under a certain constraint on θ, like C(θ)=0, which immediately raises a Pavlovian reaction in me, namely that since the curve C(θ)=0 has measure zero under the original fiducial distribution, how can this conditional solution be uniquely or at all defined. Or to avoid the Borel paradox mentioned in the paper. If I get the meaning of the authors in this section, the resulting fiducial distribution will actually depend on the choice of σ-algebra governing the projection.

“A further advantage of the fiducial approach in the case of a simple fiducial model is that independent samples areproduced directly from independent sampling from [the fiducial distribution]. Bayesian simulations most often come as dependent samples from a Markov chain.”

This side argument in “favour” of the fiducial approach is most curious as it brings into the picture computational aspects that do not have any reason to be there. (The core of the paper is concerned with the unicity of the fiducial distribution in some univariate settings. Not with computational issues.)