Archive for exponential families

conjugate priors and sufficient statistics

Posted in Statistics with tags , , , , , on March 29, 2021 by xi'an

An X validated question rekindled my interest in the connection between sufficiency and conjugacy, by asking whether or not there was an equivalence between the existence of a (finite dimension) conjugate family of priors and the existence of a fixed (in n, the sample size) dimension sufficient statistic. Outside exponential families, meaning that the support of the sampling distribution need vary with the parameter.

While the existence of a sufficient statistic T of fixed dimension d whatever the (large enough) sample size n seems to clearly imply the existence of a (finite dimension) conjugate family of priors, or rather of a family associated with each possible dominating (prior) measure,

\mathfrak F=\{ \tilde \pi(\theta)\propto \tilde {f_n}(t_n(x_{1:n})|\theta) \pi_0(\theta)\,;\ n\in \mathbb N, x_{1:n}\in\mathfrak X^n\}

the reverse statement is a wee bit more delicate to prove, due to the varying supports of the sampling or prior distributions. Unless some conjugate prior in the assumed family has an unrestricted support, the argument seems to limit sufficiency to a particular subset of the parameter set. I think that the result remains correct in general but could not rigorously wrap up the proof

factorisation theorem on densities

Posted in Statistics with tags , , , , , , on December 23, 2020 by xi'an

Another occurrence, while building my final math stat exam for my (quarantined!) third year students, of a question on X validated that led me to write down more precisely an argument for the decomposition of densities in exponential families. Albeit the decomposition is somewhat moot (and lost on the initiator of the question since this person later posted an answer ignoring measures), as it all depends on the choice of the dominating measures over X, T(X), and the slices {x; T(x)=t}. The fact that the slice does depend on t requires the measure to accept a potential dependence on t, in which case the conditional density wrt this measure can as well be constant.

double if not exponential

Posted in Books, Kids, Statistics, University life with tags , , , , , , on December 10, 2020 by xi'an

In one of my last quizzes for the year, as the course is about to finish, I asked whether mean or median was the MLE for a double exponential sample of odd size, without checking for the derivation of the result, as I was under the impression it was a straightforward result. Despite being outside exponential families. As my students found it impossible to solve within the allocated 5 minutes, I had a look, could not find an immediate argument (!), and used instead this nice American Statistician note by Robert Norton based on the derivative being the number of observations smaller than θ minus the number of observations larger than θ.  This leads to the result as well as the useful counter-example of a range of MLE solutions when the number of observations is even.

arbitrary non-constant function [nonsensical]

Posted in Statistics with tags , , , , , , , , , , , on November 6, 2020 by xi'an
When looking for properties of the negative exponential distribution, in connection with an X validated question, I came across this nonsensical paper, starting with a truncated and drifted exponential distribution being defined as a negative exponential distribution, including a nonsensical bound a in (1.1), followed by an equally nonsensical characterisation of the distribution, including a theorem with a useless function Φ… Unsurprisingly, the publisher (SCIRP) of the Open Journal of Statistics is part of Beall’s list of potential, possible, or probable] predatory publishers. (List that is now maintained by Scholarly Open Access.)

Hélène Massam (1949-2020)

Posted in Statistics with tags , , , , , , , , , , , , , , , , , , , on November 1, 2020 by xi'an

I was much saddened to hear yesterday that our friend and fellow Bayesian Hélène Massam passed away on August 22, 2020, following a cerebrovascular accident. She was professor of Statistics at York University, in Toronto, and, as her field of excellence covered [the geometry of] exponential families, Wishart distributions and graphical models, we met many times at both Bayesian and non-Bayesian conferences  (the first time may have been an IMS in Banff, years before BIRS was created). And always had enjoyable conversations on these occasions (in French since she was born in Marseille and only moved to Canada for her graduate studies in optimisation). Beyond her fundamental contributions to exponential families, especially Wishart distributions under different constraints [including the still opened 2007 Letac-Massam conjecture], and graphical models, where she produced conjugate priors for DAGs of all sorts, she served the community in many respects, including in the initial editorial board of Bayesian Analysis. I can also personally testify of her dedication as a referee as she helped with many papers along the years. She was also a wonderful person, with a great sense of humor and a love for hiking and mountains. Her demise is a true loss for the entire community and I can only wish her to keep hiking on new planes and cones in a different dimension. [Last month, Christian Genest (McGill University) and Xin Gao (York University) wrote a moving obituary including a complete biography of Hélène for the Statistical Society of Canada.]