that the median cannot be a sufficient statistic

When reading an entry on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic, it attracted my attention as I had never thought a median could be sufficient. After thinking a wee bit more about it, and even posting a question on cross validated, but getting no immediate answer, I came to the conclusion that medians (and other quantiles) cannot be sufficient statistics for arbitrary (large enough) sample sizes (a condition that excludes the obvious cases of one & two observations where the sample median equals the sample mean).

In the case when the support of the distribution does not depend on the unknown parameter θ, we can invoke the Darmois-Pitman-Koopman theorem, namely that the density of the observations is necessarily of the exponential family form,

$\exp\{ \theta T(x) - \psi(\theta) \}h(x)$

to conclude that, if the natural sufficient statistic

$S=\sum_{i=1}^n T(x_i)$

is minimal sufficient, then the median is a function of S, which is impossible since modifying an extreme in the n>2 observations modifies S but not the median.

In the other case when the support does depend on the unknown parameter θ, we can consider the case when

$f(x|\theta) = h(x) \mathbb{I}_{A_\theta}(x) \tau(\theta)$

where the set indexed by θ is the support of f. In that case, the factorisation theorem implies that

$\prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i)$

is a 0-1 function of the sample median. Adding a further observation y⁰ which does not modify the median then leads to a contradiction since it may be in or outside the support set.

Incidentally, if an aside, when looking for examples, I played with the distribution

$\dfrac{1}{2}\mathfrak{U}(0,\theta)+\dfrac{1}{2}\mathfrak{U}(\theta,1)$

which has θ as its theoretical median if not mean. In this example, not only the sample median is not sufficient (the only sufficient statistic is the order statistic and rightly so since the support is fixed and the distributions not in an exponential family), but the MLE is also different from the sample median. Here is an example with n=30 observations, the sienna bar being the sample median:

This entry was posted on November 14, 2014 at 12:14 am and is filed under Kids, Statistics, University life with tags blogging, cross validated, exponential families, mean vs. median, Pitman-Koopman theorem, sufficiency. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

6 Responses to “that the median cannot be a sufficient statistic”

Tom Loredo Says:
January 29, 2020 at 9:29 pm

Christian, a student just asked me a question about using medians that was related to sufficiency. I was delighted to find an answer at the Og. Thanks for posting this (and reposting it on CV).

Reply
- xi'an Says:
  January 29, 2020 at 9:56 pm
  
  Glad to hear this was of help, Tom!
  
  Reply
Armchair Guy Says:
November 21, 2014 at 11:02 pm

To prove that the median cannot be a function of S, don’t we need to demonstrate that the median can change even when S remains the same (and not vice versa)? Or is this more a “for all useful/practical situations” kind of argument?

Reply
- xi'an Says:
  November 22, 2014 at 6:50 am
  
  Since we are in a one dimensional setting and that S is the minimal statistic, it sounds all the same to me. If I change my sample in such a way that S is turned into an arbitrary new value s and the median is unchanged, I cannot find a function f: s -> f(s) that gives the median…
  
  Reply
Dan Simpson Says:
November 14, 2014 at 3:57 pm

This is interesting in the ABC context. I would probably prefer the median over the mean as a representation of the centre of a distribution for pretty much any model I could imagine applying ABC to.

I wonder what this does to the asymptotics. Let’s say you were trying to choose between a log-normal and a gamma model. These both exponential family distributions, so the BF computed from ABC should be consistent. But if we replaced the means in the sufficient statistics with medians, would we still be consistent. I’d hope so, because it’s a more robust measure of identical information, but maybe the robustness of the extremes wouldn’t allow proper discrimination if the distributions differ only in the tails.

Reply
- xi'an Says:
  November 15, 2014 at 11:16 am
  
  Dan: if you use solely the mean or the median to separate between log-normal and gamma or even the pair of them, the Bayes factor will not ring twice (i.e., will be inconsistent). On the opposite, if you use a collection of quantiles, including the median, you should achieve BF consistency. Now, an interesting question is about which statistic provides the most efficient BF convergence? I suspect, based on P. Fearnhead’s and D. Prangle’s work that it may be the BF itself…
  
  Reply