that the median cannot be a sufficient statistic

When reading an entry on The Chemical Statistician that a sample median could often be a choice for a sufficient statistic, it attracted my attention as I had never thought a median could be sufficient. After thinking a wee bit more about it, and even posting a question on cross validated, but getting no immediate answer, I came to the conclusion that medians (and other quantiles) cannot be sufficient statistics for arbitrary (large enough) sample sizes (a condition that excludes the obvious cases of one & two observations where the sample median equals the sample mean).

In the case when the support of the distribution does not depend on the unknown parameter θ, we can invoke the Darmois-Pitman-Koopman theorem, namely that the density of the observations is necessarily of the exponential family form,

\exp\{ \theta T(x) - \psi(\theta) \}h(x)

to conclude that, if the natural sufficient statistic

S=\sum_{i=1}^n T(x_i)

is minimal sufficient, then the median is a function of S, which is impossible since modifying an extreme in the n>2 observations modifies S but not the median.

In the other case when the support does depend on the unknown parameter θ, we can consider the case when

f(x|\theta) = h(x) \mathbb{I}_{A_\theta}(x) \tau(\theta)

where the set indexed by θ is the support of f. In that case, the factorisation theorem implies that

\prod_{i=1}^n \mathbb{I}_{A_\theta}(x_i)

is a 0-1 function of the sample median. Adding a further observation y⁰ which does not modify the median then leads to a contradiction since it may be in or outside the support set.

Incidentally, if an aside, when looking for examples, I played with the distribution

\dfrac{1}{2}\mathfrak{U}(0,\theta)+\dfrac{1}{2}\mathfrak{U}(\theta,1)

which has θ as its theoretical median if not mean. In this example, not only the sample median is not sufficient (the only sufficient statistic is the order statistic and rightly so since the support is fixed and the distributions not in an exponential family), but the MLE is also different from the sample median. Here is an example with n=30 observations, the sienna bar being the sample median:

MLnot

4 Responses to “that the median cannot be a sufficient statistic”

  1. Armchair Guy Says:

    To prove that the median cannot be a function of S, don’t we need to demonstrate that the median can change even when S remains the same (and not vice versa)? Or is this more a “for all useful/practical situations” kind of argument?

    • Since we are in a one dimensional setting and that S is the minimal statistic, it sounds all the same to me. If I change my sample in such a way that S is turned into an arbitrary new value s and the median is unchanged, I cannot find a function f: s -> f(s) that gives the median…

  2. This is interesting in the ABC context. I would probably prefer the median over the mean as a representation of the centre of a distribution for pretty much any model I could imagine applying ABC to.

    I wonder what this does to the asymptotics. Let’s say you were trying to choose between a log-normal and a gamma model. These both exponential family distributions, so the BF computed from ABC should be consistent. But if we replaced the means in the sufficient statistics with medians, would we still be consistent. I’d hope so, because it’s a more robust measure of identical information, but maybe the robustness of the extremes wouldn’t allow proper discrimination if the distributions differ only in the tails.

    • Dan: if you use solely the mean or the median to separate between log-normal and gamma or even the pair of them, the Bayes factor will not ring twice (i.e., will be inconsistent). On the opposite, if you use a collection of quantiles, including the median, you should achieve BF consistency. Now, an interesting question is about which statistic provides the most efficient BF convergence? I suspect, based on P. Fearnhead’s and D. Prangle’s work that it may be the BF itself…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s