selected parameters from observations

Posted in Books, Statistics with tags , , , , , , , on December 7, 2018 by xi'an

I recently read a fairly interesting paper by Daniel Yekutieli on a Bayesian perspective for parameters selected after viewing the data, published in Series B in 2012. (Disclaimer: I was not involved in processing this paper!)

The first example is to differentiate the Normal-Normal mean posterior when θ is N(0,1) and x is N(θ,1) from the restricted posterior when θ is N(0,1) and x is N(θ,1) truncated to (0,∞). By restating the later as the repeated generation from the joint until x>0. This does not sound particularly controversial, except for the notion of selecting the parameter after viewing the data. That the posterior support may depend on the data is not that surprising..!

“The observation that selection affects Bayesian inference carries the important implication that in Bayesian analysis of large data sets, for each potential parameter, it is necessary to explicitly specify a selection rule that determines when inference  is provided for the parameter and provide inference that is based on the selection-adjusted posterior distribution of the parameter.” (p.31)

The more interesting distinction is between “fixed” and “random” parameters (Section 2.1), which separate cases where the data is from a truncated distribution (given the parameter) and cases where the joint distribution is truncated but misses the normalising constant (function of θ) for the truncated sampling distribution. The “mixed” case introduces an hyperparameter λ and the normalising constant integrates out θ and depends on λ. Which amounts to switching to another (marginal) prior on θ. This is quite interesting even though one can debate of the very notions of “random” and “mixed” “parameters”, which are those where the posterior most often changes, as true parameters. Take for instance Stephen Senn’s example (p.6) of the mean associated with the largest observation in a Normal mean sample, with distinct means. When accounting for the distribution of the largest variate, this random variable is no longer a Normal variate with a single unknown mean but it instead depends on all the means of the sample. Speaking of the largest observation mean is therefore misleading in that it is neither the mean of the largest observation, nor a parameter per se since the index [of the largest observation] is a random variable induced by the observed sample.

In conclusion, a very original article, if difficult to assess as it can be argued that selection models other than the “random” case result from an intentional modelling choice of the joint distribution.

Typo in MCSM

Posted in Books, Statistics with tags , , , , on February 20, 2010 by xi'an

I received the following email from Liaosa Xu this morning about Monte Carlo Statistical Methods:

I am reading your text to learn MCMC. However I feel confused for Example 5.14.  I don’t understand how to derive Likelihood of y by the integration of product completed data’s likelihood and conditional density of z given y. Since conditional density of z is just f(z-theta)/(1-F(a-theta)), then we actually integrate f(z-a)^2 over [a, +infinity). How to finally get the likelihood of y? I feel confused about this problem. Could you help me out. Thanks a lot for your answer.

and there is indeed a mistake in the example. The integration of the complete likelihood is

$L(\theta|\mathbf{y}) = \int L^p(\theta|\mathbf{y}) f(\mathbf{z}|\mathbf{y},\theta) \text{d}\mathbf{z}$

where $L^p (\theta|\mathbf{y})$ is the part of the likelihood only involving $y$ (this is how it should appear on page 175 of Monte Carlo Statistical Methods). In the current printing, we somehow got confused with the EM completion scheme this example illustrates and wrote

$L(\theta|\mathbf{y}) = \mathbb{E}[L(\theta|\mathbf{y,Z})]$

and

$L(\theta|\mathbf{y}) = \mathbb{E}[L(\theta|\mathbf{y,Z})]= \int L^c(\theta|\mathbf{y},\mathbf{z}) f(\mathbf{z}|\mathbf{y},\theta) \text{d}\mathbf{z}$

which is plain wrong, as pointed above by Liaosa Xu.

Ps-If you see the above LaTeX formula for the correct decomposition twice, it is because I wrote it twice in the HTML code. (If you don’t, forget about the following.) For some incomprehensible reason, including the first formula for the complete likelihood erases the two following lines, at least on my output. I tried to play with the entries in the LaTeX formula but did not find a clear culprit!