observed vs. complete in EM algorithm
While answering a question related with the EM algorithm on X validated, I realised a global (or generic) feature of the (objective) E function, namely that
![E(\theta'|\theta)=\mathbb E_{\theta}[\log\,f_{X,Z}(x^\text{obs},Z|\theta')|X=x^\text{obs}]](https://s0.wp.com/latex.php?latex=E%28%5Ctheta%27%7C%5Ctheta%29%3D%5Cmathbb+E_%7B%5Ctheta%7D%5B%5Clog%5C%2Cf_%7BX%2CZ%7D%28x%5E%5Ctext%7Bobs%7D%2CZ%7C%5Ctheta%27%29%7CX%3Dx%5E%5Ctext%7Bobs%7D%5D&bg=000000&fg=B0B0B0&s=0&c=20201002)
can always be written as
![\log\,f_X(x^\text{obs};\theta')+\mathbb E_{\theta}[\log\,f_{Z|X}(Z|x^\text{obs},\theta')|X=x^\text{obs}]](https://s0.wp.com/latex.php?latex=%5Clog%5C%2Cf_X%28x%5E%5Ctext%7Bobs%7D%3B%5Ctheta%27%29%2B%5Cmathbb+E_%7B%5Ctheta%7D%5B%5Clog%5C%2Cf_%7BZ%7CX%7D%28Z%7Cx%5E%5Ctext%7Bobs%7D%2C%5Ctheta%27%29%7CX%3Dx%5E%5Ctext%7Bobs%7D%5D&bg=000000&fg=B0B0B0&s=0&c=20201002)
therefore always includes the (log-) observed likelihood, at least in this formal representation. While the proof that EM is monotonous in the values of the observed likelihood uses this decomposition as well, in that
![\log\,f_X(x^\text{obs};\theta')=\log\,\mathbb E_{\theta}\left[\frac{f_{X,Z}(x^\text{obs},Z;\theta')}{f_{Z|X}(Z|x^\text{obs},\theta)}\big|X=x^\text{obs}\right]](https://s0.wp.com/latex.php?latex=%5Clog%5C%2Cf_X%28x%5E%5Ctext%7Bobs%7D%3B%5Ctheta%27%29%3D%5Clog%5C%2C%5Cmathbb+E_%7B%5Ctheta%7D%5Cleft%5B%5Cfrac%7Bf_%7BX%2CZ%7D%28x%5E%5Ctext%7Bobs%7D%2CZ%3B%5Ctheta%27%29%7D%7Bf_%7BZ%7CX%7D%28Z%7Cx%5E%5Ctext%7Bobs%7D%2C%5Ctheta%29%7D%5Cbig%7CX%3Dx%5E%5Ctext%7Bobs%7D%5Cright%5D&bg=000000&fg=B0B0B0&s=0&c=20201002)
I wonder if the appearance of the actual target in the temporary target E(θ’|θ) can be exploited any further.
Xi'an's Og 
Leave a Reply