## Bhattacharyya distance versus Kullback-Leibler divergence

Posted in Books, Kids, Statistics with tags , , , , on January 10, 2015 by xi'an

Another question I picked on Cross Validated during the Yule break is about the connection between the Bhattacharyya distance and the Kullback-Leibler divergence, i.e.,

$d_B(p,q)=-\log\left\{\int\sqrt{p(x)q(x)}\,\text{d}x\right\}$

and

$d_{KL}(p\|q)=\int\log\left\{{q(x)}\big/{p(x)}\right\}\,p(x)\,\text{d}x$

Although this Bhattacharyya distance sounds close to the Hellinger distance,

$d_H(p,q)=\left\{1-\int\sqrt{p(x)q(x)}\,\text{d}x\right\}^{1/2}$

the ordering I got by a simple Jensen inequality is

$d_{KL}(p\|q)\ge2d_B(p,q)\ge2d_H(p,q)^2\,.$

and I wonder how useful this ordering could be…