## Bhattacharyya distance versus Kullback-Leibler divergence

**A**nother question I picked on Cross Validated during the Yule break is about the connection between the Bhattacharyya distance and the Kullback-Leibler divergence, i.e.,

and

Although this Bhattacharyya distance sounds close to the Hellinger distance,

the ordering I got by a simple Jensen inequality is

and I wonder how useful this ordering could be…

January 10, 2015 at 12:40 am

They’re all very similar. The wiki page for Bhattacharyya distance gives the precise (equality) relationship between d_B and d_H (d_B is not a metric). So I guess that the Bhattacharyya distance and the Hellinger distance define equivalent topologies.

The ordering is that KLD is (topologically) stronger than Hellinger which is stronger than total variation. And convergence in the Hellinger metric implies that L2 funcitonals converge.