Bhattacharyya distance versus Kullback-Leibler divergence

Another question I picked on Cross Validated during the Yule break is about the connection between the Bhattacharyya distance and the Kullback-Leibler divergence, i.e.,




Although this Bhattacharyya distance sounds close to the Hellinger distance,


the ordering I got by a simple Jensen inequality is


and I wonder how useful this ordering could be…

One Response to “Bhattacharyya distance versus Kullback-Leibler divergence”

  1. They’re all very similar. The wiki page for Bhattacharyya distance gives the precise (equality) relationship between d_B and d_H (d_B is not a metric). So I guess that the Bhattacharyya distance and the Hellinger distance define equivalent topologies.

    The ordering is that KLD is (topologically) stronger than Hellinger which is stronger than total variation. And convergence in the Hellinger metric implies that L2 funcitonals converge.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: