Archive for E.T. Jaynes

Frequency vs. probability

Posted in Statistics with tags , , , , , , , on May 6, 2011 by xi'an

Probabilities obtained by maximum entropy cannot be relevant to physical predictions because they have nothing to do with frequencies.” E.T. Jaynes, PT, p.366

A frequency is a factual property of the real world that we measure or estimate. The phrase `estimating a probability’ is just as much an incongruity as `assigning a frequency’. The fundamental, inescapable distinction between probability and frequency lies in this relativity principle: probabilities change when we change our state of knowledge, frequencies do not.” E.T. Jaynes, PT, p.292

A few days ago, I got the following email exchange with Jelle Wybe de Jong from The Netherlands:

Q. I have a question regarding your slides of your presentation of Jaynes’ Probability Theory. You used the [above second] quote: Do you agree with this statement? It seems to me that a lot of  ‘Bayesians’ still refer to ‘estimating’ probabilities. Does it make sense for example for a bank to estimate a probability of default for their loan portfolio? Or does it only make sense to estimate a default frequency and summarize the uncertainty (state of knowledge) through the posterior? Continue reading

Jaynes’ back on track!

Posted in Books, Statistics, University life with tags , , on March 30, 2011 by xi'an

Following the cancellation of my reading seminar on Jaynes’ Probability Theory, and requests from several would-be-attendees, I am giving a one-day [crash] course on the book on April 11. It will be at ENSAE, salle 11, from 9:30 till 4:00pm [or earlier if I exhaust the slides, the material or the audience], with a break at noon. Once again, it is open to everyone, but attendants must register with Nadine Guedj [at]. Several copies of Probability Theory are available  in the library. The slides are available as earlier as

Obviously, this is a last call!

Jaynes’ re-read

Posted in Books, Statistics, University life with tags , , , , , , , on March 21, 2011 by xi'an

On many technical issues we disagree strongly with de Finetti. It appears to us that his way of treating infinite sets has opened up a Pandora’s box of useless and unecessary paradoxes.”  E.T. Jaynes, PT, p.xxi

On Friday, despite the cancellation of the reading seminar on Jaynes’ Probability Theory, I completed my slides on Chapters 4 (Elementary hypothesis testing) to 14 (Simple applications of decision theory), plus of course Chapter 20 (Model comparison). I skipped Chapter 15 (Paradoxes of probability theory), despite its extensive and exciting coverage of the marginalisation paradoxes which saw Jaynes opposing David, Stone, and Zidek (and even the whole Establishment, page 470), as it would have taken me another morning at the very least… (Next year, maybe, if the seminar resumes?!)

Continue reading

Bertand’s paradox [R details]

Posted in Books, R, Statistics with tags , , , , , , , on March 20, 2011 by xi'an

Some may have had reservations about the “randomness” of the straws I plotted to illustrate Bertrand’s paradox. As they were all going North-West/South-East. I had actually made an inversion between cbind and rbind in the R code, which explained for this non-random orientation. Above is the corrected version, which sounds “more random” indeed. (And using wheat as the proper, if weak, colour!) The outcome of a probability of 1/2 has not changed, of course. Here is the R code as well:


for (t in 1:10^3){

 #distance from O to chord

 while (dchord>1){
 #Generate "random" straw in large box till it crosses unit circle


 #endpoints outside the circle
 if ((sum(a^2)>1)&&(sum(b^2)>1)){


 #chord inside
 if (max(abs(theta),abs(thetb))<pi/2)

 if (runif(1)<.1) lines(rbind(a,b),col="wheat")


As a more relevant final remark, I came to the conclusion (this morning while running) that the probability of this event can be anything between 0 and 1, rather than the three traditional 1/4, 1/3 and 1/2. Indeed, for any distribution of the “random” straws, hence for any distribution on the chord length L, a random draw can be expressed as L=F⁻¹(U), where U is uniform. Therefore, this draw is also an acceptable transform of a uniform draw, just like Bertrand’s three solutions.

Jaynes’ seminar cancelled!

Posted in Statistics with tags , , on March 17, 2011 by xi'an

The reading seminar on Jaynes’ Probability Theory that was planned to start on next Monday at CREST. is cancelled for lack of enough registered students. I am thus alas forced to cancel it, although I had now reached the most interesting sections of the book. I think I will finish reading it and writing my slides in preparation for next year when I plan to propose the reading seminar in conjunction with Massimiliano Gubinelli from Dauphine who would then provide a probabilist look at the book. The slides will thus get updated as I proceed through:

I am obviously sorry this seminar did not attract more students but neither this is a major disaster—especially when considering the dire events that took place this week in Japan—as it led me to read Jaynes’ Probability Theory carefully.

Bertrand’s paradox

Posted in Books, Statistics, Travel, University life with tags , , , on March 17, 2011 by xi'an

While progressing through Jaynes’ Probability Theory for my classes next week, I reached the end of Chapter 12 where he proposed a “resolution” of Bertrand’s paradox. Jaynes gives the following description of the paradox:

Bertrand’s problem was stated originally in terms of drawing a straight line `at random’ intersecting a circle (…) we do no violence to the problem if we suppose we are tossing straws onto the circle (…) What is the probability that the chord thus defined [by a random straw] is longer than the side of the inscribed equilateral triangle?

My understanding of the paradox is that it provides a perfect illustration of the lack of meaning of “random” and of the need for a proper definition of the σ-algebra leading to a probabilised space. Different σ-algebras lead to different probabilities, e.g., 1/4, 1/3, 1/2… However, Jaynes considers there is a “correct” answer and endeavours to construct an invariant distribution on the location of the centre of the chord, achieving Borel’s distribution

p(x) = \frac{x}{\sqrt{1-x^2}}

on the chord proportion L/2R. When illustrating this approach, I tried to bring an empirical vision and generated “random” straws by picking both ends at random on the (-10,10)^2 square. Here is a subsample of 10³ such straws interesting with the unit circle. The empirical distribution of the chord proportion is actually quite in agreement with Borel’s distribution, with a probability of being longer than the side of 1/2, as shown below but this does not validate Jaynes’ argument, simply illustrates that I picked the same σ-algebra as his. (Every σ-algebra considered by Bertrand could as well have been used for the simulation.)

Reading seminar on Jaynes’ Probability Theory

Posted in Statistics with tags , , on March 14, 2011 by xi'an

The reading seminar on Jaynes’ Probability Theory is planned to start on next Monday at CREST. (All welcome, just register with Nadine Guedj at I have so far covered Chapters 4-8 and I need to speed up if I want to keep my schedule. So far, reading carefully through the book has induced neither a burst of criticisms (as it occurred with Keynes’ Treatise on Probability) since there is no strong departure from traditional Bayesian inference, nor a foundational shift in my understanding of the field. The chapter on testing is quite well-written and starting with testing rather than estimation is a brilliant idea. The only annoying part is the permanent bashing of measure theory, which leads to an informal processing of improper priors. Here are the first slides written up to now: