Archive for E.T. Jaynes

Jaynes’ back on track!

Posted in Books, Statistics, University life with tags , , on March 30, 2011 by xi'an

Following the cancellation of my reading seminar on Jaynes’ Probability Theory, and requests from several would-be-attendees, I am giving a one-day [crash] course on the book on April 11. It will be at ENSAE, salle 11, from 9:30 till 4:00pm [or earlier if I exhaust the slides, the material or the audience], with a break at noon. Once again, it is open to everyone, but attendants must register with Nadine Guedj [at ensae.fr]. Several copies of Probability Theory are available  in the library. The slides are available as earlier as

Obviously, this is a last call!

Jaynes’ re-read

Posted in Books, Statistics, University life with tags , , , , , , , on March 21, 2011 by xi'an

On many technical issues we disagree strongly with de Finetti. It appears to us that his way of treating infinite sets has opened up a Pandora’s box of useless and unecessary paradoxes.”  E.T. Jaynes, PT, p.xxi

On Friday, despite the cancellation of the reading seminar on Jaynes’ Probability Theory, I completed my slides on Chapters 4 (Elementary hypothesis testing) to 14 (Simple applications of decision theory), plus of course Chapter 20 (Model comparison). I skipped Chapter 15 (Paradoxes of probability theory), despite its extensive and exciting coverage of the marginalisation paradoxes which saw Jaynes opposing David, Stone, and Zidek (and even the whole Establishment, page 470), as it would have taken me another morning at the very least… (Next year, maybe, if the seminar resumes?!)

Continue reading

Bertand’s paradox [R details]

Posted in Books, R, Statistics with tags , , , , , , , on March 20, 2011 by xi'an

Some may have had reservations about the “randomness” of the straws I plotted to illustrate Bertrand’s paradox. As they were all going North-West/South-East. I had actually made an inversion between cbind and rbind in the R code, which explained for this non-random orientation. Above is the corrected version, which sounds “more random” indeed. (And using wheat as the proper, if weak, colour!) The outcome of a probability of 1/2 has not changed, of course. Here is the R code as well:


lacorde=rep(0,10^3)
plot(0,0,type="n",xlim=c(-2,2),ylim=c(-2,2))

for (t in 1:10^3){

 #distance from O to chord
 dchord=10

 while (dchord>1){
 #Generate "random" straw in large box till it crosses unit circle

 a=runif(2,-10,10)
 b=runif(2,-10,10)

 #endpoints outside the circle
 if ((sum(a^2)>1)&&(sum(b^2)>1)){

 theta=abs(acos(t(b-a)%*%a/sqrt(sum((b-a)^2)*sum(a^2))))
 theta=theta%%pi
 thetb=abs(acos(t(a-b)%*%b/sqrt(sum((b-a)^2)*sum(b^2))))
 thetb=thetb%%pi

 #chord inside
 if (max(abs(theta),abs(thetb))<pi/2)
 dchord=abs(sin(theta))*sqrt(sum(a^2))
 }
 }

 lacorde[t]=2*sqrt(1-dchord)
 if (runif(1)<.1) lines(rbind(a,b),col="wheat")
 }

lecercle=cbind(sin(seq(0,2*pi,le=100)),cos(seq(0,2*pi,le=100)))
lines(lecercle,col="sienna")

As a more relevant final remark, I came to the conclusion (this morning while running) that the probability of this event can be anything between 0 and 1, rather than the three traditional 1/4, 1/3 and 1/2. Indeed, for any distribution of the “random” straws, hence for any distribution on the chord length L, a random draw can be expressed as L=F⁻¹(U), where U is uniform. Therefore, this draw is also an acceptable transform of a uniform draw, just like Bertrand’s three solutions.

Jaynes’ seminar cancelled!

Posted in Statistics with tags , , on March 17, 2011 by xi'an

The reading seminar on Jaynes’ Probability Theory that was planned to start on next Monday at CREST. is cancelled for lack of enough registered students. I am thus alas forced to cancel it, although I had now reached the most interesting sections of the book. I think I will finish reading it and writing my slides in preparation for next year when I plan to propose the reading seminar in conjunction with Massimiliano Gubinelli from Dauphine who would then provide a probabilist look at the book. The slides will thus get updated as I proceed through:

I am obviously sorry this seminar did not attract more students but neither this is a major disaster—especially when considering the dire events that took place this week in Japan—as it led me to read Jaynes’ Probability Theory carefully.

Bertrand’s paradox

Posted in Books, Statistics, Travel, University life with tags , , , on March 17, 2011 by xi'an


While progressing through Jaynes’ Probability Theory for my classes next week, I reached the end of Chapter 12 where he proposed a “resolution” of Bertrand’s paradox. Jaynes gives the following description of the paradox:

Bertrand’s problem was stated originally in terms of drawing a straight line `at random’ intersecting a circle (…) we do no violence to the problem if we suppose we are tossing straws onto the circle (…) What is the probability that the chord thus defined [by a random straw] is longer than the side of the inscribed equilateral triangle?

My understanding of the paradox is that it provides a perfect illustration of the lack of meaning of “random” and of the need for a proper definition of the σ-algebra leading to a probabilised space. Different σ-algebras lead to different probabilities, e.g., 1/4, 1/3, 1/2… However, Jaynes considers there is a “correct” answer and endeavours to construct an invariant distribution on the location of the centre of the chord, achieving Borel’s distribution

p(x) = \frac{x}{\sqrt{1-x^2}}

on the chord proportion L/2R. When illustrating this approach, I tried to bring an empirical vision and generated “random” straws by picking both ends at random on the (-10,10)^2 square. Here is a subsample of 10³ such straws interesting with the unit circle. The empirical distribution of the chord proportion is actually quite in agreement with Borel’s distribution, with a probability of being longer than the side of 1/2, as shown below but this does not validate Jaynes’ argument, simply illustrates that I picked the same σ-algebra as his. (Every σ-algebra considered by Bertrand could as well have been used for the simulation.)

Reading seminar on Jaynes’ Probability Theory

Posted in Statistics with tags , , on March 14, 2011 by xi'an

The reading seminar on Jaynes’ Probability Theory is planned to start on next Monday at CREST. (All welcome, just register with Nadine Guedj at ensae.fr) I have so far covered Chapters 4-8 and I need to speed up if I want to keep my schedule. So far, reading carefully through the book has induced neither a burst of criticisms (as it occurred with Keynes’ Treatise on Probability) since there is no strong departure from traditional Bayesian inference, nor a foundational shift in my understanding of the field. The chapter on testing is quite well-written and starting with testing rather than estimation is a brilliant idea. The only annoying part is the permanent bashing of measure theory, which leads to an informal processing of improper priors. Here are the first slides written up to now:

Jaynes’ re-read

Posted in Books, Statistics, University life with tags , , , on July 2, 2010 by xi'an

Following the two past PhD courses on JeffreysTheory of Probability and KeynesTreatise On Probability, I will propose next year a reading course at CREST on Jaynes’s Probability Theory.

Jeffreys and Jaynes share a lot in common as physicists who both significantly contributed to Bayesian statistical theory and as writers of books with almost identical titles and with very ambitious and similar scopes. It is thus no surprise that Jaynes dedicates his book to Jeffreys. There are also differences, the most obvious one being that Jeffreys published his foundational book before his 50th birthday, while Jaynes’ book came out more than ten years after his death (under the scholarly supervision of Larry Bretthorst). The time difference between both books is not that revelant, however, in that Jaynes’s Probability Theory is what Persi Diaconis call “wonderfully out of date” in his review. (Meaning “Jaynes’s focus on basic philosophical issues [rather than on computational techniques] is timeless“, with the conclusion that  “this is the best introduction to Bayesian statistics that I know”!)

I plan to cover in the lectures what I consider to be the most significant aspects of Jaynes’s work. The corpus of work corresponding to the logical foundations of probability theory and the opposition of Jaynes to (Feller’s) measure theory, Bourbakism, Kolmogorov’s axioms, (Feller’s) countable additivity, de Finetti’s principles, and other probabilistic paradoxes will not be adressed, even though a second course by a probabilist colleague of mine at Dauphine may follow this one. The lectures will focus on

  1. the meaning and motivation of prior distributions (Chapter 6), culminating in the definition of the entropy principle (Chapter 11)
  2. the rules of hypothesis testing (Chapter 4) and the central role of evidence (Chapters 9 and 18)
  3. the special case of transformation groups (Chapter 12) and the debate about marginalisation paradoxes (Chapter 15)
  4. Bayesian estimation (Chapter 6) and the criticisms on decision theory (Chapters 13 and 14)
  5. Model comparison (Chapter 20) and the pathologies of orthodox methods (Chapters 16 and 17)

The dates of the course are already set: March 21, 24, 28, 31 and April 04 [2011]…

Follow

Get every new post delivered to your Inbox.

Join 1,020 other followers