On randomness

A while ago, I posted how strangely people seem to be attracted by re- and re-explaining Bayes’ theorem when I see it as a tautological consequence of the definition of conditional probability (and hence of limited interest per se, although with immense consequences for conducting inference). Through the “spam” book mentioned earlier this week, I noticed that the same (or even worse) fatal attraction holds for randomness! (Although I had already posted on the “truly random” generators…) Having access only to one chapter, I read with a sense of growing puzzlement through Tommaso Toffoli’s chapter and came with the following comments, which are nothing but a Saturday afternoon idle thoughts!

Measure theory, and much of the axiomatic apparatus that goes into what is often called the “foundations” of probability, is just about developing more refined accounting techniques for when the outcome space becomes so large (viz., uncountably infinite) that simple minded techniques lead to paradoxes: “If a line consists of points, and a point has no length, how come a line has length?”

Since I got very perplexed at Krzysztof Burdzy’s foundational book, it may be that I am missing a brick in my mind construction about probability:  however, I cannot fathom why the foundations of probability theory based on measure theory need to be attacked (over and over again). I am very happy with the concept that a random variable is a measurable function and that only measurable sets have a probability value… This gives me a mathematically valid framework for conducting probability calculus and hence Bayesian statistics. Whether or not it corresponds to a (physical) reality is of no concern to me since it is used to handle statistical models that are themselves formalisations.

Is probability then something objective (von Mises), that has to do with facts out there, and that we try to internalize, or something subjective (de Finetti), that has to do with our beliefs, which we then project onto the outside world?

The above quote shows that the link with the Search for Certainty is not that artificial: Toffoli also opposes von Mises and de Finetti to Jaynes, making maximum entropy the foundational principle. He however objects to von Mises’ experimental version of probability and proposes a redefinition of probability as logic with incomplete information, sticking with the Laplacian number of positive cases over number of possible cases. Obviously, this eliminates measure-theoretic subtleties!

In the present case, it is the invertibility of the microscopic dynamics that guarantees that the fine-grained entropy associated with an initial description remains constant as the system—and accordingly its description—evolves in time.

(This kind of Newspeak does sound like a computer-generated text and taken in isolation would certainly validate labelling the book as spam! Or pass as fashionable nonsense?) The quote actually relates to a section where Toffoli introduces an invertibility condition on the construction of a probability, by which he seems to mean stationarity (or constant entropy). However, a probability is usually defined without a reference to any time structure, unless one considers a stochastic process… (Interestingly, the paper notes later that “entropy has to do with a distribution of states, that is, with a partition of the whole set of microstates; it doesn’t need to know anything about a dynamics“, but it appears to be in criticism of entropy.)

Or instead it was truly necessary to wait for quantum mechanics in order to have a non-anthropomorphic explanation of this teleological ‘least action’ principle?”

In the end, the paper does not define at all what probability is or should be, except for this intuitive finite world combinatoric construct at the beginning. By the middle of the chapter, the discussion has drifted to a discussion of the nature of quantum mechanics, forgetting about “randomness, probability, and entropy [which] are human-made categories rather than physical quantities”… Although this is of very minor importance (hence the link to the xkcd comic), I am thus puzzled as to why this paper was included into the book.