Avanti

2010. Time to move over to Wordpress. Yes.

But before we go, here's Cosma Shalizi on the Neyman-Pearson lemma and William James:

When last we saw the Neyman-Pearson lemma, we were looking at how to tell whether a data set x was signal or noise, assuming that we know the statistical distributions of noise (call it p) and the distribution of signals (q). There are two kinds of mistake we can make here: a false alarm, saying "signal" when x is really noise, and a miss, saying "noise" when x is really signal. What Neyman and Pearson showed is that if we fix on a false alarm rate we can live with (a probability of mistaking noise for signal; the "significance level"), there is a unique optimal test which minimizes the probability of misses --- which maximizes the power to detect signal when it is present. This is the likelihood ratio test, where we say "signal" if and only if q(x)/p(x) exceeds a certain threshold picked to control the false alarm rate.

CRS goes on to elaborate, then gets to William James and the will to believe:

Let's step back a little bit to consider the broader picture here. We have a question about what the world is like --- which of several conceivable hypotheses is true. Some hypotheses are ruled out on a priori grounds, others because they are incompatible with evidence, but that still leaves more than one admissible hypothesis, and the evidence we have does not conclusively favor any of them. Nonetheless, we must chose one hypothesis for purposes of action; at the very least we will act as though one of them is true. But we may err just as much through rejecting a truth as through accepting a falsehood. The two errors are symmetric, but they are not the same error. In this situation, we are advised to pick a hypothesis based, in part, on which error has graver consequences.

This is precisely the set-up of William James's "The Will to Believe". (It's easily accessible online, as are summaries and interpretations; for instance, an application to current controversies by Jessa Crispin.) In particular, James lays great stress on the fact that what statisticians now call Type I and Type II errors are both errors:

There are two ways of looking at our duty in the matter of opinion, — ways entirely different, and yet ways about whose difference the theory of knowledge seems hitherto to have shown very little concern. We must know the truth; and we must avoid error, — these are our first and great commandments as would-be knowers; but they are not two ways of stating an identical commandment, they are two separable laws. Although it may indeed happen that when we believe the truth A, we escape as an incidental consequence from believing the falsehood B, it hardly ever happens that by merely disbelieving B we necessarily believe A. We may in escaping B fall into believing other falsehoods, C or D, just as bad as B; or we may escape B by not believing anything at all, not even A.

Know the truth! Shun error! 2010, Excelsior!

The whole thing here.

Imbecility and the Absolute

The autumn harvest of my verbal vegetable garden seems to have been decimated by some mysterious sort of pestilence - which is to say, my ha'penny writing's worth even less than that these days - so I'll just pass along a couple of amusing anecdotes about the James brothers.

The first, an anecdote about Henry James, Ford Madox Ford (then known as Ford Madox Hueffer), another writer (name not given), and a dog named Maximilian, is related in Javier Marias's Written Lives, a series of brief sketches of famous writers:

"He [James] spoke with so many interpolations and parentheses that this occasionally got him into difficulties: one afternoon, he went out for a walk along the Rye road, as was his custom, with Hueffer and another writer and with his dog Maximilian, who liked to chase sheep and who was , for this reason, kept on a leash, but one long enough to allow him considerable freedom of movement. At one point, in order to conclude one particularly immense sentence with due emphasis, James stopped and planted his walking stick firmly in the ground, and in that position held forth for a long time while his companions listened in reverential silence, and the dog Maximilian, running about, back and forth, as the fancy took him, wound his leash around the walking stick and the gentlemen's legs, leaving them trapped. The Master finished his speech and wanted to continue on his way, but found himself immobilised. When he did, with some difficulty, extricate himself, he turned, eyes blazing, to Hueffer, reproachfully brandished his walking stick and cried: 'Heuffer! You are painfully young, but at no more than the age to which you have attained, the playing of such tricks is an imbecibility! An im . . . be . . . cility!' "

The second involves pictures. William James and Josiah Royce were being photographed by James's daughter Peggy. Here are the two pictures:




Peggy took the pictures one after the other. After the click of the first shot, James said to Royce, "Royce, you're being photographed! Look out! Damn the Absolute!"

Mithridates