A curiosity

Suppose that L is an unknown natural number (1,2,3,...) and as far as we know, all positive numbers are equally likely candidate for L. The number L is input into a machine, and pressing a button generates a natural number between 1 and L, which number is shown.

Consider the probabilistic distribution for L. Initially, there is no well-defined distribution. The information we have is that L is any number between 1 (inclusive) and infinity (exclusive), but there is no meaningful uniform measure on that set.

You press the button once and get a number x₁. You press it again and get x₂.

After pressing the button once, you know that L≥x₁. But you still don't have a well-defined posterior distribution of values of L. (You can try to generate one by supposing L is chosen at random between 1 and M and then taking the limit as M goes to infinity. This won't work. You get the unwelcome conclusion that for all y we have P(L>y)=1.)

But once you get the second data point, you do have a well-defined posterior distribution of values of L. Approximately (for large x, x₁ and x₂) the posterior probability that L≥x will be max(x₁,x₂)/x, assuming that max(x₁,x₂)≤x (of course, if x is less than one of the x_i, the probability that L≥x is 1). Thus, with probability approximately 1/2, we can say that L is no more than twice as large as the larger of x₁ and x₂.

This is curious. You don't have a well-defined posterior distribution with one data point, but with two you get one. And then as you gather more and more data points, you get standard Bayesian convergence. With enough data points, you can be fairly confident that L is pretty close to the largest of your data points.

I suppose this is yet another one of those phenomena where the unconditional probabilities are undefined, but the conditional ones are defined.

Another fun probability case

There is a physical constant T. You know nothing about it except that it is a positive real number. However, you can do an experiment in the lab. This experiment generates a number t which is uniformly distributed between 0 and T, and the number t is stochastically independent each time the experiment is run.

Suppose the experiment is run once, and you find that t=0.7. How should you estimate T? More exactly, what subjective probability distribution should you assign to T? This is difficult to solve by standard Bayesian methods because obviously either your priors on T should be a uniform distribution on the positive reals, or your priors on the logarithm of T should be a uniform distribution on all the reals. (I actually think the second is more natural, but I did the calculations below only for the first case. Sorry.) The problem is that there is no uniform probability measure on the positive reals or on the reals. (Well, we can have finitely additive measures, but those measures will be non-unique, and anyway won't help with this problem.)

So perhaps the conclusion we should draw from this is that you don't learn anything about T when you find out that t=0.7 other than the deductive fact that T is at least 0.7. But this is not quite right. For suppose you keep on repeating the experiment. If you draw a point at each measured value of t, you will eventually get a dotted line between 0 on the left and some number on the right, and the right hand bound of that interval will be a pretty good estimate for the value of T, and not just a lower bound for T. But if the first piece of data only gives a lower bound, then, by similar reasoning, further pieces of data either will be irrelevant (if they give a t that's less than 0.7) or will only give better (i.e., higher) lower bounds for T, and we'll never get an estimate for T, just a lower bound.

So, the first piece of data should give something. (The reasoning here is inspired by a sentence I overheard someone—I can't remember who—say to John Norton, perhaps in the case of Doomsday.)

Now here is something a little bit fun. We might try to calculate the distribution for T after n experiments in the following way. First assume that T is uniformly distributed between 0 and L (where L is large enough that all the t measurements fit between 0 and L), then calculate a conditional distribution for T given the t measurements, and finally take the limit as L tends to plus infinity. Interestingly, this procedure fails if n=1, i.e., if we have only one measurement of a t value—the resulting limiting distribution is zero everywhere. However, if n>1, then the procedure converges to a well-defined distribution of T. Or so my very rough sketches show.

So there is a radical difference here between what we get with one measurement—no distribution—and what we get with two or more measurements—a well-defined distribution. I have doubts whether standard Bayesian confirmation can make sense of this.