Sequence classic: "I don’t think you could get up to 99.99% confidence for assertions like “53 is a prime number.”

Suppose you say that you’re 99.99% confident that 2 + 2 = 4.

Then you're a dillbrain.

Then you have just asserted that you could make 10,000 independent statements, in which you repose equal confidence, and be wrong, on average, around once. Maybe for 2 + 2 = 4 this extraordinary degree of confidence would be possible

Yes, how extraordinary that I can say every day that the guy in front of me at the bodega won't win the Powerball. Or that [SystemRandom().random() >= 0.9999 for i in range(10000)] makes a list that is False in all but one spot.

P(x|y) is defined as P(x,y)/P(y). P(A|A) is defined as P(A,A)/P(A) = P(A)/P(A) = 1. The ratio of these two probabilities may be 1, but I deny that there's any actual probability that's equal to 1. P(|) is a mere notational convenience, nothing more.

No, you kneebiter.

"The Map is Not the Territory" (I shall henceforth refer to this as "Teenage Mutant Ninja Turtles") is especially rich given how Yudkowski and other lesswrong types tend to be massive metaphor fetishists.

Suppose I said, "I have this clock that I really like. It's a very nice clock. So, I am going to measure everything I can in terms of the times registered on this clock."

"OK," you might say, while wondering what the big deal is.

"In fact, I am going to measure all speeds as the time it takes to travel a standard unit of distance."

"Uh, hold on."

"And this means that, contrary to what you learned in Big University, zero is not a speed! Because the right way to think of speed is the time it takes to travel 1 standard distance unit, and an object that never moves never travels."

Now, you might try to argue with me. You could try to point out all the things that my screwy definition would break. (For starters, I am throwing out everything science has learned about inertia.) You could try showing examples where scientists I have praised, like Feynman or whoever, speak of "a speed equal to zero". When all that goes nowhere and I dig in further with every reply, you might justifiably conclude that I am high on my own supply, in love with my own status as an iconoclast. Because that is my real motivation, neither equations nor expertise will sway me.

Yud argues that 0 and 1 are not probabilities in exactly this way. He says that you can't turn a probability of 0 or 1 into an odds ratio, because you'd be dividing by 0. This and everything that followed is just getting high off his own supply. One could try showing how he presumes his own conclusion. One could try showing how he breaks the basic idea that probabilities by their nature add up to 100% (given an event E, what can Yud say is the probability of the event E-or-not-E?). One could even observe that the same E. T. Jaynes he praises in that blog post uses 1 as a probability, for example in Chapter 2 of Probability Theory: The Logic of Science (Cambridge University Press, 2003). If you really want to cite someone he admires, you could note that Eliezer Yudkowsky uses 1 as a probability when trying (and failing) to explain quantum mechanics, because he writes probability amplitudes of absolute value 1.

As an academic, I have to hold myself back from developing all those themes and more.

Btw…

I am not going to discuss the actual experiments that have been done on calibration—you can find them in my book chapter on cognitive biases and global catastrophic risk1—because I’ve seen that when I blurt this out to people without proper preparation, they thereafter use it as a Fully General Counterargument, which somehow leaps to mind whenever they have to discount the confidence of someone whose opinion they dislike, and fails to be available when they consider their own opinions.

lol

Mr Yudkowsky is supposedly able to understand advanced maths but doesn't know what rounding is. I think we did rounding in 3rd or 4th grade....

No but what if it's actually divisible by Theta Prime?