## Don’t be so sure…

One of the great insights of Bayes’ Theorem is the gradation of belief. This is in fact not how most people intuitively reason! Most people have this intuitive feeling of black-and-white, zero-or-one, believe-or-don’t-believe. When they’re thinking about something they want to believe, they think “Does the available evidence allow me to believe it?” and when they’re thinking about something they don’t, they think “Does the available evidence force me to believe it?”

But if you start a more consistent form of reasoning, one that follows Bayes’ rules, then that “binariness” disappears immediately. And one of the simplest and most direct consequences of that is that there is nothing you can reason about that will ever get to be a certain thing. In other words, there are (almost, I’ll get to it in a bit) no propositions $A$ such that $P(A)=1$ or $P(A)=0$.

To see that, let’s take a look at the product rule. For any two propositions $A$ and $B$ while reasoning on background information $X$:

$P(A|BX)P(B|X) = P(AB|X) = P(B|AX)P(A|X)$

Let’s show that taking any proposition to absolute certainty leads us to nonsensical results.

Suppose we want to reason about the propositions A = “The sky is blue.” and B = “I see the sky being green.”

If I’m absolutely sure that the sky is blue, that means $P(A|X) = 1$, right? Furthermore, there are a few other conclusions. Reasoning naïvely, if the sky is blue, I couldn’t possibly be seeing it be green; conversely, if I see it as being green, it couldn’t possibly be blue. Therefore, $P(A|BX) = P(B|AX) = 0$, and the product rule just says that $0 = 0$, and I learn nothing. Everything is undefined.

Let’s reason in a more mature way. It’s not, in fact, true that it’s impossible for me to see a green sky under the hypothesis that it’s blue, nor that it be blue under the hypothesis that I’m seeing it be green. Maybe I’m wearing yellow-coloured glasses. Maybe I have a rare genetic condition. So let’s say that both $P(A|BX)$ and $P(B|AX)$ are nonzero but very small. In that case, we can use Bayes’ Rule:

$P(A|BX) = \frac{P(B|AX)P(A|X)}{P(B|X)}$

However, we can reexpress the denominator:

$P(A|BX) = \frac{P(B|AX)P(A|X)}{P(B|AX)P(A|X)+P(B|\bar AX)P(\bar A|X)}$

If I’m positively certain that the sky is blue, then the above is just:

$P(A|BX) = \frac{P(B|AX)*1}{P(B|AX)*1+P(B|\bar AX)*0} = \frac{P(B|AX)}{P(B|AX)} = 1$

What does that mean? It means that, if you’re absolutely certain of a proposition, then nothing you could possibly observe would ever change your mind about it. You’d just find new explanations – maybe you developped a rare eye disease while you were asleep, maybe someone is pranking you, maybe you’re hallucinating -, but you’d never ever conclude that you were wrong about the sky being blue. 0 and 1 are not probabilities, in the same way that $+\infty$ and $-\infty$ are not real numbers (in fact, that analogy is actually exact).

So saying that something you believe has probability $1$ (or $0$) is equivalent to saying that nothing will ever change your mind. Now, I don’t know about you, but I personally would like there to be some amount of evidence that was enough to change my beliefs.

There are exceptions to that, however. The way Probability Theory as logic is defined, we have that $P(\top) = 1$ and $P(\perp) = 0$. That is to say, the probability of a proposition that’s true has to be $1$, and the probability of a proposition that’s false has to be $0$. And there is a certain class of propositions that are “absolutely” true or false: tautologies and contradictions.

Contradictions are saying something you know to be false. From propositional logic, we have that $A \bar A = \perp$. That is, a proposition that says a thing and its negation is always false. Thus, they must have the same probability: $P(A \bar A) = P(\perp) = 0$. But the product rule still applies, and so $P(A\bar A) = P(A|\bar A)P(\bar A) = 0$. Since that has to be true for every possible proposition $A$, it follows that $P(A|\bar A) = 0$: the probability of a contradiction is always $0$.

Conversely, a tautology is just reaffirming something you already know to be true. We know that $A A = A$. That is, saying the same thing twice doesn’t change anything. From that, $P(A A) = P(A)$. But the product rule applies, from which $P(A A) = P(A|A)P(A) = P(A)$. And this also has to be true regardless of what proposition $A$ may be, which means $P(A|A) = 1$: the probability of a tautology is always $1$.

So, in general, if the propositions you’re reasoning about include a contradiction, they’ll be $0$, whereas tautologies will be struck out and add nothing new.

But of course, in real life you don’t really ever reason about “pure” propositions. Your background knowledge will never include “The sky is blue.” or “Peano Arithmetic is sound.” It will only include things like “I believe the sky is blue.” or “It seems to be the case that Peano Arithmetic is sound.” or, in general, for any proposition $A$, “I think $A$ is true (or false).” You’re always reasoning from inside your head. So even if you do observe something that looks like it ought to be logically forbidden by your background knowledge, that just means your background knowledge is, in fact, wrong.

In real life, $0$ and $1$ are not probabilities. Ever. That’s all.