Simply put, evidence is any observation that changes your probability assignments for a given hypothesis. That’s pretty much it. We can further define evidence for a hypothesis as one that makes the hypothesis more likely, and evidence against a hypothesis as one that makes it less likely.
Generally, for some observation to affect your probability estimates of anything, there needs to be some cause-and-effect chain connecting the observation and the hypothesis. For instance, the hypothesis “It has rained.” and the observation “I see the street is wet.” are connected: it rains, the water makes the street wet, then photons coming from somewhere bounce off the street and hit your retinas, which send an electrical signal to your brain provoking you to make the observation.
That’s valid for anything at all. The chain can be even longer: I may call my friend and tell them that it has rained, and so their probabilistic assignment that it has rained will change, because there is a causal chain connecting the rain to them.
And that’s why other people’s information is evidence for stuff. Because there is some cause-and-effect chain between the stuff and them telling you that the stuff is true.
Information travelling between brains amongst honest folk is evidence, just as well as observing the event itself (although it’s not just as much, since the more steps there are in the chain, the lower your probability will be).
And evidence has an interesting mathematical meaning, too. Suppose you have some hypothesis , and some observation . is evidence about if (as usual, is your background knowledge). But what is ? By Bayes’ Theorem:
So for to not be equal to , we must have that which automatically means that .
Now let’s characterise mathematically evidence for and against a hypothesis.
The meaning of the above is that is evidence for if and only if is more likely to be observed when is true than otherwise. Likewise:
is evidence against if and only if is less likely to be observed when is true than otherwise.
But Probability Theory doesn’t deal with hypotheses and observations. It deals exclusively with propositions. So in fact, what and actually mean propositionally, in the equations, is:
: “Hypothesis H is true.”
: “Observation E has been made.”
Therefore, the probabilities have different meanings:
: “The probability that hypothesis H is true.”
: “The probability that observation E will be made.”
: “The probability that hypothesis H is true, given that observation E has been made.”
: “The probability that observation E will be made, given that hypothesis H is true.”
But the rest of the reasoning is perfectly sound (mathematics works by any name you give it).