Appendix E: Probability Theory - Codes: The Guide to Secrecy from Ancient to Modern Times - page 543

the ratio of the number of ways of success at getting 7 out of 12 tails divided
by the total number of possible outcomes.
In a more general scenario, we could define probability to be a map from a
subset of the power set satisfying certain closure properties. However, to keep
the description simple, we will stick with the power set, which we call the sample
space , and the subsets of the power set are called events , as well as outcomes .
Now suppose that we have two experiments S and T with random events

S and T . Then we can put them together and speak about the joint random

events ). For instance, suppose that we have a standard deck of 52

cards and S consists of the event the value of the card , while

U =( S , T

T consists of the

event the suit of the card . Then ) represents all the possibilities of the

52 outcomes of choosing a card. If p s,t represents the probability that a card is
drawn with value s and suit t , then given a fair deck with p s =1 / 13, p t =1 / 4,
and p s,t =1 / 52.
In general, we let p s,t denote the probability that s

U =( S , T

∈ S and t ∈ T both

occur. It follows that

p s =
t ∈ T

p s,t ,

so the probability of a fixed s ∈ S occurring is the sum of all the probabilities

of t ∈ T occurring along with s ∈ S occurring.

Independence

The two random events s ∈ S and t ∈ T , are called independent if

p s,t = p s · p t . (E.1)

For instance, in the deck of cards illustration above, the suit and value of
the cards are independent events.

Conditional Probability

If we know that event s ∈ S has occurred, then the probability that t ∈ T will

occur given that p ( s ) > 0, is defined as follows:

p t | s = p s,t

p s ,

(E.2)

called the conditional probability of t given s .

Notice that if we combine (E.1)-(E.2), we get that

s and t are independent events if and only if p t | s = p t .

In other words, the probability that t occurs is unaffected by the probability
that s occurs.
In what follows, we assume that s and s

are event subsets of S .