Information Technology Reference
In-Depth Information
Chapter 21
Mathematical Background
In this chapter, we gather some key mathematical concepts for better understanding
this topic. This is not intended to be an introductory tutorial, and it is assumed that
the reader already has some background on probability theory, linear algebra, and
optimization.
When writing this chapter, we have referred to [
1
-
4
] to a large extent. Note that
we will not add explicit citations in the remaining part of this chapter. The readers
are highly encouraged to read the aforementioned material.
21.1 Probability Theory
Probability theory plays a key role in machine learning and information retrieval,
since the design of learning methods and ranking models often relies on the proba-
bility assumption on the data.
21.1.1 Probability Space and Random Variables
When we talk about probability, we often refer to the probability of an uncertain
event. Therefore, in order to discuss probability theory formally, we must first clarify
what the possible events are to which we would like to attach a probability.
Formally, a probability space is defined by the triple
(Ω,
F
,P)
, where
Ω
is the
2
Ω
is the space of (measurable) events; and
P
is
the probability measure (or probability distribution) that maps an event
E
space of possible outcomes,
F
⊆
∈
F
to a
real value between 0 and 1.
Given the outcome space
Ω
, there are some restrictions on the event space
F
:
•
∅
F
The trivial event
Ω
and the empty event
are all in
;
•
F
is closed under (countable) union, i.e., if
E
1
∈
F
and
E
2
∈
F
The event space
,
then
E
1
∪
E
2
∈
F
;
