Databases Reference
In-Depth Information
Thus, z-score normalization using the mean absolute deviation is
v
i
N
A
s
A
v
0
i
D
.
(3.11)
The mean absolute deviation,
s
A
, is more robust to outliers than the standard deviation,
A
. When computing the mean absolute deviation, the deviations from the mean (i.e.,
j
x
i
N
x
j) are not squared; hence, the effect of outliers is somewhat reduced.
Normalization by decimal scaling
normalizes by moving the decimal point of values
of attribute
A
. The number of decimal points moved depends on the maximum absolute
value of
A
. A value,
v
i
, of
A
is normalized to
v
0
i
by computing
v
i
10
j
,
v
0
i
D
(3.12)
.j
v
0
i
j/<
where
j
is the smallest integer such that
max
1.
Example 3.6
Decimal scaling.
Suppose that the recorded values of
A
range from 986 to 917. The
maximum absolute value of
A
is 986. To normalize by decimal scaling, we therefore
divide each value by 1000 (i.e.,
j
D 3) so that 986 normalizes to 0.986 and 917
normalizes to 0.917.
Note that normalization can change the original data quite a bit, especially when
using z-score normalization or decimal scaling. It is also necessary to save the normaliza-
tion parameters (e.g., the mean and standard deviation if using z-score normalization)
so that future data can be normalized in a uniform manner.
3.5.3
Discretization by Binning
Binning is a top-down splitting technique based on a specified number of bins.
Section 3.2.2 discussed binning methods for data smoothing. These methods are also
used as discretization methods for data reduction and concept hierarchy generation. For
example, attribute values can be discretized by applying equal-width or equal-frequency
binning, and then replacing each bin value by the bin mean or median, as in
smoothing
by bin means
or
smoothing by bin medians
, respectively. These techniques can be applied
recursively to the resulting partitions to generate concept hierarchies.
Binning does not use class information and is therefore an unsupervised discretiza-
tion technique. It is sensitive to the user-specified number of bins, as well as the presence
of outliers.
3.5.4
Discretization by Histogram Analysis
Like binning, histogram analysis is an unsupervised discretization technique because it
does not use class information. Histograms were introduced in Section 2.2.3. A his-
togram partitions the values of an attribute,
A
, into disjoint ranges called
buckets
or
bins
.
