Geoscience Reference
In-Depth Information
Table 10.4 Definitions of common model validation metrics. o i and m i are respectively observed
and modelled values at time and location i , n is the number of data pairs and ( ) denotes the mean
value
Ideal
score
Validation metrics
Formula
Range
X
n
1
n
Mean bias error
(MBE)
MBE
D
.m i
o i /
1
to
C1
0
iD1
t
X
n
1
n
. m i o i / 2
Root-mean-square
error (RMSE)
RMSE D
0toC1
0
i1
X
n
.m i
m/.o i
o/
i
D
1
Correlation
coefficient ( r )
r
D
t
t
1to1
1
X
n
X
N
m/ 2
o/ 2
.m i
.o i
i
D
1
i
D
1
ˇ ˇ ˇ ˇ
ˇ ˇ ˇ ˇ
X
n
2
n
m i
o i
Fractional gross error
(FGE)
FGE
D
0-2
0
m i
C
o i
iD1
m i o i
o i
X
n
1
n
Normalised mean
bias error
(NMBE)
NMBE
D
1to
for
non-negative
variables
C1
0
iD1
t
m i
2
X
n
1
n
o i
Normalised
root-mean-square
error (NRMSE)
NRMSE
D
0to
C1
0
o i
i
D
1
10.5.3
Metrics
The evaluation typically starts with an analysis of the plots of the forecast values
against observations for a particular location. This method, implemented for near
real-time monitoring, is very valuable in detecting outliers and identifying jumps in
performance. Then, the core of the evaluation process is the computation of metrics
defined to provide a quantitative characterisation of the agreement between model
results and observations over specific geographic regions and time periods. The most
common metrics used to quantify the departure between modelled and observed
quantities are described in Table 10.4 .
-
The BE captures the average deviations between two data sets with negative
values indicating underestimation and positive overestimation of the model.
-
The RMSE combines both the bias and the standard deviation. It is strongly
dominated by the largest values due to squaring. Especially in cases where
prominent outliers occur, the usefulness of RMSE is questionable, and the
interpretation becomes difficult.
-
r indicates the extent to which temporal and spatial patterns in the model match
those in the observations.
 
Search WWH ::




Custom Search