Data Utilization in Flood Inundation Modelling - Flood Risk Science and Management - page 219

Table 11.3 Recommendations for various model performance measures (after Hunter 2005)

Name Remarks Equation Recommendation

A þ B
A

Bias Predictions that count

(A,B,C 1 )

Recommended for summarizing aggregate model
performance þ C (i.e. under- or overprediction)

A þ B

PC Heavily in uenced by

the most common
category and hence,
implicitly domain size

Not recommended for either deterministic or uncertain
calibration
The values for D are usually orders of magnitude larger than the
other categories and may also be trivially easy to predict.
Therefore, in many instances, PC will provide an overly
optimistic assessment of model performance

A þ B þ C þ D

A

ROC
analysis
(F,H)

Arti cial minimizing
and maximizing of F
and H respectively

H ¼ Summarizes two different types of model error (i.e. under- or

overprediction) that can occur and is potentially a useful tool
for exploring their relative consequences and weighting in any
subsequent risk analyses. Therefore, worthy of further
consideration/development

A þ C

B

F ¼

B þ D

ð AxD Þð CxB Þ

PSS Underprediction

relative magnitudes
of F and H

Not recommended for either deterministic or uncertain
calibration. Small F and large H are typical in this type of
application and, as such, the measure fails to adequately
penalize overprediction. Signi cant overestimates of the
ooded area are therefore only graded slightly poorer than
optimal simulations. This also results in the preferential
rejection of underpredicting parameter sets during uncertain
calibration

ð B þ D Þ x ð A þ C Þ

A
A þ B þ C

F h1i

Correct prediction of Recommended for both deterministic and uncertain

calibration ooding . A relatively unbiased measure that simply and

equitably discriminates between under- and overprediction. As
such, optimal simulations will provide the best compromise
between these two undesirable attributes

A B

F h2i Overprediction Recommended for deterministic calibration (if

underprediction is preferable)

A þ B þ C

. Explicitly penalizes
overprediction but suffers as a result during uncertain
calibration. Overpredicting simulations are wrongly retained to
offset the bias introduced by the measure and provide an
acceptable compromise between inundation map accuracy
and precision. The bene ts of rejection are reduced accordingly

A C

F h3i Underprediction Recommended for deterministic calibration (if overprediction

is preferable). Here the measure was not tested within the
uncertain calibration methodology though for other reaches,
events and study objectives, F h3i may provide a useful
alternative to F h1i and F h2i . It is not sensitive to domain size and
appears to favour overprediction similar to PSS

þ þ

A B C

1 A ¼ area correctly predicted as ooded by model.
B ¼ area predicted as ooded that is actually non- ooded (over-prediction).
C ¼ area predicted as non- ooded that is actually ooded (under-prediction).
D ¼ area correctly predicted as non- ooded.