Operational Dust Prediction - Mineral Dust: A Key Player in the Earth System - page 246

- The FGE is a measure of the overall model error. It behaves symmetrically with

respect to under- and overestimation, without overemphasising outliers.

- The NMBE and NRMSE are dimensionless versions of MBE and RMSE, built

to facilitate comparison between the behaviour of different variables.

10.5.4 Examples of Near Real-Time Evaluation

The evaluations of dust forecasts are mainly conducted by the weather centres
generating the forecasts or institutions working in collaboration with them. Eval-
uating the model forecasts against satellite- and ground-based observations is used
to detect problems early on and also to provide a first indication of the accuracy of
the products to the users. The evaluation systems developed in the framework of
the WMO SDS-WAS NAMEE Regional Centre and MACC-II project are presented
here.

The WMO SDS-WAS Dust Model Evaluation Initiative

For the evaluation of the WMO SDS-WAS multi-model ensemble, the dust optical
depth (DOD) forecast by the models is first drawn together with the AERONET
AOD observations in monthly charts for selected dust-prone stations. Then, different
evaluation metrics (see Table 10.4 ) are computed in order to quantify the agreement
between predictions and observations for individual stations and different regions
(Sahara-Sahel, Middle East and Mediterranean) as well as different temporal
scales (monthly, seasonal and annual basis). Comparison statistics are restricted
to observations with low Ångström exponent (AE) values (<0.6) to ensure that
forecast and observations are only compared during episodes where dust is the
largest contributor. However, there will always be a small portion of particles from
other sources, so a small negative bias can be expected. Figure 10.4 shows an
example of routine verification of the multi-model ensemble products over the
AERONET station of Santa Cruz, Tenerife. In this specific example, all models
tend to overestimate the DOD for at the beginning of the month, while the dust
episodes in mid-August and at the end of the month are better represented with
various degrees of skills.

The MACC-II Evaluation

In the evaluation of the MACC-II forecasts, the DOD at 550 nm wavelength forecast
by the MACC model are drawn together with the AERONET and MODIS retrievals
of AOD in monthly charts for selected stations, as well as the contributions of
other aerosol types. However, the scoring metrics that are calculated on a monthly
averaged time frame for different regions or stations are always calculated for the