Hardware Reference
In-Depth Information
Fig. 3.5
Algorithms
performance comparison on
the reduced benchmark
problem
18
16
14
12
10
8
6
Es
MFGA
MOGA-II
MOSA
NSGA-II
MOPSO
APRS
4
2
2%
5%
10%
20%
30%
40%
50%
Percentage of evaluated points
solutions found by each algorithm, a set of four metrics has been selected considering
the following criteria:
Accuracy: measured as the distance between the obtained Pareto front and the
reference (or real) front.
Uniformity: measured as the distribution of the solution set in the trade-off curve.
Extent: measured as the coverage of the objective space considering boundary
points.
The D-metric,
-metric and
-metric have been selected from [
5
], while the ADRS
(Average Distance from Reference Set) metric has been selected from [
15
]. Both
D-metric and ADRS provide indication of accuracy,
-metric of uniformity and
∇
∇
-metric of extent.
A fair evaluation of non-deterministic algorithms requires several repeated runs
without changing any parameter besides the random generator seed. Notwithstanding
the relative small search space consisting of only 9,216 designs, very large variations
can be observed in the algorithms behavior and a rigorous study needs to analyze also
this aspect. It was proved that 10 repetitions were a good trade off among statistical
issues, purposes of the evaluation and significance of the problem. Preliminary tests
were performed in order to estimate the best choices for the tunable parameters which
then have been kept fixed.
Algorithms parameters are usually problem-dependent. Some of them depend
also on the user expectations: the optimal choices (if any) for parameters controlling
the ratio between exploration and exploitation phase (like temperature schedule in
MOSA, for example) are strictly related to the number of evaluations the user can
afford. It was decided to tune these parameters considering the largest target (i.e.
50% of the design space, as described below) and accepting possible worse results
in the partial samples.
The evaluation process then proceeds checking at fixed numbers of explored
points the quality of the non-dominated front found so far. The steps selected for the
Search WWH ::
Custom Search