Elements of Difficulty Level in Mathematics* - Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing - page 84

2.2 Subjective Methods
The method making decision subjectivity is commonly used. But, the value is not
possible that there is credibility because there is no way to find out whether the
value is correct or not. We have built educational system using this methods
(Jiromaru 2012)
If a human decide difficulty level subjectivity, these stages may be reasonable.
But, there are 2 problems. One is accuracy, another is resolution. These points
have the potential of decreasing the resolution of test.
2.2.1 Accuracy of Difficulty Level
AHP is one of the theory took account of subjective accuracy. AHP is based on
mathematics and psychology. AHP uses subjective methods and analyze mathe-
matically. On the other hand, making decision subjectivity in mathematics don't
analyze mathematically. So, why don't we use AHP for making decision subjec-
tivity in mathematics? Its answer is the way of decision making in AHP.
Fig. 3 A simple AHP hierarchy9
AHP (The Analytic Hierarchy Process) is a structured technique for dealing
with complex decisions. Steps of AHP are as follow. At first, deciders break prob-
lem down into “goal”, “criterions” and “alternatives”. In particular, model of AHP
represent hierarchy diagram like fig. 3.
Goal which is top layer is only one because of an end destination. Criterions set
under goal, and alternatives set under criterions.
Next, decider calculates comparative importance between elements about each
layer of criterions and alternatives. We use “pairwise comparisons” for collecting
basic data. Pairwise comparisons aren't the method which determines to evaluate
many elements all at once. It is the method which determines relative merits to
take one pair methods.
Decider calculates proper vector and eigenvalue, and calculates Consistency
Index (CI) using eigenvalue, and calculates importance of element using proper
vector. If decider finishes calculating importance of all elements, decider can get
final assessment of alternatives.