Information Technology Reference
In-Depth Information
exactly predict real human interest with a single objective index. However, their
work never showed any concrete method to predict human evaluations with these
objective indices.
6.3
Rule Evaluation Support with Rule Evaluation Model
Based on Objective Indices
In practical data mining situations, a human expert repeatedly performs costly
rule evaluation procedures. In these situations, the useful experiences gained
from each evaluation, such as focused attributes, interesting combinations, and
valuable facts, is not explicitly used by any rule selection system, but is tacitly
stored in the mind of the human expert. For these problems, we have suggested
a method using rule evaluation models based on objective rule evaluation indices
as a way to explicitly describe the criteria of a human expert, thus re-using the
human evaluations.
6.3.1
Constructing a Rule Evaluation Model
We considered the process of modeling the rule evaluations of human experts as
the process to clarify the relationships between the human evaluations and the
features of input if-then rules. Based on this consideration, we decided that the
rule evaluation model construction process could be implemented as a learning
task. Fig. 6.1 shows this rule evaluation support method based on the re-use of
human evaluations and objective indices for each mined rule as a rule evaluation
model.
This method is iteratively carried out its training phase and its prediction phase.
In the training phase, the attributes of a meta-level training data set are
obtained by objective indices such as recall, precision and other rule evaluation
values. The human evaluations for each rule are combined as classes of each
instance. To obtain this data set, a human expert has to evaluate a part or
Training Phase:
Human Expert
Prediction Phase:
New Mined Rules
Mined Rules
Dataset for
Mining
New Dataset
for Mining
Calculating Objective
Rule Index Values
Evaluation labels
for each rule
Test dataset
for the rule
evaluation model
Calculating Objective
Rule Index Values
Training dataset
for a rule
evaluation model
Rule ID
Accuracy
Recall
...
Human Expert
Unknown
Unknown
Rule ID
Accuracy
Recall
...
Human Expert
new1
new2
0.75
0.88
0.24
0.56
...
...
1
2
0.85
0.90
0.14
0.65
...
...
Interesting
Not Interesting
...
new n
...
n
0.95
0.32
...
Unkown
0.59
0.01
...
Not Understandable
Learning Algorithm
Selection Scheme
Learning Algorithm
The Rule Evaluation Model
Display with
the predictions
Human
Expert
Predicting rule evaluations
using the learned rule evaluation
model
A Rule Evaluation Model
Fig. 6.1.
Overview of the rule evaluation support method using rule evaluation models
