Java Reference
In-Depth Information
Number of Event Tokens: 4
Number of Outcomes: 2
Number of Predicates: 4
...done.
Computing model parameters ...
Performing 100 iterations.
1: ...loglikelihood=-15.942385152878742
0.8695652173913043
2: ...loglikelihood=-9.223608340603953
0.8695652173913043
3: ...loglikelihood=-8.222154969329086
0.8695652173913043
4: ...loglikelihood=-7.885816898591612
0.8695652173913043
5: ...loglikelihood=-7.674336804488621
0.8695652173913043
6: ...loglikelihood=-7.494512270303332
0.8695652173913043
Dropped event T:[p=23.6, s=,, p1=6, p1_num, p2=., p2_eos,
p21=.6, p1f1=6,, f1=,, f2=bok]
7: ...loglikelihood=-7.327098298508153
0.8695652173913043
8: ...loglikelihood=-7.1676028756216965
0.8695652173913043
9: ...loglikelihood=-7.014728408489079
0.8695652173913043
...
100: ...loglikelihood=-2.3177060257465376 1.0
We can use the model as shown in the following sequence. This is the same technique we
used in the section
Using the TokenizerME class
. The only difference is the model used
here:
try {
paragraph = "A demonstration of how to train a
tokenizer.";
InputStream modelIn = new FileInputStream(new File(
".", "mymodel.bin"));
TokenizerModel model = new TokenizerModel(modelIn);
