Information Technology Reference
In-Depth Information
4.2 Empirical Assessment
In order to evaluate the performance of the security oracle on regular web ap-
plications, we tested it on a real world case study written in PHP, Yapig version
0.95b. Yapig is an open source web application that implements an image gallery
management system, allowing users to publish and comment pictures organized
in galleries. It consists of 9,113 lines of code and 53 source files, with 160 user-
defined functions and 2,638 branches.
Tabl e 2.
Test cases automatically generated for Yapig
Vulnerability
Generated test cases Used test cases
Safe tests Candidate attacks Real attacks Safe tests Attacks
1 1
600
10
10
100
10
1 2
420
4
4
40
4
2 1
163
155
155
160
16
2 2
68
19
0
0
0
2 3
14
0
0
0
0
2 4
175
19
19
170
17
2 5
52
7
7
50
5
2 6
299
0
0
0
0
Total
1,791
214
195
520
52
At first, we generated test cases using our tool [1,2]. Data about test case gen-
eration are reported on Table 2. The first column reports the name of the vul-
nerability, where the syntax
i j
is used to group together similar vulnerabilities.
In fact, vulnerabilities with the same
i
insist on the same sink statement (e.g.,
the same
echo
statement), but having different sequences of target branches, i.e.
different execution flows. All the vulnerabilities in Table 2 refer to
upload.php
page, which is responsible to handle the functionality of uploading pictures in
Yapig. Users may select pictures from their local hard drive and upload them on
the server by resorting to the capabilities offered by this page.
The second and third columns of Table 2 report respectively the number of
safe test cases and candidate attacks that has been automatically generated.
Eventually, the fourth column reports the number of real attacks that are still
available after manual filtering. As in the previous experiment, the test cases
have been selected to respect a 1:10 ratio between attacks and safe tests for each
vulnerability (fifth and sixth column).
A total of 2,005 distinct test cases has been generated. 1,791 of them are
safe by construction (no attack fragments have been injected), while 214 are
candidate attacks. After manual inspecting them, real attacks resulted to be
195. 520 safe executions and 52 code injection attacks have been considered in
total, 50% of them have been used for training and 50% for assessment. Tuning
of cost-factor has been performed by using the same procedure described in
Section 4.1. For two kernels (PTK and uPTK), two distinct configurations of the
cost factor gave symmetrical results (high precision/low recall and vice versa),
Search WWH ::
Custom Search