Information Technology Reference
In-Depth Information
Fig. 2.
Communication between tester and subject
Paris and Arts developed a network interface for QuickCheck that enables
it to check network protocols on remote machines. The basic process is shown
in Figure 2. They use two channels to interact with the system under test; one
channel is used to induce certain behaviour from the subject (invoking behaviour
by invoking functions with Erlang messages), and the other is the network-level
channel, which is used to inspect the actual packets that are sent over the network
as a result, and to send packets over the network to the subject. The model on
the tester machine contains certain constraints that the messages are expected
to adhere to, and every time a message is received, it is checked against those
constraints. The model then uses the information on these packets to generate
valid replies and transition the subject stack through the state machine. Using
these two channels enables black box testing of the subject by observing and
steering the behaviour using the two interfaces the stack has: to the user through
the API (managed through the subject controller), and to the network (using
the network channel).
Paris and Arts have used this framework, along with a QuickCheck specifica-
tion that they have manually generated from the TCP RFC [23]. They have used
this as the basis for testing a number of TCP stacks, including the Linux kernel
release, as well as a specialised Erlang TCP implementation. As mentioned pre-
viously, the state machine is not complete; there are some legal transitions which
are not explicitly stated in the graph. There are also many self transitions which
are not represented in the graph, but are referred to in the natural language of
the RFC. Therein lies the rationale for applying the approach presented here;
we want to find the real specification, and not merely an idealized one, so that
we have a more authoritative basis for understanding how the system works.
4.1 Results
Any testing technique for TCP implementations is necessarily limited by the
time taken waiting for responses for the server. Some of the operations could take
several minutes to execute (mainly due to waiting for time-outs, or packets that
may have been lost on the network). Consequently, the process of collecting tests
was a very time-consuming process. For this reason, the process was limited to
